Intro

Welcome! This is the first of hopefully many posts on my blog. Not big on ice-breakers but wanted to briefly acknowledge some prior art or inspo that drove me to start this site. 1) this post (opens in a new tab) by Jacob Thornton many years ago, I first read it when I had discovered web development and the linguistic style was really special - I instantly knew I wanted to write about my process like this some day. 2) All tech twitter banter, the platforms have too much viscosity for my taste now but some tweets will permanently reside in my brain rent-free somewhere. In these posts, I try my best to verify and avoiding the spread of misinformation about technology, if something's off please don't hesitate to email me corrections.

Topic:

RFCINDEX

Tldr:

This is my newest fad, a redesign attempt of datatracker (opens in a new tab). Unlike the original datatracker, RFCINDEX provides LLM functionality + "attempts" to enhance the datatracker experience with various HCI patterns I found in the wild. In no way affiliated with ietf, nor attempting to replace or compete with the original. I felt like building something so this is what I concocted. In the nascent stages of development, the platform doen't do anything bleeding edge with LLMs, basic contextual prompt-engineering is probably the most complex feature. I am however trying to scaffold this so that its able to evolve over time, more specifically I plan on conducting evaluations of LLM use cases or attempt implementing ideas coined in published research in a feature-toggle manner. If all goes well, then 1/3/5+ years from now, this'll be a platform that helps achieve the things outlined here (opens in a new tab), or it gets tucked away and forgotten gracefully. TBD.

Overview:

This project initially came to mind when I first learned what an LLM is (2022 gpt press). I resisted shiny object syndrome until the topic engulfed all media I consume. After enough fomo, I decided to bite. Upon surveying what people have built using gen ai products I saw the vast array of use cases in tooling such as Langchain, plethora of use cases in OpenAI's official cookbook, but I didn't find anything that operates on RFCs. A few things I hope to eventually answer: how might we enrich the consumption of RFC documents using frugally innovative applications of LLMs? how can this be juxtaposed with popular HCI patterns found in various platforms that exist today? This document outlines the constituent parts of the project divided into the following sections: frontend, backend, core. In nascent stages of development, I focus on completing the circuit, the basics of an initial rapid prototype - searching for RFCs, factual QA on the RFC in focus, operational crud. In future posts I'll deep dive on certain aspects of the frontend and backend to explain design decisions and I'll follow up with dedicated posts whenever I explore any new LLM use cases or major feature additions.

Frontend:

This platform is built using React, minimal dependencies, and an emphasis on alignment with an OpenAPI Spec that (will be discussed in future posts) ideally will glues all constituent parts of RFCINDEX. Some notable features I wanted to point out are 1) Multi-LLM QA Response fusion 2) the omni search for finding RFCs 3) GitHub auth for idempotent, offline-first pii linking (your github id and all your interactions with the QA features are mapped one-to-one and stored locally). The repository (opens in a new tab) is scaffolded using vite and open for scrutiny / suggestions. Please consider consulting the official datatracker (opens in a new tab) repo for notarized development efforts surrounding RFC tooling/platforms. This project isn't that formal or designed with adoption at production scale (not yet at least).

Backend:

The backend is built using FastAPI, OpenAI API, and deployed via AWS EC2. I chose EC2 because having the flexibility to design the overall platform without conforming to vendor specific implementation details made the most sense at the time. Additionally, its really simple to terraform the system however we see fit if there's just a running container that can harbor an API, a plethora of frugal data persistence options (a collocated sqlite db), apache airflow etl that operates on csv, or quite literally anything that can operate inside a linux/ubuntu environment. I do plan on scaffolding everything into a monorepo such that it can be taken out of ec2 and deployed else where or disassembled / containerized atomically. No sophisticated design patterns or structural style is present in current state, and the core of the backend can be summarized as a collection of controllers and enpoints that interface with OpenAI API. The repo (opens in a new tab) is currently incrementally being expanded to support multi-llm response fusion + use case evaluations. In future posts I'll lead with what that means for the system as a whole.

Core:

This is the nucleas of RFCINDEX in many ways. I define the "Core" as the subset of functionality facilitating central RFCINDEX features. For example, controllers that delegate QA inputs and route them to (for now just OpenAPI) but will be expanded such that the following can be accomplished:

(stay tuned for pretty diagrams in the future)

  Scenario 1.
      question -> (... layers of stuff ...) -> a singular
      llm tasked with generating responses

  Scenario 2.
      question -> (... layers of stuff ...) -> a collection
      of llms tasked with generating several responses -> (...
      layers of stuff ...) -> all llm responses fused together
      to provide a single response -> response in synopsis 
      format

Preview:

QA demo:

Omnisearch demo:

QA future state: