FinBot · Resume Guide

"FinBot is a domain-specific banking FAQ chatbot built directly on the OpenAI Chat Completions API — no LangChain, just the raw API. I started here intentionally because I wanted to understand how LLM conversations work at the foundation level before using abstractions.

The core is a structured system prompt that constrains the bot to banking topics, defines 9 intent categories, and enforces JSON output format. Every response comes back as JSON with intent, answer, follow-up suggestions, and an escalation flag for complex queries.

Multi-turn memory is handled manually — I store conversation history in a session dict and pass it back with every API call. History is trimmed to the last 10 turns to stay within token limits. I exposed this through FastAPI with rate limiting via slowapi and Pydantic validation for all endpoints."

"It doesn't — the OpenAI API is completely stateless. Every API call is independent, the model remembers nothing between calls.

Multi-turn conversation works by passing the entire conversation history in every request. The messages array contains: the system message, then alternating user and assistant messages from the conversation so far, then the new user message. The model sees all of this and responds in context.

In FinBot, I store conversation history in a server-side Python dict keyed by UUID session IDs. On every POST /chat call, I retrieve the session history, append the new user message, send the full array to OpenAI, get the response, and append the assistant's reply to the session history for next time."

"Zero-shot classification means classifying text into categories using only the category names — no labelled training examples needed. GPT-4o understands what 'loan_eligibility' or 'interest_rates' means semantically, so it can classify queries into those categories just by knowing the names.

I used it in FinBot by embedding the category list in the system prompt: 'Classify the user's question into one of these categories: loan_eligibility, interest_rates, documentation_required...' GPT-4o classifies the query AND generates the answer in the same API call.

Why not train a classifier? Three reasons: I had no labelled training data. A separate classifier model means separate infrastructure to maintain. And zero-shot through GPT-4o works well enough — the category names are self-explanatory. If I needed higher accuracy and had thousands of labelled examples, I'd use few-shot or fine-tuning."

"Structured JSON means downstream systems — web frontends, mobile apps, analytics pipelines — can parse the response reliably without screen-scraping text. Each response has a defined schema: intent, answer, follow_up_suggestions, escalate_to_human, confidence.

Enforcement: I use response_format={"type": "json_object"} in the API call. This is OpenAI's native JSON mode — it guarantees the response is valid parseable JSON. Without this, the model sometimes adds markdown code fences or preamble text which breaks json.loads().

I also wrote a _parse_response() fallback function — if for any reason JSON parsing fails, it catches the exception and returns the raw text as the answer rather than crashing. Production systems need graceful degradation."

"I used slowapi — a rate limiting library for FastAPI. It integrates via a decorator: @limiter.limit('20/minute') on the chat endpoint. The key function get_remote_address identifies clients by IP address, so each IP gets 20 requests per minute independently.

Why it's critical: every POST /chat call costs money — OpenAI charges per token. Without rate limiting, a single script running in a loop could send thousands of requests per minute, draining API credits completely. Also prevents denial-of-service attacks on the service itself.

If the rate limit is exceeded, slowapi returns a 429 status with a retry-after header — the client knows exactly when to retry. I kept 20/minute as a reasonable limit for an internal banking tool where usage is deliberate, not automated."

"Without condensation, follow-up questions break intent classification. If a user asks 'What documents are needed for a home loan?' and then follows with 'What about for NRIs?', the second message alone gives GPT-4o no context to classify. It might classify it as 'general_banking' when it should be 'documentation_required'.

Condensation solves this: before classification and answering, I send the last 3 turns of chat history + the new question to the LLM with a separate prompt asking it to rewrite the question as standalone. 'What about for NRIs?' becomes 'What documents are required for NRI applicants to apply for a home loan?' — then THAT gets classified and answered accurately.

This adds one extra API call for follow-up questions, which costs a small amount of tokens. But the accuracy improvement is worth it, and the condensation call uses max_tokens=150 so it's cheap."

"Fundamentally different problem, fundamentally different architecture.

FinBot — answers from knowledge baked into GPT-4o's training. The banking FAQs it handles (loan types, general documentation requirements, account features) are well-known facts that don't change frequently. No external documents needed. Good for general, stable knowledge.

FinDoc QA — answers from your specific documents. When you need to answer from a 200-page internal policy document, RBI circular from last month, or your bank's proprietary loan handbook — that content isn't in GPT-4o's training data. RAG retrieves from your documents at query time.

Building FinBot first gave me a deep understanding of how the Chat Completions API works — message history, prompt design, structured output — which made building FinDoc QA's LangChain integration much more intuitive. The progression was deliberate."