Medha

Every LLM-powered app I'd built up to this point followed the same recipe (pun intended): call an API, write a good prompt, wrap it in a nice UI. That's a legitimate way to build things, but at some point I wanted to actually understand what was happening inside the model I was calling and not just how to prompt one.

So for my recipe app Rasaveda, I decided to skip the API entirely. Intially, I had one made, but then I felt like I was not making any clear progress in actual machine building. So I ditched the entire external API callings. No OpenAI, no HuggingFace inference endpoint, no pretrained weights. I wrote a decoder-only transformer from scratch in PyTorch, trained it on a single Colab T4, and shipped it as the actual language model powering the app in production.

This post is a lazy attempt at what that looked like. The architecture, the training runs, the mistakes, and what I'd tell someone about to try the same thing (do at your own risk).

What Rasaveda actually does

Rasaveda is a full-stack recipe intelligence app: you give it the ingredients sitting in your kitchen, it does a semantic vector search (ChromaDB + all-MiniLM-L6-v2) over 365 recipes to find the best matches, tells you exactly what you're missing, and can critique or explain any cooking step conversationally. It also has a somewhat unnecessary but delightful feature where you pick a theme by clicking one of 36 Indian states on a geographically accurate SVG map (original idea lol).

The part I actually want to talk about is RasavedaGPT, the model that generates every word of AI output in the app, running in-process inside the FastAPI backend.

Why build the model instead of calling one

Two reasons, one practical and one selfish.

The practical one: I wanted a fully self-contained, dependency-free inference path without any API keys, no rate limits, no per-token cost, no vendor to go down at 2am. For a small, domain-specific task like "reason about recipes," a giant general-purpose model is overkill anyway.

The selfish one: I wanted to actually build a transformer with my own hands. I started this project to actually learn some bits of machine learning anyway, the attention, positional embeddings, the training loop, the tokenizer instead of just always sitting one abstraction layer above it. If you've only ever fine-tuned or prompted models, there's a specific kind of understanding you only get from watching your own loss curve fail to go down and having to figure out why.

The architecture

RasavedaGPT is a small decoder-only transformer and architecturally a tiny GPT, nothing exotic:

Hyperparameter Value
Total parameters 6,392,320
Vocabulary size 6,000 (custom BPE)
Context length 512 tokens
Embedding dimension 256
Attention heads 8
Transformer layers 6
Feed-forward dim 1,024 (4× d_model)

At 6.4M parameters, this is small enough to run inference on CPU comfortably inside a FastAPI request without any GPU needed in production. That size wasn't an accident: the task is narrow (recipes, not general reasoning), so I sized the model to the problem instead of defaulting to "bigger is safer."

Training in two stages

I trained in two passes rather than fine-tuning directly on recipe data from a randomly-initialized model, because a model that's never seen coherent English at all struggles to learn a narrow task and fluency at the same time.

Stage 1: pre-training on WikiText-2, 3 epochs with a cosine LR schedule, just to teach the model what language looks like at all:

Epoch Loss Perplexity
1 5.844 345.3
2 4.995 147.7
3 4.739 114.3

Stage 2: fine-tuning on recipe tasks, 12 epochs over 2,139 examples (repeated 8× per epoch, ~17,112 examples/epoch):

Epoch Loss
1 2.439
3 0.787
6 0.404
9 0.279
12 0.236

Both stages together ran in about 40 minutes on a single Colab T4. That's the part that still surprises me, you don't need a cluster to get a model that's genuinely useful, as long as you've scoped the task tightly.

Task tokens instead of prompt templates

Since I control the training data, I didn't need to coax behavior out of the model with elaborate prompt engineering. Instead I trained in three explicit task tokens directly into the vocabulary:

  • [RECOMMEND] → structured JSON recommendations for the ingredient-matching page
  • [IMPROVE] → step-by-step JSON critique of a recipe's cooking technique
  • [CHAT] → natural-language conversational answers, grounded in ChromaDB retrieval context

At inference time, the backend just prefixes the input with the right token and the model already knows which "mode" to respond in and what output shape to produce. It's a small thing, but it's a nice reminder that a lot of what prompt engineering does for a general-purpose LLM, you can just train directly into a specialized one.

What actually went wrong

  • The first fine-tuning run overfit hard because I under-repeated the dataset (2,139 examples is not a lot), and one pass per epoch wasn't enough signal for the model to generalize the JSON output structure reliably. Repeating each epoch's data 8× (with shuffling) fixed it.
  • Vocabulary size mattered more than I expected. I started with a larger vocab "for safety" and got worse convergence on a dataset this small. I learned that a 6,000-token BPE vocab tuned specifically to WikiText-2 + the recipe corpus outperformed a generic larger one.
  • JSON output from a from-scratch model is genuinely fragile. The [RECOMMEND] and [IMPROVE] tokens needed real supervision on exact output formatting, not just "here's roughly what good output looks like". Small models don't have the slack to infer structure you didn't explicitly show them.

Would I do this again?

For a narrow, well-defined task with a dataset I control, yes definitely, without hesitation. The entire training loop, tokenizer, and model file are a few hundred lines of PyTorch I understand completely, which is worth a lot when something breaks in production at 11pm.

Would I do this for a general-purpose assistant? Nope. That's what pretrained foundation models are for, and reinventing that wheel doesn't teach you anything a good research paper wouldn't. But for a well-scoped, well-understood domain, training small is underrated. It's cheap, it's fast, it's fully yours, and you come out the other side actually understanding the thing you shipped.