We built an assistant
on a zero budget

There was no specification at the start. There was a link to a conversation, and I said let us turn what is discussed here into an assistant app. That was it.

It is not a bad start, really. A specification tells you what the product is. A conversation shows you what people actually ask. And the assistant’s job is not to describe the product, it is to answer the question.

In the first version I wired every question to the brand’s knowledge base. If something was not in there, it said “I don’t have that information.”

Then I asked this: for patients with hypertension and diabetes, which probiotic strains matter more, and which prebiotics suit them best? It could not answer. But the brand’s product is only one example inside that question. If you pull the assistant’s limit down to the product, what you have is not an assistant, it is a catalogue.

So we split the scope in two.

The brand’s own information. Product positioning, what can be claimed, what cannot. This part is binding. If the information is not there, it says so, and it does not make anything up.

The scientific side. Strains, doses, studies. Here it does not stay quiet. But it is not free either. It writes the strain code in full (genus, species, strain), puts the type of evidence next to every claim (meta-analysis, randomised trial, observational, animal, in vitro), and never skips a drug interaction warning.

It is bad when an assistant says “I don’t know.” It is worse when it acts like it knows without naming a source.

I asked how I could run this for free. We stopped in two places looking for the answer.

First we tried installing a model on my own machine. It worked, but the app runs on a server, not on my machine. So it is good while you are building, useless once you ship.

Then we moved to free cloud models. These do the job, but you pay for them in uncertainty rather than money. Requests join a queue, the model sometimes disappears, an answer sometimes takes minutes.

Here is what I took from it. Deciding to stay free is not a preference you improve later. It is the thing that shapes the architecture from day one. The two sections below are results of it.

Free models answer somewhere between one and a half and five and a half minutes. On Vercel’s free plan, functions get cut off at sixty seconds.

So the connection was closing before the model could finish. The error on the screen looked like a model problem, but the problem was the limit itself. We moved the app somewhere that allows long requests (Render). A free model being slow is not a defect, it is a known cost. Wherever you host it has to be able to take that.

I was not the only helper touching the code. There was a second agent running inside the editor, wired to a cheap and fast model. I wrote it a long rules file. What it would do, what it must never do, all of it explained with reasons.

Then I looked and saw it had written clinical figures with no source into the knowledge folder. The rules file did say “do not invent studies, numbers or strain codes.” The small model did not see that line among all the reasoning.

We did three things. We deleted the file. The project had not been in a git repository until that day, so we set one up, because the real problem was not being able to see what the agent changed. And we rewrote the rules file. We cut the reasoning short, wrote the prohibitions like commands, and removed write access to the knowledge folder entirely.

Small, fast models skip long explanations and follow short, clear prohibitions. A rules file is not a persuasive text, it is a list of boundaries.

I did not make this mistake, but I paid for it, so I am writing it down.

A provider’s name came up in conversation. I repeated the sentence. The other side took that as “let us do this” and changed the whole configuration. The next day I found myself asking why we had opened an account with that provider Correction. And I was right to ask. I had not approved it, I had only repeated what was said.

Undoing it was not hard, but we lost hours. The rule that came out of it applies to me as much as to the other side. Repeating a name is not approval. On decisions that are a hassle to reverse, one clear sentence of approval is cheaper than a half-hour rollback later.

Today the app started timing out. There was nothing wrong with the code. The free model tier was busy, requests were waiting in a queue, and now and then the model did not answer at all.

The easy fix was right there. The paid version of the same model is around $0.003 a question, so a few dollars a month. We did not take it. The account deliberately sits at zero balance. On this project, being free is not a goal, it is a line in the brief.

So we looked for the fix in resilience rather than in budget. We lined up several free models one after another. If one gets queued or disappears, the request moves to the next. We defined the timeout limits inside the app, and we do not throw away a half-finished answer.

The system is still slow. But the slowness is no longer an error, it is expected. The difference shows up in what you tell the user. Saying “an error occurred” and saying “free models are busy right now, your request is in line” make the same delay look completely different.

One. Say the constraint at the very start. “This has to be free” is an architectural decision on day one and a rollback job on day four.

Two. Do not try to persuade an AI in its rules file. Write it short, write it like a command, list the prohibitions. Put the reasoning somewhere else.

Three. Repeating a sentence is not approval. Whether the other side is a person or a model, ask for clear approval on anything expensive to reverse.