Why we built a Content Lab

On 2 November I ran the first evaluation set on YuktiBOS. It is the AI business system we are building. The posts cost about Rs 5 each. I called them pathetic.

There was no sequence. There was no story and no insight. The advice was from 2020. Some of it contradicted what I actually believe about hiring agencies. The articles stopped at a step called "review the research" and never got written. Titles were cut off everywhere, and posts had no title at all.

Then I pasted one template into ChatGPT, with a Jio telecom case study as the topic. It wrote something far better. It had a narrative, real numbers from documents it had read (an annual report, a government survey, a regulator's data), honest scores, a "what changed" comparison and an FAQ.

So the plain chat tool, with no context, was beating the product I was building. That is a bad place to be. This post is about what we did next.

Why a lab and not another prompt tweak

My first instinct was to fix prompts one at a time. The problem is that I could not tell whether a change helped. I was reading three or four pieces, deciding by feel, and changing something else the next day.

So we built the Content Lab. It is an admin page where one brief can be run through many "variants" and many AI models, and every piece is scored and stored. A variant is a way of building the writing job. For example: a plain model with one line of instruction; a "human brief" assembled by code and handed to one writer; an "editor's brief" with a revision; the same brief run three times with a judge picking the best.

The idea in my own words: models are the copywriters, the orchestration is the company process, and the prompts are the instructions to each employee. The judge answers one question: do we like it, and would the market?

How we did it

The lab runs a matrix of client, variant, model and brief. The core brief set has nine formats: two LinkedIn posts, an Instagram post, an Instagram carousel, a Jio IPO case study, a landing page, a three-email sequence, a Google search ad and a Meta ad.

Each piece gets a scorecard. The first version (v6) weighted Engagement 35, Conversion 25, Craft 20 and Truth 20, with a hard gate on invented statistics or company claims. The verdict was Usable, Light edit or Not usable. We also compare every piece against real posts collected from LinkedIn, which is a separate story.

I also asked for the cost of every call to be recorded, so we can see cost next to quality.

What the early runs showed

For the first few runs I read the output myself and gave it a score out of 10. These are my reading scores, not measurements, and I treat them that way.

• Before any fixes: about 3 out of 10.
• After we cut the instruction pile and added craft cards: about 6.
• After the writer returned plain text and an "angle" step was added: about 7. One post, on a one-line value proposition, had a real story and a before-and-after line.
• A plain-chat baseline, no context: about 5. Fluent but generic. It quoted US salaries to an Indian audience.
• A model comparison with a judge that also compared against 17 real high-performing Indian LinkedIn posts: the judge averaged 6.0 for Opus, 5.0 for Sonnet, 4.5 for GPT-5.5 and 4.0 for Gemini 3.1 Pro (preview). Opus became the default writer.

The best post I read in that stretch began: "Hire the agency for the step you can already measure. Keep the step you can't explain yet." It had a concrete case and one clear test. I rated it about 8.

The big finding: we were drowning the writer

Doc 101 is the audit I asked for after I felt content was better before we added all the skills, flows and plays. Here is what it found.

• A post writer was carrying about 68,000 characters of skill text. An article writer carried about 75,000. That is roughly 17,000 to 19,000 tokens of rules to write 250 to 1,800 words.
• The checker, which ran on the cheapest model, held about 65,000 characters of skill text and was told to rewrite sentences "same meaning and length". That flattened the voice.
• The instructions contradicted themselves. Hashtags: "5 to 10" in one place, "at most three" in another. "Return only JSON" in one place, "give three hook variants" in another. A skill for one client's real-market article, 45,615 characters long, was wired to the social post writer.
• "Never invent" appeared five times across layers. The model wrote defensively.

Plays, which I had suspected, were not the cause. They had stopped being injected into writer prompts on about 19 October.

What moved the score most, in order: removing 65,000 to 75,000 characters of skill text and contradictions; plain-text writing plus examples instead of rule lists; and an angle step that names the obvious take so the writer can beat it.

What it changed in YuktiBOS

• The writer now gets a short brief. Compliance moved to a checker that runs afterwards and flags hard errors only.
• The writer returns plain text. A small step extracts hashtags and the like. We stopped asking one call to write and format.
• One writer per piece, orchestrated by code. Research and post-audit can run in parallel. We stopped chaining agents that rewrite each other.
• An angle step, a bank for the founder's point of view, and real examples instead of invented ones.
• Defaults decided after the human review (a separate post): Opus writes; posts use the human brief; articles use the editor's brief with best-of-3; Gemini does not write.

What I got wrong

I cut costs before I measured quality. I moved the brief step to the cheapest model to save money, and the most important step became the weakest. Quality first, then optimise.

I also let tests run without a safety plan. On 2 November the daily cap on one provider ran out twice from our own test runs (Rs 500 at about 08:30, then Rs 1,500 at 18:55), and the tests stopped. Pieces now pause and resume after the reset.

What is still open

• My own point of view bank was empty during these runs, so the angle step was guessing. It needs my real beliefs, written by me.
• The craft cards still use illustrative examples. They need three of my best real posts per platform.
• Quality is not at the level I want. A piece I would publish under my name with no edits is my 9.5 out of 10. We are not there.

Numbers at a glance