What I learned building an AI product from scratch

Most people talk about AI. I spent the last three years actually building with it — not advising on it from a slide deck, not running a strategy workshop, but shipping a real product that real people use.

In 2023 I co-founded Gaite AI: a platform that helps people navigate the big life events most of us face — paying for college, caring for an aging parent, buying a house — by pairing structured guidance with generative AI and agents. We're in pilot with 1,200 users across employer partnerships. I built the RAG pipeline, designed 45 life-event frameworks, and handled most of the product end-to-end.

Here's what I learned that I didn't expect.

RAG is harder than the demos suggest

Every AI demo you've ever seen makes retrieval-augmented generation look trivial. Upload some documents, ask a question, get a magical answer. In production, none of it is trivial.

You have to think hard about chunking. About whether your embeddings actually represent what you think they represent. About how to handle documents that contradict each other. About what happens when the user asks a question that's adjacent to your corpus but not in it. About when to retrieve and when not to. About re-ranking. About hallucination guardrails.

The "boring infrastructure" around the model is the product. The model is just the engine.

Frameworks matter more than the model you pick

The model you use today will be irrelevant in six months. GPT-4 became GPT-4o became GPT-5 in less time than it takes most enterprises to approve a vendor.

What persists is the product structure around the model. Our 45 life-event frameworks — the schemas, the decision points, the data we know to ask for at each stage — are what makes Gaite useful. Swap the underlying model and we'd still be useful. Strip the frameworks and we'd just be ChatGPT with a logo.

This is the thing I'd tell any AI product leader: spend less time arguing about which model and more time on the structure that makes the model do something specific.

Users don't care about your tech stack

Not one pilot user has asked me whether we use OpenAI or Anthropic. Not one has asked about our embedding model. They ask whether the thing works, whether they trust the answer, whether they got what they needed.

The amount of LinkedIn content that treats the AI stack as the product is wild. It's not. The product is what the user can do.

The boring stuff is most of the work

Auth. Data import. State management. Error handling for when the model returns something weird. Onboarding. Pricing. Permissions. Audit logs. Compliance. Customer support tooling.

By volume, easily 80% of building Gaite has been the work I'd have been doing in any SaaS product. The AI part is the most interesting 20%, but it's still 20%.

If you're building an AI product and you don't enjoy the boring 80%, you'll fail at the interesting 20%.

The most underrated skill is translation

The work I do that almost nobody else can: sit with an engineer talking about embedding drift in the morning, then walk into a sales call and explain ROI to a CFO in the afternoon. Both honestly. Neither dumbed-down for the other.

Most AI projects don't fail because the tech doesn't work. They fail because the people building it can't communicate with the people funding it, and vice versa. Translation is the actual skill that's in short supply.

If you're building something interesting in this space, I'd love to hear about it.