What separates an assistant that impresses in a demo from one that survives two hundred users.
Most teams pick a vector store from a benchmark chart and then discover the thing that hurts is metadata filtering, tenancy or reindexing. If you already run PostgreSQL, pgvector is usually the right answer for longer than people expect.
The cost per token looks obviously cheaper until you account for utilisation. GPUs bill by the hour whether or not anyone is asking questions, and that single fact decides most of these decisions.
A classifier on every request costs time and money on every request. What to check on input, what to check on output, and how to decide whether the guardrail is worth what it takes.
A hundred real cases, per-task metrics instead of one score, and a judge calibrated against human labels. That is enough to catch regressions and to decide whether a model change is safe.
An approval step is the cheapest safety control and the easiest one to place badly. Put it too early and you have automated nothing; too late and you are rubber-stamping. Four placements and when each is right.
Fine-tuning teaches behaviour and format. Retrieval supplies knowledge. Prompting shapes the immediate task. Most teams reach for the expensive one to fix a problem it cannot fix.
Teams upgrade the generation model to fix a retrieval problem. The chunk boundaries, what context travels with each chunk, the embedding model and a reranker account for most of the quality gap.
Traditional monitoring tells you the workflow succeeded. With a model in the middle, succeeding and being right are different things. What to capture, and the three dashboards worth building.
The Model Context Protocol turns an internal API into something an agent can call. The protocol is the easy part; the design decisions are which operations to expose, at what granularity, and under whose identity.
Teams argue about which model to use and then ship an assistant that answers confidently from the wrong document. The decisions that actually decide quality are about chunks, permissions, evaluation and cost.
Most of the friction people blame on the model is a missing paragraph in the repository. What belongs in a context file, what does not, and why the ones that grow past a page stop working.
The bottleneck moved from writing to reviewing, and the old review habits do not scale to it. Four gates that keep quality without turning one senior engineer into a queue.
One keeps you in the loop at every keystroke, the other takes a task away and brings back a diff. Most teams that argue about which to standardise on end up using both, for reasons that are easy to state.
A coding agent with shell access and repo context is a different tool from an autocomplete. The gains are real and they land unevenly; the cost is that review becomes the bottleneck. What we changed after six months.
An open-source assistant that lives on your own server, talks to you through the messaging app you already use and runs tools on your behalf. Genuinely useful, and the most over-permissioned thing most people will ever install.
The model is rarely the expensive part. Platform operations, the APIs you call, the retries and the human who checks the output usually add up to more. A worked breakdown of one real invoice-processing flow.
Automations fail on a Tuesday at 3am and nobody notices until the customer calls. Four mechanisms — idempotency keys, bounded retries, a dead letter queue and an alert that names the business event — fix most of it.
An AI node in a workflow is a non-deterministic step in a deterministic pipeline. That is fine for classification and drafting, and dangerous for routing and writes. Here is where we draw the line.
In Make, every module run is money, so the cheapest scenario and the fastest scenario are usually the same one. Five patterns that cut consumption by half without changing what the automation does.
The three tools do the same demo. They diverge on pricing shape, where your data sits, and what happens when a workflow needs real code. Pick on those, not on the connector count.
Knowledge Bases, Guardrails and Agents remove a lot of plumbing. They do not remove the retrieval, permission and evaluation decisions, and that is where projects still fail.
n8n on a single container is a demo. Running it for a company means queue mode, a real database, a plan for credentials and a restore you have tested. Here is the shape that survives.
Whatever platform draws the boxes, the automations that stay up have the same three pieces underneath. Here is the shape, and the five decisions that make it hold under load.
A prompt is production configuration that changes behaviour silently. Treat it like code, and build the smallest evaluation that can tell you whether a change made things better.
An agent that clicks through a web interface can automate systems that have no API at all. It is also the most fragile automation you can build. Where the trade actually lands, and how to make the fragile version survive.
Not a strategy deck. A sequence: find the work, ship one thing that matters, build the platform underneath it, then let other people build. What goes wrong at each stage and how to tell if it is working.
Vertex gives you a managed vector search, a grounding API and an evaluation service. Two of those are worth taking as-is. Here is how we assemble a production assistant on Google Cloud.
The model is the easy part. What decides whether an Azure OpenAI assistant reaches production is token quota per region, private networking and whether retrieval respects who is asking.
Oracle's managed model service covers the plumbing for a standard assistant, including a vector-capable database you probably already have. Here is the architecture and the decisions it does not make for you.