Bedrock in production: what the managed pieces do and do not solve

Bedrock is the fastest way to get a working assistant on AWS. A Knowledge Base pointed at an S3 bucket gives you chunking, embedding, a vector store and a retrieval API in an afternoon, with no model hosting and no GPU quota request.

That afternoon is real and it is worth taking. What it does not do is make the four hard decisions for you, and teams that assume it has are the ones still stuck in pilot six months later. We wrote about those decisions generally in the four that decide a RAG project; here is what they look like specifically on Bedrock.

What the managed pieces genuinely solve

Knowledge Bases handle ingestion, chunking, embedding and a managed vector store (OpenSearch Serverless by default, or Aurora pgvector, Pinecone, Redis). The retrieve-and-generate API returns citations with source URIs, which is the feature that most changes user trust. If your corpus is documents in S3 and your access model is "everyone who can use the assistant can see everything", this is genuinely most of the work done.

Guardrails give you content filters, denied topics, PII redaction and, importantly, contextual grounding checks that score whether the answer is supported by the retrieved passages. That grounding score is the cheapest hallucination control available and it applies independently of the model.

Provisioned throughput versus on-demand is a real cost decision, not a formality. On-demand is right until you have steady traffic; provisioned makes sense past a few hundred thousand tokens an hour and is the only way to get guaranteed capacity for a latency SLA.

What you still have to decide

Permissions. This is the one that sinks projects. A Knowledge Base retrieves from its whole index; if your documents have different audiences, you need metadata filtering — attach metadata to each document at ingestion and pass a filter at query time derived from the caller's identity. Building that filter from the caller's groups, keeping it in sync when someone changes team, and proving it at audit time is your work, not Bedrock's. Do not postpone it: retrofitting permissions onto a live index means re-ingesting everything.

Chunking for your documents. The default fixed-size chunking is fine for prose and poor for tables, contracts and anything with structure. Bedrock supports hierarchical and semantic chunking, plus a custom Lambda transform. For technical documentation and contracts we almost always end up with a custom transform that keeps section headers attached to each chunk, which improves retrieval more than any model change.

Retrieval quality and hybrid search. Vector-only retrieval misses exact terms: product codes, error numbers, names. Enable hybrid search on the OpenSearch Serverless backend so keyword matching runs alongside vectors. Then measure — with a set of fifty real questions and the correct source document for each, checked on every change.

Model choice per step. You are not obliged to use one model. A cheap fast model for query rewriting and classification, a strong model for the final answer, and possibly a third as a judge in your evaluation harness. Bedrock makes switching a parameter, which is the main argument for using it over a direct provider integration.

Cost, honestly

Three lines: tokens, the vector store and the embeddings. The vector store surprises people — OpenSearch Serverless has a minimum capacity floor that costs real money per month whether or not anyone asks a question. For small corpora, Aurora Serverless with pgvector is often cheaper. Price both before you commit; the difference on a small assistant can be larger than the token bill.

Track cost per question from day one, tagged by team. An assistant nobody can attribute cost to is an assistant that gets switched off in the next budget round.

Where it fits

Bedrock is the right default on AWS when your data is already there, your compliance story is easier if nothing leaves the account, and your team is small enough that managed plumbing is worth more than maximum control. When you need custom retrieval logic, a re-ranker you tune yourself, or a model that Bedrock does not host, the retrieval layer is the part you take back, not the whole stack — and that is how we usually build it in an applied AI engagement.

What to do this week

Write your fifty-question evaluation set with the expected source document for each, before you build anything. It takes a morning with a domain expert, it is the only thing that tells you whether a change helped, and it is the artefact every stalled RAG project turns out not to have.