OCI Generative AI: where it fits and what you build around it

If your data is already in Oracle Cloud, the argument for running the assistant there is the ordinary one: no egress, no second compliance conversation, and a single identity model. OCI's generative AI stack is smaller than the equivalents on the other clouds, and for a standard retrieval-augmented assistant that turns out not to matter much.

Here is what the pieces are and where the work actually is.

The pieces

OCI Generative AI is the managed inference service: hosted models available on demand, plus dedicated AI clusters when you need guaranteed capacity and predictable latency. The dedicated cluster model is worth understanding early because it is the difference between an SLA you can promise and one you cannot.

Generative AI Agents is the managed RAG service. Point it at a data source, and it handles ingestion, retrieval and grounded answers with citations. For a documentation assistant over content in Object Storage, this is most of the build.

23ai with AI Vector Search is the piece that changes the architecture. Oracle Database 23ai stores vectors as a native type and queries them in SQL alongside your relational data. If your operational data is already in Oracle, this removes an entire component: no separate vector database to run, sync and secure, and — more importantly — retrieval can join a vector similarity search against a permissions table or a business filter in one query.

That last property is the strongest technical argument in the stack. The common failure in RAG systems is that the vector store knows nothing about your permission model. A vector index inside the database that already holds your access control lists does not have that problem.

Select AI lets you query your data in natural language through the database, generating SQL. Useful for analyst-facing tools, and a completely different risk profile from a document assistant — it needs its own read-only user, its own row-level security and a hard limit on what it can touch.

What you still decide

The managed service removes plumbing and not judgement. The four decisions from what makes or breaks a RAG project all remain:

Chunking for your document types. Default chunking handles prose and mangles tables and contracts. If your corpus is structured, expect to write your own ingestion step that keeps section headings attached to chunks.

Permissions at retrieval time. If different users should see different documents, the filter has to be applied in the retrieval query, not after. With 23ai this is a WHERE clause against your existing authorisation data, which is the easiest version of this problem available anywhere. With Object Storage sources, it is metadata filtering and you own keeping it in sync.

Hybrid retrieval. Vector-only search misses exact identifiers — part numbers, error codes, customer references. Combine it with a keyword search; in 23ai that is a text index alongside the vector index, in one query.

Evaluation. Fifty to two hundred real questions with the correct source for each, run on every change. Nothing in the managed service writes this for you and nothing else tells you whether a change helped.

Identity and cost

Use resource principals and dynamic groups so the application authenticates to the Generative AI service without any credential — the pattern from OCI policies and dynamic groups. Scope the policy to the specific compartment.

On cost: on-demand inference for pilots and variable load, dedicated AI clusters once you have steady volume and a latency requirement. Track cost per question labelled by team from the first week, cache embeddings for unchanged content, and use a small model for query rewriting and classification. The economics are the same everywhere; what differs on OCI is that the egress line is smaller, which matters when the assistant serves data-heavy responses.

When to choose it

Choose OCI's stack when your data and your users are already there, particularly when your system of record is Oracle Database — the 23ai vector integration is a genuine architectural simplification, not a marketing feature.

Choose something else when you need a specific model that OCI does not host, when your team has already built on another provider's tooling, or when the model landscape is moving faster than you want to be locked to one vendor's hosted selection. Keeping the retrieval layer separate from the generation layer is the hedge that makes that switch cheap, and it is how we build these regardless of provider in applied AI engagements.

What to do this week

Write the evaluation set: fifty real questions from real users, each with the document that should answer it. Then, separately, check where your authorisation data lives. If the answer is "the same database as the content", you have a materially simpler retrieval design available than most teams do.