RAG on IBM i: Db2 for i Has No Vector Store, So Where Do the Embeddings Go?

Short answer: not in Db2 for i. There is no vector store in Db2 for i as of September 2026, IBM has not announced one, and every RAG tutorial you have read assumes a database you do not have. The embeddings go somewhere else, and IBM's own answer to where is OpenSearch running on Power, with Db2 for i staying exactly what it already is, the system of record.

That is the whole answer. The rest of this is what it means when you actually go to build the thing, because "put it in OpenSearch" quietly hides about six decisions.

The RAG loop, one stage at a time

Retrieval-augmented generation has a boring shape. You take source material, turn it into numbers a machine can compare, keep those numbers in something built to search them quickly, and at question time pull back the handful of passages closest to the question and hand them to a model as context. The model then answers using your content instead of only what it absorbed during training.

Written out like that it sounds like one system. It is six stages, and on IBM i they do not all live in the same place.

Db2 for i has no vector store. Db2 does.

This is where people get confused, and it is worth being precise because the two products share most of a name and nothing else that matters here. Db2 for i is the database built into IBM i. Db2 for Linux, UNIX and Windows is a separate product on a separate release schedule. In June 2025, Db2 12.1.2 added a native VECTOR data type, so embeddings can sit in a table alongside business data and be searched with SQL distance functions. IBM's own announcement lists retrieval-augmented generation as a headline use case for it.

Db2 for i did not get that. The July 2026 Technology Refreshes, IBM i 7.6 TR2 and 7.5 TR8, brought real Db2 for i work ... SQL Error Logging filters, trigger pseudo columns, timestamp conversion functions, seventeen new IBM i services ... and not one AI item. If you were quietly hoping a Technology Refresh would close this gap without anybody making an announcement about it, that was the TR to do it in, and it did not.

So when somebody asks whether there is a vector store in Db2 for i, or goes looking for a vector database for IBM i, the honest answer in September 2026 is no, and IBM has not published a date.

IBM's answer is OpenSearch, and it is a direction rather than a product

The clearest public statement on this came out of POWERUp in New Orleans and an IT Jungle piece on May 4, 2026 connecting the dots on IBM's AI strategy for IBM i. Ashwin Srinivas, senior product manager for AI on IBM Power, said plainly that IBM does not support vector storage on Db2 yet, and pointed at OpenSearch as the direction, because it "allows you to search across vector spaces, across text, across structured data" in one hybrid engine.

The hybrid part is the interesting bit and it is easy to skim past. A pure vector database answers one kind of question well. OpenSearch is being positioned to answer the fuzzy semantic question, the keyword question and the structured filter in a single query, which is much closer to what a real business question looks like when somebody asks it out loud. "Find me returns like this one, from this customer, in the last eighteen months" is three different searches wearing one sentence.

Two caveats matter. IBM said OpenSearch on Power. It did not say OpenSearch on IBM i. Those are different machines even when they are the same frame, and IBM has published no supported IBM i package, no reference architecture, no connector and no timeline. And this came out of a conference session and trade press coverage, not a product announcement. It is the current answer, and it is worth designing toward. It is not a supported stack you can open a PMR against.

watsonx.ai makes the vectors. It does not keep them.

There is a persistent assumption that buying watsonx gets you the whole retrieval stack in the box. It does not. watsonx.ai gives you a text embeddings API and embedding models to call it with, IBM's own slate retriever models among them, and IBM's documentation is direct about what happens next: you can store the generated vectors in a vector database. Which is to say, one you provide.

That is not a complaint, it is a design boundary, and knowing where it sits saves you a procurement conversation and a surprised architect. watsonx.ai is the model layer, for embedding at index time and for generation at answer time. The index itself is a separate component you stand up, run, secure and back up. In IBM's stated direction for Power, that component is OpenSearch.

Bob's RAG workflow is a developer feature, not your knowledge base

The Bob Premium Package for i, generally available June 24, 2026 alongside Bob 2.0, includes support for retrieval-augmented generation workflows on IBM i among its roughly forty IBM i skills and workflows and thirty IBM i tools. That one line has been read in a few shops as "Bob solves the vector problem." It does not. IBM has not published where those workflows put embeddings, what indexes them, or whether anything persists between sessions.

What Bob is unambiguously good at is the developer job: reading source directly out of QSYS and the IFS instead of making somebody download it to a PC first, explaining a program nobody has touched since 2003, converting fixed-format RPG, and writing Db2 for i SQL in its Database Mode. IBM's July 2026 Power announcement cites Heartland Co-Op getting new-to-platform developers to understand complex applications 60 percent faster with the Premium Package. That is a real, measured result about code comprehension. It is not a claim about an enterprise retrieval layer, and it should not be the reason you skip building one. The wider Bob and Orchestrate picture is in how Bob, watsonx Orchestrate and the IBM i MCP Server fit together.

On-box, off-box, and what leaves the building

Two questions decide most of this architecture, and neither of them starts out technical.

The first is what leaves the IBM i partition. Embedding a document means sending its contents to a model. If that model is hosted outside your walls, then the text of every runbook, contract and support ticket you index has left the building, once, to whoever runs that endpoint. That may be perfectly acceptable. It should still be a decision somebody signs, made before the indexing job runs, because you cannot un-send it afterward.

The second is that a vector index is not an anonymizer. In almost every RAG design the original passage text is stored beside its vector, because the whole point is to hand that text back to the model. Treat the index as a second, searchable copy of your source documents, sitting on a box that probably has weaker controls than the one you took them from. That is what it is.

What to do first

Do not start with the vector index. Start with the question you want answered, because a good half of what shops want from "AI on our data" is not a retrieval question at all.

If the question is exact and structured ... on-hand quantity for a part number, open orders for a customer, last week's shipments by carrier ... you do not want retrieval-augmented generation. You want an agent running SQL against live data, and the shortest route to that today is the IBM i MCP Server, covered in our MCP Server explainer. Embedding your order file is the wrong tool for that job and it will give you approximate answers to questions that have exact ones.

If the question is genuinely fuzzy ... how have we handled a return like this before, what does this error usually mean, what does our policy actually say ... then you want RAG, and the order to build it in is short.