Vector database vs graph database for AI: how to choose

A supplier calls to say a component is two weeks late. Somebody now has to name every open order that is exposed. An assistant that has read the supplier contract will answer confidently and be wrong, because the answer is not written in any document. It has to be walked, order by order. That is the difference between a vector database and a graph database.

A vector database and a graph database solve different retrieval problems. “Find policy passages about shipping exceptions” is a similarity problem. “Which suppliers, products and open orders are affected by a delayed component?” is a relationship and path problem.

Two terms do the work here. An embedding is a list of numbers that stands in for a piece of text, an image or a product. Things with similar meaning get numbers that sit close together. A graph stores entities as nodes and the links between them as typed relationships, so the connection itself is data you can query.

Pick the store that fits the questions your users ask and you pay for one build. Pick the wrong one and you pay for two, while your team keeps answering the hard questions by hand. Autonomous Technologies is an engineering firm. We build and run the AI and data systems behind Shopify stores and commerce businesses, for the founders and operators who answer for what a customer is shown. Our engineering work is paid for by the build you skip: it starts with the ten questions on one page, below, so you buy one store and not two.

This is for whoever will be asked why the answer was wrong

Read this if you are designing or buying an AI assistant that reads company data, and you will be held responsible when it answers a customer. It assumes you already have an operational database and are deciding what sits beside it.

Skip this if you are shipping a demo, or your corpus is a few dozen documents that one person maintains. Plain search over those documents will teach you more, faster.

The decision in one table

Pinecone defines a vector database by the embeddings it stores and the nearest entries it returns. Neo4j defines a graph database by nodes, relationships and the queries that traverse them (Pinecone docs, Neo4j docs).

Take one question to both stores before you buy either

Go back to the late component.

Neither column answers the question alone. The passage explains the policy. The path names the orders. Ship the one your users are actually asking for first.

What a vector database does well

An embedding model turns a chunk of text, an image or a product into a vector. That is a list of numbers, laid out so related things sit close together. At query time the app embeds the question, finds close vectors, then filters by tenant, language, product line or access scope.

That helps when a user does not know the words used in the source. A support bot can match “my parcel is late” to an article called “shipping exceptions and carrier handoff.”

Closeness is not truth. A high score says only that two items sit near each other. It does not prove the passage is current, or that it applies to this customer. Store the document id, source link, version, tenant and access rights as metadata, fetch the source passage, and make the app cite it or decline.

The vector retrieval design choices buyers miss

Chunking, the way you cut a document into retrievable pieces, changes what can be retrieved at all. Large chunks can bury a precise answer. Tiny chunks can separate a rule from its exception. Filters must be enforced server-side, not merely described in a prompt. Re-indexing has to be triggered when an approved source changes.

A useful pilot is not a chatbot demo. Write real queries, name the sources that count as evidence, and measure whether retrieval returns them near the top under real access rules.

What a graph database does well

A graph database makes relationships first-class data. In a property graph, a product, supplier, warehouse and order can be nodes, while SUPPLIED_BY, STOCKED_AT and CONTAINS are typed relationships carrying their own fields, dates and status. A query follows a declared pattern rather than inferring a connection from likewise worded text.

This matters where relationships are the product logic. “Which open orders are exposed if supplier S cannot ship component C?” The answer walks supplier, part, product, stock and order. A vector search may find the right policy text. It must not decide which orders are hit.

Graphs also suit identity and fraud work, access rights, service maps and org charts. The benefit is one repeated kind of question answered with the path it walked.

The graph design choices buyers miss

Graph flexibility does not remove data modelling. Teams still need canonical entity IDs, relationship types, direction, temporal rules, constraints and ownership. “Customer” linked to “account” is not enough if the relationship could mean buyer, bill-to contact, subsidiary or beneficial owner. Those distinctions decide whether a traversal is safe.

Start from high-value questions and work backwards. Name the path, the entities, the relationship semantics, the system that owns each fact, latency expectations and how changes are passed on. If the answer can be supplied with one or two stable relational joins, a graph database may be an extra running surface.

GraphRAG is an architecture, not a synonym for graph database

GraphRAG joins retrieval-augmented generation to graph-shaped evidence. A pipeline pulls entities and links out of documents, builds a knowledge graph, and hands the nearby part of it to a model. Microsoft’s GraphRAG project works this way, and its own docs warn that indexing can burn a lot of model time. Microsoft GraphRAG docs

That suits questions that need a connected picture across a document collection. It also adds extraction errors, model cost and review work. An extracted statement should not silently become an allowed business relationship. Keep provenance on nodes and edges: source document, passage, extraction method, confidence, reviewer status and refresh date.

When the right answer is both

The common hybrid pattern:

1. Use vector retrieval to find the documents, cases or products that fit.
2. Match what you found to canonical ids, where the rights allow it.
3. Use a graph walk or a SQL query to apply the hard business rules.
4. Give the model only that evidence, with citations and a bounded reply.

What we saw on two of our own systems

The first is a repair assistant for one vehicle model. Its users ask “which page says this”. We indexed 6,264 records from 57 sources in plain full-text search with a ranking score. A 23-row alias table maps slang and market names onto search terms. No embedding model. The 20-question test set passes 20 of 20. None of the 178 queries logged since 26 August 2026 came back empty. Keyword search misses paraphrases; so far the log says it has not.

The second is the code index for our own workspace. Its users ask “what depends on this” before they change it. That is a graph question, so it is a graph: 170,505 nodes and 449,346 edges over 16,603 files.

When the other choice wins

The strongest objection to starting with a vector store is that similarity cannot enforce a rule. The assistant can cite the right policy and still name the wrong orders. Where the answer must be exact and auditable, such as entitlements, stock exposure or fraud, the graph or a SQL path wins. Similarity only finds the supporting text.

Two cases from the systems above show the line.

Similarity beats our repair index the day a user asks in words no alias row covers. The 23-row alias table is a hand-written stand-in for what an embedding does on its own. The query log is the trigger: the first paraphrase it misses is the day embeddings go in.

The graph beats a vector index over our code the moment the question is “who calls this”. A vector index could return functions that look alike. It cannot return the caller, because a caller is a link, not a resemblance.

Failure modes to plan for

Where this breaks: the store is not what makes the answer trustworthy

Here is the objection worth taking seriously. Teams pick the store, ship it, and the answers are still wrong in ways nobody predicted. The store was never the control.

Three gaps cause most of it. Nobody wrote down what a correct answer looks like, so every failure becomes an opinion. Retrieval runs as the application, not as the person, so a customer finds the leak before a test does. And the corpus moves, the model is upgraded, a category is renamed, and quality drifts with no alarm.

The fix is the same on either side of this comparison. Write the evaluation set before you write the retrieval code. Keep it small enough that a person can read every case. Then run it adversarially, on purpose, and record what broke.

That discipline is the deliverable. On the Outbound Engine adversarial build we kept 418 builder and critic rounds written down as they happened, 7 to 19 August 2026. Every accepted behaviour has an argument on the record. A store cannot give you that. A process can.

Procurement questions for engineering leaders

• Which user questions are similarity questions, and which need deterministic paths?
• How are tenant boundaries and document permissions applied before retrieval?
• Can we inspect the retrieved items, scores, graph path and source version behind a production answer?
• What evaluation set decides retrieval quality before rollout?

Your next step is ten questions on one page

Write down ten decisions your users cannot answer reliably today. Mark each one as similarity, relationship, calculation, or a mix. Build one thin slice around the highest-value question, with a source owner, access control and a test set. That page tells you which store to buy. It often tells you that you need less than you thought. Bring it to a systems call and you leave with the ten questions sorted into similarity, relationship and calculation, the store that fits them named, and the budget for the second build you no longer need. That is what the call is for: one build paid for, not two.