The Knowledge Graph You Bought Won't Save Your AI

There's a market selling you the tools to build your own context layer. Most of the companies that buy them never reach production — and the reason is a distinction the vendors work hard to blur.

Last time, I left you with a thought experiment. If the largest technology companies in the world are fighting over who owns the context layer for your business — and none of them is building the one layer that decides whether your AI actually works — then the obvious move is to build that layer yourself. Own it. Depend on no one.

There is a market ready to sell you exactly that. It is large, it is growing fast, and it is full of genuinely good technology. And most of the companies that buy into it never get to production.

That failure rate isn’t a knock on the tools. It’s a signal — and once you understand what it’s signaling, you understand the whole context layer problem better than most of the people selling into it. So this piece is about the build-your-own path: who’s selling it, why it stalls, and the one distinction the entire category depends on you not noticing.

The market you’re about to walk into

If you go looking to build your own context layer — your own knowledge graph, your own semantic backbone for AI — you’ll find a mature and crowded field waiting for you. There are graph databases that have run in production for a decade. There are semantic-layer tools that define your metrics once and serve them everywhere. There are ontology platforms with deep academic pedigree, RDF engines, entity-resolution systems, and a new wave of “build your graph faster” accelerators that promise to turn your existing tables into a graph with a few clicks. The analysts size this market in the billions and climbing at north of 20% a year. It is not vaporware. Some of these are excellent, serious pieces of engineering.

And here is the number that should give you pause before you sign anything: by most industry estimates, fewer than one in six of these projects makes it past the pilot stage. The rest stall — not on the technology, but on something the technology was never going to solve for them. The buyer guides, written by the vendors themselves, are surprisingly honest about where the bodies are: projects die on ontology design, on entity resolution, and on a slow realization that arrives six to nine months in. Teams discover they’ve started accumulating — the vendors’ own words — “custom validation layers and semantic constraint logic that grows faster than the core application.”

Read that phrase again, because it’s the whole story hiding in plain sight. The tool gave them a graph. Then they had to hand-build, on top of it, an ever-growing pile of logic that checks whether the thing in the graph is right. And that pile grew faster than everything else, because the tool didn’t provide it and was never going to. They didn’t buy a context layer. They bought an empty engine and slowly discovered they’d signed up to build the hard part themselves.

To see why, you have to see the distinction the market is built on blurring.

Two things both called “knowledge graph”

The words “knowledge graph” get stretched across two things that are not the same, and the gap between them is where your project lives or dies.

The first is a generalized knowledge graph. It’s an engine for representing any entities and any relationships. It knows how to store the fact that A relates to B, how to let you query the connection, how to infer that if A contains B and B contains C then A contains C. What it does not know is anything about your business. It arrives empty. “Customer,” “order,” “delivery,” “billing block,” and every rule connecting them is something you have to pour in yourself. The engine is domain-agnostic by design — that’s its selling point, and it’s exactly why it’s crowded and increasingly cheap. A general graph engine is a commodity, and the market knows it, which is why every vendor is racing to add “AI-assisted” graph-building to make the pouring-in feel faster.

The second is a domain-specific knowledge graph. This is one where the domain’s real entities, relationships, and rules are already encoded — where the graph doesn’t just know that an order relates to a delivery, but knows what a correct order-to-delivery-to-billing sequence looks like, and therefore what a broken one looks like. It knows that a delivery with no invoice behind it might be a revenue leak or might be a perfectly legitimate consignment that bills later — and it knows which questions to ask to tell them apart. The rules aren’t poured in by the buyer. They’re native, because someone who understands the domain built them in.

Here’s the move the market makes: it sells you the first and lets you believe you’re getting the second. The demo shows a graph answering business questions beautifully — because the demo was hand-built by experts who poured in exactly the right domain knowledge for exactly those questions. Then you buy the engine, point it at your data, and discover it’s empty. The “AI-assisted graph builder” extracts your tables and their relationships — it reads your schema and infers that this table joins to that one — and calls the result a knowledge graph. But it has only extracted structure. It has learned what your data is. It has learned nothing about whether your work is right, because that isn’t in the schema to extract.

Why the domain graph can’t be extracted — only encoded

This is the hinge, so let me be precise about it.

Everything the generalized tools can automate is a form of extraction. They read what already exists — your tables, your columns, your foreign keys, your query history — and they structure it. Modern ones do this well; they’ll auto-generate an ontology from your database and save you weeks of documentation. But extraction has a hard ceiling: it can only surface what’s already written down somewhere. And the thing that decides whether your work came out right is, definitionally, not written down. It’s the difference between what your system recorded and what your business intended — and intent doesn’t live in the schema. A delivery record doesn’t carry a field that says “this one’s a legitimate consignment, that one’s a leak.” That judgment lives in the process, in the exceptions your business has run for years, in the heads of the people who know which is which. No amount of reading the tables extracts a rule that was never in the tables.

Which means the domain graph — the one that would actually catch the problem your AI is about to miss — cannot be generated. It has to be authored, by someone who carries the domain rules that were never written down and can encode them. That’s not a limitation the next release of the tool fixes. It’s a category boundary. Extraction and authorship are different acts. The entire generalized-graph market is an extraction market, priced and built for a problem — “structure my existing data” — that is genuinely being commoditized. The thing you actually needed sits on the other side of a line the tools can’t cross, because it was never a tooling problem. It was a knowledge problem wearing a tooling costume.

This is why the projects stall at exactly the six-to-nine-month mark. That’s how long it takes to build the graph, point it at production, and discover that the graph doesn’t know which exceptions are real — and to start hand-writing the “validation logic that grows faster than the application.” That growing pile is the domain graph they didn’t buy, being built by hand, by people who are learning the domain rules the hard way because no tool handed them over.

What this means for an SAP customer specifically

If you run SAP, this distinction is not academic, and it cuts sharper for you than for most.

Your ERP is the most domain-specific system you own. Decades of your business’s actual operating logic are encoded in how your order-to-cash, procure-to-pay, and record-to-report processes actually run — including every legitimate exception that looks like an error and every quiet error that looks fine. A generalized graph tool pointed at your SAP data will extract the structure beautifully: it will learn your tables, map your relationships, auto-document your fields. And it will know nothing about whether a given transaction did the right thing, because that knowledge was never a table. It’s the operating reality on top of the tables.

And yes — this is the moment to talk about the graph you were just handed. At Sapphire this spring, SAP put a knowledge graph at the core of its new platform: 452,000 tables, 7.3 million fields, by SAP’s own count — fifty years of ERP engineering made machine-readable. It is the largest structural extraction ever shipped, and it’s genuinely useful. Now run it through the same test. It encodes what SAP’s tables mean — SAP’s meaning of SAP. It does not, and cannot, encode whether your transaction did the right thing, because your operating reality was never in those tables either. SAP’s CEO says customers store their most mission-critical domain know-how in SAP. What’s stored is the record of what you did. What was never stored is whether it was right.

One more wrinkle before you point any tool at those tables: as of this spring, SAP’s API policy gates what autonomous tooling may extract at all — third-party AI extractors now need an endorsed pathway. So even the extraction half of this market has an access question hanging over it. That’s a different essay.

So when a vendor — or an internal team, or a systems integrator — proposes to “build you a knowledge graph for your SAP data so your agents have context,” the question that separates a production system from a stalled pilot is simple: are you extracting my structure, or encoding my rules? If the answer involves pointing an AI-powered tool at your tables and generating a graph, you’re getting the first, and you’ll meet the wall at month six. If the answer involves someone who actually understands how your specific processes run — in project mode, in steady-state run, in cutover, in hypercare, at quarter-close — sitting down and encoding what correct looks like across all of those, you’re getting the second. Those are different projects with different price tags and, more importantly, different outcomes. Most of what’s being sold is the first, dressed up to look like the second.

The one line to carry out of this

The build-your-own-context-layer market is real, and for the problem it actually solves — extracting structure from your existing data — it’s mature and worth buying. But that was never the problem that breaks your AI after go-live. The problem that breaks your AI is the layer that knows whether your work came out right, and that layer can’t be extracted from anything, because it was never written down. It has to be authored by someone who carries the domain.

That’s why the largest technology companies keep shipping the first at ever more heroic scale and calling it the second, why the graph-tool market can’t sell it to you no matter how good the demo looks, and why one in six of these projects reaches production while the rest quietly discover they signed up to build the hard part by hand. The generalized graph is a commodity. The domain graph is the moat. And the reason the whole industry keeps selling you the first while you need the second is that the first can be manufactured — and the second can only be known.

Next time: if the domain graph has to be authored rather than generated, what does authoring it actually look like — and how do you tell, before you commit a budget, whether the people offering to build yours can tell a real exception from a broken one?