AI Is Only as Good as the Data Layer Beneath It

Sit through any AI vendor pitch and you will hear about model size, context windows, reasoning scores, benchmark wins. All real. All also the most swappable part of the stack.

What you cannot swap without rebuilding from the ground up is the data layer: the schema, the validation rules, the entity graph, the attribution models, the benchmark engines that decide what the AI is even capable of knowing about your market.

The pattern repeats across every industry I have watched. The model becomes a commodity within a couple of release cycles. The proprietary data layer stays a moat for years.

Benefits makes the point sharper than most, because the source material was built for compliance, not for the questions distribution teams ask. No model, at any size, can infer office-level attribution from raw Schedule A text without the attribution pipeline that produces it. No amount of prompt cleverness substitutes for per-product premium models built and verified against 24 years of proprietary AskGMS data.

A model pointed at filings can only summarize them. Pointed at structured intelligence, it can answer questions. The lever is not the model. It is the data layer underneath.

Generic AI platforms model the world as "companies" holding "documents." Benefits intelligence needs a schema whose entities and relationships mirror how the market actually operates.

Ours centers on:

• Employer — size, industry, geography, matched buying-team contacts (4M+ across the database), benefit ratings, retirement KPIs where available
• Plan — plan year, participant counts, year-over-year linkage, product mix
• Product — individual lines across 23 categories, with modeled premium and five-tier benchmark flags
• Carrier — attributed premium, product penetration, broker network composition
• Broker firm — the resolved entity across every name variant
• Broker office — the producing location, distinct from any filing hub
• Broker agent — the individual producer, with verified email, phone, and LinkedIn across 257,600+ agents
• Compensation entry — classified across 14 fee types with peer benchmark flags

Every entity has defined attributes, allowed relationships, and rollup rules. Employers connect to plans, plans to products, products to carriers, plans to offices through attribution, offices to agents, compensation to plans and brokers with benchmark context.

This is not a generic knowledge graph wearing benefits labels. It is a schema shaped by the queries carriers, brokers, and vendors actually run: filter employers by dental premium flag, rank offices by attributed premium in a territory, compare compensation across fee types for a segment. A model on this schema walks real relationships. A model on filing PDFs walks text.

Schema defines which entities exist. Validation defines which data is allowed to populate them, and it matters more for AI than for any human-driven workflow, because models amplify whatever you feed them.

Every record entering the intelligence layer passes through gates:

• Field-level — required attributes present, types correct, values in range for the entity
• Cross-field — premium consistent with participant counts and product mix; compensation plausible for the products reported; carrier and product combinations internally coherent
• Relationship — attributed offices resolve to known entities; carrier links match product types; temporal sequences contain no impossible transitions

Records that fail are corrected or excluded by failure type and severity, and when failures cross a critical threshold, the release itself stops until a human has reviewed what changed. Bad data does not slip into search silently; at sufficient scale, it blocks publication entirely.

Feed a model unvalidated filing artifacts, duplicate broker entities, misattributed offices, premium outliers, and it will return wrong answers with total confidence. Validation is the gate that guarantees the model is reasoning over verified intelligence rather than raw submission noise. It is also an auditability story: when a sales director asks where a number came from, the answer traces to a validated record in a defined schema, not to a model's reading of unstructured text.

Schema names the entities. Relationship modeling defines how they connect, and pre-computes those connections so the model never has to guess them at query time.

The graph holds:

• Employer-carrier ties across products and years, for carrier-mix analysis and win/loss tracking
• Employer-broker ties at office and agent level, for book analysis and relationship-change detection
• Broker-carrier ties across attributed books, for distribution strategy and compensation patterns
• Cross-domain ties between group benefits and retirement on the same employer, with 838,500+ retirement plans connected for cross-sell signal
• Sentiment ties linking benefit ratings and reviews to employer, broker, and carrier profiles

These edges are persistent and verified, not inferred from language patterns on the fly. When the model traverses this graph, it follows facts. That is why a relationship question, "which brokers gained share in my territory," resolves on an intelligence platform and dissolves on a summarization tool. The graph exists or it does not. Engineering builds it. The model only walks it.

The common shortcut: embed filing text in a vector database, retrieve relevant chunks at query time, ask the model to synthesize. Retrieval-augmented generation over raw documents.

For general knowledge, RAG is reasonable. For benefits intelligence, it fails structurally:

• No resolution. Vector similarity matches text chunks, not organizations. "Smith & Associates" and "Smith and Assoc LLC" retrieve as different chunks with nothing forcing the model to merge them.
• No attribution. Retrieved chunks carry Schedule A as filed, filing hubs, not producing offices.
• No computed metrics. Per-product premium, benchmark flags, compensation peer comparisons do not live in filing text. They exist only after modeling produces them.
• No timeline. Chunks are independent documents. The model has to infer continuity across years from text alone, unreliably.
• No graph traversal. "Brokers who gained share" requires aggregating attributed-premium change across a broker-employer graph. Retrieval returns passages, not graph computations.

Vector search on filings is a filing database with a chat box. The model's fluency makes the output sound authoritative; the architecture makes it unreliable for the questions that matter.

Here is the whole stack the way I think about it:

Most AI products in this market start at layer five and work backward, if they work backward at all. We started at layer one and built up over more than two decades.

The model at the top is interchangeable. The layers beneath it are not. That is the entire reason the intelligence layer, and not the LLM, is the product.

The next article takes a related angle: why pre-computed context in the data environment makes AI dramatically more useful than any amount of prompt engineering against raw filings.