AeroLift Agentforce Grounding Demo

A side-by-side comparison of two ways to ground an Agentforce agent in technical product documentation. The demo ships an end-to-end runnable setup — content, preprocessing, Apex retriever, agent metadata, evaluation harness, and a live-demo script — built around a fictional industrial pump manufacturer.

The point isn't to "prove" that one approach beats the other. The point is to show when, why, and by how much they differ, on a realistic dataset with a fixed evaluation set, so the trade-offs become decision-grade information for an architect.

Note on language: the demo is intended for a German-speaking audience, so the source content, agent prompts, and evaluation questions are in German. This README is in English because the patterns and findings travel; the demo itself stays in its native tongue.

---

What we are comparing

Two grounding strategies, both reading from the same eight Markdown source documents, both running against the same agent shell with an identical system prompt:

• Variant A — Data Library PDFs are uploaded into a Salesforce Data Library. The platform's default chunker indexes them; a standard retriever serves the agent. This is the out-of-the-box path.

• Variant B — Hybrid Search Index + ID Pre-Filter The Markdown sources are pre-processed into atomic, structured records (one record per table row, one per prose section), enriched with first-class metadata fields (productIds, atexZone, tempMaxC, foerderleistungM3h, …), ingested into a Data Cloud DMO, and exposed via a Hybrid Search Index (BM25 + embedding). An Apex @InvocableMethod retriever wraps that index and passes a pre-filter on detected product IDs into the hybrid_search() call.

Variant B comes in two sub-flavours that the harness measures separately:

• B-naive — hybrid search without the product-ID pre-filter (isolates the value of the pre-processing alone)
• B-boosted — hybrid search with the ID pre-filter (the full Variant B); "boosted" is the colloquial demo label, the technical mechanism is a Salesforce pre-filter expression on productIds__c

That three-way split — A, B-naive, B-boosted — is what makes the contribution of the boost itself observable, instead of bundling everything into a single "with/without" comparison.

---

Why this domain was chosen

Real product catalogues are full of patterns that LLM-based grounding handles poorly out of the box. The fictional AeroLift Industries pump catalogue was designed deliberately to expose three of them:

1. Tables that carry the spec values get destroyed by default chunking. The product overview is an 11-column table with six models. Default chunkers tend to split tables across chunks, separating the row label (AL-3000-SX) from the cell that contains the answer (150 °C).

2. Closely related model IDs embed almost identically. AL-3000, AL-3000-S, and AL-3000-SX share most of their characters and most of their semantic context. Pure semantic retrieval routinely picks the wrong one. A hybrid index with a pre-filter on the exact ID solves this cleanly.

3. Multi-criteria questions need structured filters, not similarity. "Which pump is ATEX Zone 1 certified and delivers at least 120 m³/h?" needs an atex_zone == "1" AND flowrate >= 120 filter. A vector index that only sees prose can't reliably enforce that.

Each of the three is reproducible; the eval set covers all of them.

---

How fairness is preserved

A demo that's stacked against Variant A is not interesting. So the eval set is designed to be honest:

• 15 questions, balanced across four categories: factual-precise, multi-criteria, multi-hop, id-disambiguation.
• At least three questions are explicitly "fair winnable" by Variant A. These are direct lookups on unambiguous IDs where the default chunker has no trouble. They live in the eval set as a check against the demo becoming a straw-man argument. If Variant A doesn't pass them, something is broken; if Variant B doesn't pass them, the trade-off case becomes a lot weaker.
• Both agents use the same system prompt, verbatim. The only thing that varies is which action the topic invokes.

The eval harness logs every answer, every retrieved chunk, and the diagnostic info from the Apex retriever (which IDs the boost matched, which path was taken). The results — JSON for analysis, Markdown for reading — are the ground truth for the live demo.

---

Repository layout

content/
  source/         Markdown master documents (single source of truth, German)
  pdf/            Generated PDFs for Variant A (regenerated by build-pdfs.sh)
data-cloud/
  records/        JSON records from preprocess.ts (per-file + all-records)
  csv/            CSV ingestion files for Data Cloud (UTF-8 BOM)
  metadata/       Declarative specs for DLO, DMO, Search Index
eval/
  questions.yaml  15 questions with ground truth + expected failure-mode-A
  results/        Eval-run output: run-*.json/.md gitignored (regenerate per
                  run); comparison*.html are committed demo artifacts
force-app/main/default/
  classes/        ProductIndexRetriever, RetrieverRestEndpoint (+ tests)
  genAiFunctions/ Bundle for the Vector Search action
  genAiPlugins/   Topic A and Topic B (identical structure, different action)
docs/
  manual-agent-setup/  Bot + GenAiPlannerBundle metadata for UI reference (not deployed)
  demo-script.md       Live-demo runbook template
  screenshots/         Placeholders for the runbook screenshots
scripts/
  build-pdfs.sh         PDF generation (Pandoc + Eisvogel + xelatex)
  preprocess.ts         Markdown to atomic records + CSVs
  install-tex-packages.sh  One-time TeX package installer
  eval/                 Evaluation harness (TypeScript, ~7 modules)
DEPLOY.md           Full step-by-step setup runbook

---

Quick start

The full setup lives in DEPLOY.md. The quick form:

# 1. Local toolchain (macOS)
brew install pandoc
brew install --cask basictex
sudo scripts/install-tex-packages.sh

# 2. Build artefacts
npm install
./scripts/build-pdfs.sh        # 8 PDFs for Variant A
node scripts/preprocess.ts     # 164 records + CSVs for Variant B

# 3. Salesforce deploy (Apex + GenAiFunction + GenAiPlugins)
sf project deploy start --source-dir force-app/main/default \
  --test-level RunSpecifiedTests \
  --tests ProductIndexRetrieverTest --tests RetrieverRestEndpointTest

# 4. UI-only setup (Data Cloud, Library, Agents) — see DEPLOY.md sections 4–6

# 5. Evaluate
cp .env.example .env && $EDITOR .env
node scripts/eval/run-eval.ts --skip-judge --only Q01,Q02   # smoke test
node scripts/eval/run-eval.ts                                # full run, with Claude judge

---

Learnings from building this

This was the most interesting part to write down, because much of what we discovered along the way is the kind of thing that doesn't make it into the public Salesforce documentation yet. The list below covers the major themes; the deeper detail is in the commit history and the [Inference]-marked comments in the metadata files.

1. Pre-processing matters more than the embedding model

If you remember one thing from this demo: when your domain has tables, similar identifiers, or numeric specs that need filtering, the embedding model is not the lever you should be tuning. The lever is what you feed it.

In Variant B, the embedding model is the same one Salesforce assigns by default. The win comes from chunking on table rows, attaching first-class metadata fields to each chunk, and reserving the lexical search lane for exact-match cases (product IDs). On the eval set, Variant B-boosted reliably clears the questions that Variant A misses — and it does so without any model choice that Variant A wouldn't also have access to.

2. Hybrid search is the antidote to ID-disambiguation failure

Pure semantic retrieval is a near-perfect tool for paraphrased questions, and a near-disastrous tool for taxonomies of similar SKUs. In our test data, AL-3000-S and AL-3000-SX are two different products with two different ATEX zones (Zone 2 vs Zone 1), and a vector-only index will mix them up roughly half the time. The fix is not a better embedding — it is to add a lexical search lane (BM25) and let the retriever apply a pre-filter when an ID pattern matches in the query.

Salesforce Data Cloud's Hybrid Search Index does exactly this. It runs BM25 and embedding cosine in parallel and fuses the rankings via one of three configurable rankers (Linear Fusion, Deep Fusion, or Reciprocal Rank Fusion, default keyword_weight α = 0.5). On top of that, the index supports up to 10 pre-filter fields declared at creation time — these are what we use to restrict candidate chunks to the matched product ID before fusion ranking even runs. The combination of fusion ranking + ID pre-filter is what makes the disambiguation deterministic.

The detection-and-filter logic lives in 30 lines of Apex (ProductIndexRetriever.extractProductIds + buildProductIdFilter). It is the single highest-leverage piece of code in the entire demo.

References: Salesforce Help — Create a Hybrid Search Index · Trailhead — Optimize Hybrid Search Results for RAG.

3. Salesforce metadata reality is messier than the docs suggest

The most painful surprises were not in our own code; they were in figuring out which Salesforce metadata is actually deployable as classic SFDX source format in the current release, and which only exists in the UI or via the Data Cloud REST API.

• DLO and DMO are not classic SFDX in Spring '26. Despite long-standing community examples that put them under force-app/main/default/..., the current production path goes through Data Cloud Datakits or Connect REST calls. We kept the schemas as YAML specs under data-cloud/metadata/ so the UI setup has a single source of truth, but they don't ship with sf project deploy.

• Vector / Hybrid Search Index is UI-only. Same story. The metadata API doesn't expose them yet (per the Spring '26 Metadata Coverage Report). DEPLOY.md has the click path.

• Bot, BotVersion, GenAiPlannerBundle deploy is brittle. We initially built the agent shells as deployable XML. Validation revealed that the Bot.type enum doesn't accept EinsteinServiceAgent (despite our skill references suggesting otherwise — the actual valid value is ExternalCopilot, with agentType=EinsteinServiceAgent being a separate field), and GenAiPlannerBundle rejected our plannerInstructions element entirely. The pragmatic decision was to deploy the deterministic parts via SFDX and create the agent shells through the UI, since BotDefinition records can't be cleanly deleted via API anyway. The retired metadata is preserved under docs/manual-agent-setup/ as a reference for the UI configuration.

• GenAiPlugin schema needs the field names from a real org retrieve, not from documentation skills. We retrieved a working GenAiPlugin from the target org, diffed against the skill template, and found three inconsistencies: <pluginType>Topic</pluginType> (not Standard), <genAiPluginInstructions> as a structured sub-element (not a flat <pluginInstructions> string), and <genAiFunctions><functionName> (not <function>). The deployed XML reflects the real schema; the rationale is in inline comments.

• Master Language is a restricted picklist that did not accept de_DE. We set the topic language to en_US (with German content); the agent understands and answers in German anyway. This is documented as an inline comment, since it is a likely future-edit pitfall.

• DLO and DMO do not behave the same way regarding __c suffixes. DLO field names must NOT contain double underscores — the validator rejects any __c suffix. The DMO mapping layer then automatically appends __c to every field. So the same logical field is productIds in the DLO and productIds__c in the DMO. SQL queries against the index DLM go through the DMO column names, which means the Apex retriever references productIds__c even though the source data does not.

• DLO field types are restricted. Only Text, Number, DateTime. Booleans become Text ("true" / "false"), Picklists become Text, LongTextArea becomes Text. Plan for that early — the YAML schema in data-cloud/metadata/dlo/ reflects what works.

• Data Stream wizard creates its own DLO. A pre-built DLO is not reusable; the wizard insists on creating a fresh one as part of the stream definition (1:1 binding). Stream first, DLO inferred — not the other way around.

• Hybrid Search filter expressions support = and prefix-LIKE only. No leading wildcards, no INSTR, no STARTS_WITH. The filter for multi-value productIds had to be modeled as productIds__c = 'AL-3000-SX' OR productIds__c LIKE 'AL-3000-SX;%' — one equality clause for single-ID rows, one prefix-LIKE clause for multi-ID rows where the searched ID happens to come first alphabetically. The embedding score covers the rest.

• Agent Runtime Users live on the Einstein Agent license. Custom Permission Sets must declare <license>Einstein Agent</license> — and once created, the license field is not updatable, so getting it right on the first deploy matters.

• ConnectApi.CdpQuery.queryAnsiSqlV2 requires Data-Cloud-query permission separately. The Agent Runtime User needs both the custom permset (Apex class access to ProductIndexRetriever) AND a Data-Cloud-query permset like the standard CDPAdmin or xDO_DataCloud_Base_PSG. Symptom of missing the second one is silent: the action gets called, returns result: [], and the trace shows Insufficient Privileges: This feature is not currently enabled for this user.

• Action output Show in conversation is a UI override, not a schema setting. copilotAction:isDisplayable: false in output/schema.json is honored by the metadata layer but does not turn off the runtime rendering of the action output as a separate panel. Net effect via API: the LLM produces only a brief intro ("Hier sind die Details ...") expecting the panel to appear separately, while the API caller only sees the intro. The eval harness aggregates result[] from the response to get the user-visible content; that is the cleanest workaround until the schema setting wins over the UI default.

The general lesson: when working with Salesforce metadata for AI features in the current release, retrieve a known-working example from the target org before trusting any external schema reference, including official docs and skill templates. The release surface moves faster than the docs.

4. Apex has a few sharp edges around invocable methods

These cost us an iteration cycle each, but they're easy to remember once seen:

• One @InvocableMethod per class, full stop. Our first design had two (searchBoosted, searchNaive); the validator rejected the class. We kept searchBoosted annotated for the agent path and made searchNaive a regular global static method, callable via the REST wrapper for the evaluation baseline.

• inner is a reserved identifier. It's not in any list we could find upfront, but the compiler will tell you fast. Renamed to payload.

• A global REST method can only return global types. Our inner classes (Request, Response, SearchResult) had to be promoted from public to global for the REST wrapper to compile. Same applied to every @InvocableVariable field.

• ConnectApi.CdpQuery.queryAnsiSqlV2 has no public test stub. Apex test mocking patterns we have for HttpCalloutMock don't translate. We decided early to test the deterministic bits (ID extraction, filter construction, SQL composition) and accept that the platform call itself doesn't get coverage. That capped ProductIndexRetriever at ~47% coverage. For demo orgs that's fine; for production-grade work, this becomes a real architectural decision: either accept the gap, refactor to a dependency-injection seam, or wait for an official mock helper.

5. Same-org callouts don't need Named Credentials

We initially designed the Apex retriever as a Connect REST callout via a Named Credential, with External Credential, principal mapping, the works. After validating against current Salesforce documentation we tore that out: a same-org Apex class can call ConnectApi.CdpQuery.queryAnsiSqlV2 directly, no network hop, no auth setup. The simpler version is roughly 100 lines of metadata shorter and removes a dozen UI-configuration steps from the runbook. The lesson is to validate the architecture against the docs for the deployment shape you actually have before importing patterns from multi-org or external integration scenarios.

6. The eval harness is what makes the live demo defensible

Live demos go wrong. Network blips, agent latency, cache misses, unexpected-but-correct paraphrases. The eval harness is what turns "trust me, this works" into "here's the JSON, here's the score per question, here's the exact diagnostic info from the retriever." The 18-minute live demo gets about 5 minutes of actual agent interaction; the rest is architecture narration plus walking through the eval results that already ran. Without the eval, we'd be hostage to whichever way the live agents felt that day.

This shape — automate the evaluation, narrate the architecture live — is the pattern we'd reuse for any future grounding demo.

7. Production deploys to demo orgs still trip over coverage thresholds

Our org accepted the deployment, but the standard 75% Apex code coverage requirement on Production deploys would have rejected the retriever class (47% coverage, since ConnectApi.CdpQuery isn't mockable). For demo orgs this works because new classes are deploying together with their tests in a single transaction; for follow-up edits, coverage will eventually become a gate. The right place to address this is with a dependency-injection seam that lets the retriever be tested with a fake query executor — a refactor worth doing if this code ever migrates to a production-quality codebase.

---

Where to look next

• DEPLOY.md — full setup runbook for cloning the demo into a new org
• docs/demo-script.md — live-demo template; fill in the question IDs after running the eval
• eval/questions.yaml — the 15-question evaluation set with ground truth and expected failure modes for Variant A
• data-cloud/metadata/ — DLO / DMO / Search Index specs (UI-driven, not SFDX-deployable in this release)

---

Disclaimer

All model numbers, specifications, certificate numbers, regulatory references, and company names in this repository are fictional. "AeroLift Industries" and the AL-3000 / AL-3500 pump series do not exist; ATEX certificate numbers follow the real DEKRA EXAM GmbH naming pattern but are entirely invented. Resemblance to real products or certifications is unintentional.

License

MIT. Use freely for demos, training, evaluation. If adopting in production: keep the disclaimer, swap the fictional product data for real values.