Write, Change, Recall, Forget: MongoDB's Pete Johnson on How Retrieval Drives Agent Performance

Technological concepts & key takeaways (MongoDB + AI agents)

Retrieval as the main driver of agent performance & cost

Pete Johnson emphasizes that agent effectiveness—and especially cost-adjusted performance—depends heavily on effective retrieval, not just larger context windows or better prompting.

He describes enterprise agent architectures as cycles that often:

• start with RAG forced by early LLM context limitations,
• become less critical (“dead”) when larger context windows arrive,
• then re-emerge as a major priority when token/context costs become painful under scale.

MongoDB’s approach to modern search (lexical + vector + ranking)

MongoDB continues adding capabilities developers need for AI vector search, including:

• Hybrid search: combines lexical search with vector search into a single result set.
• Aggregation pipeline features that fuse/merge results in one query:
  • rank fusion
  • score fusion
• $dollar re-rank (re-reranking tool):
  • designed to reduce the extra engineering and round-trips typically required to call a reranker after vector+lex retrieval.

Core positioning: MongoDB vector search is meant to leverage existing MongoDB infrastructure—replication, sharding, security, and aggregation pipelines—rather than forcing teams to build and maintain a separate stack.

Vector search data model & schema flexibility

MongoDB is schema-flexible, not “schemaless.”

• Documents can vary in shape.
• The JSON/BSON document model supports storing text and vector arrays as document attributes.
• This enables easier evolution than rigid SQL schema migrations—useful for AI workloads where retrieval features and attributes change over time.

Voyage AI embeddings (acquisition-driven “better together”)

MongoDB acquired Voyage AI. Pete highlights embedding-model features including:

• shared embedding space across model sizes (embeddings stay compatible across Voyage model variants)
• a “matrioska/multi-level” (nesting-doll-like) concept
• embedding support that can act as a configurable plug-in, even though MongoDB customers can use other embedding models

A key claim: embeddings aren’t commoditized yet—embedding quality differences can materially affect retrieval quality.

Chunking and semantic retrieval tradeoffs (RAG / agentic memory)

For RAG-style memory, chunking is a core engineering problem:

• Too-small chunks lose context.
• Too-large chunks increase storage and can reduce retrieval “fidelity.”

Pete argues developers currently need iterative experimentation (“chunk size vs cost vs retrieval quality”). Voyage’s approach is presented as reducing this effort via:

• Contextualized chunking: rather than choosing one chunk size blindly, it uses a representation strategy that returns one embedding vector balancing context and chunk fidelity.

Embedding dimensions optimization

Another tuning axis is embedding dimensionality:

• Higher dimensions can improve retrieval, but cost more in storage and indexing overhead.

Voyage’s “betroka reasoning” is described as:

• nested-doll style incremental testing (e.g., generate 1024 dims once, derive 512 tests via truncation),
• reducing re-embedding / reprocessing costs.

When to care: scale and model quality thresholds

Rough operational thresholds Pete provides:

• Scale becomes more important around ~100,000 vectors.
• Retrieval quality differences between embedding models can be significant:
  • cited improvement up to ~14% on a benchmark like RAGBench/Rtechweb.
• Retrieval improvements can be the difference between correct answers and hallucinations—even before rerankers.

“Bitter lesson engineering” in retrieval stacks

Pete frames maturation of retrieval systems as replacing earlier “manual hacks” with more integrated capabilities:

• easier reranking integration (via $dollar re-rank)
• less custom embedding-pipeline maintenance (via auto embeddings)

Overall: move from hand-built glue toward database-native features.

Auto embeddings (reduce pipeline plumbing)

MongoDB/Voyage “auto embeddings” concept:

• specify a collection, attribute, Voyage model, and dimension settings
• MongoDB automatically embeds and updates vector indexes when documents or attributes change

Goal: eliminate custom embedding refresh jobs and reduce ongoing maintenance.

---

Agent memory: “state of the art” patterns & the hardest part

Shift from “just stuff context into the LLM” to agentic memory

Pete describes evolution across generations:

• early: one-shot context window patterns (e.g., early ChatGPT-like usage)
• then: RAG for proprietary data injection and recency
• then: tool use / MCP-style connectivity for time-sensitive info (example: stock prices)
• finally: looping/agentic execution makes context windows insufficient, since each step needs fresh relevant memory

Memory types trending in enterprises

Enterprises are shifting from “store everything” token maxing toward more selective memory, such as:

• short-term memory: curated to what’s relevant for the current pass
• long-term memory: curated across sessions (not everything)
• taxonomic memory (emerging in larger orgs):
  • store/retrieve only vocabulary/terms relevant to the current vertical/task context per loop

Objective: reduce token costs and improve relevance by avoiding muddy, irrelevant context.

Forgetting is the hardest part

Pete uses a “write, change, recall, forget” framing:

• memory entries have a half-life
• systems must support ongoing correction/curation, not just initial summarization

He argues agent memory is harder than databases historically because agents are only ~18–24 months into building.

Where robustness comes from

He suggests robustness can come from:

• better embeddings/rerankers to improve retrieval quality across memory types
• graph/hierarchy approaches where vector search happens inside leaf nodes (multi-level indexing)

He cautions against over-relying on extra LLM passes:

• it increases token cost and latency
• embeddings/rerankers are positioned as lower-cost and faster tools

---

Build vs buy analysis (agent ecosystem maturity)

“Buy” isn’t a complete strategy

Pete describes companies in three camps:

1. bought one tool and stop
2. experimented but struggle with ROI (often because they chose the wrong problem)
3. advanced teams building sophisticated memory/optimization, sometimes buying larger platforms

How to pick problems

Choose problems where you have:

• usable data quality
• existing metrics to measure improvement

Call center / lifecycle metrics are highlighted because ROI estimation is easier.

No “lampstack for agents” yet

Agent tooling is still evolving rapidly, and enterprises need customization. He notes there’s no turnkey “React/Angular/LAMP-stack equivalent” for agents that avoids meaningful engineering.

---

Customer examples & customer readiness observations

Advanced customers outside the US

Pete claims that among the most advanced companies he spoke to recently, leading examples were outside the US—notably Mexico City and São Paulo.

11 Labs as an example

He cites 11 Labs as a customer example:

• multi-agent platform with micro-agents per customer
• implies strong memory/context handling needs

---

Product/feature highlights explicitly mentioned

• Atlas Search (managed lexical keyword search)
• Vector search integrated as document attributes (JSON/BSON + vector arrays)
• Hybrid search (lexical + vector)
• rank fusion and score fusion (single API call via aggregation pipeline stages)
• $dollar re-rank (one-call hybrid reranking)
• Auto embeddings (automatic embedding/index updates on document/field changes)
• Voyage embeddings features
  • shared embedding space across model sizes
  • contextualized chunking
  • betroka reasoning for dimensions
• MCP integration (tool calling / agent connectivity theme, discussed broadly)

---

Main speakers / sources

• Pete Johnson — Field CTO of AI at MongoDB (primary speaker)
• Nathan — host/interviewer of The Cognitive Revolution (secondary speaker)
• MongoDB / Atlas Search / Voyage AI — referenced product and technical sources
• Customers referenced — e.g., 11 Labs plus discussions across Mexico, Brazil, India, Europe (anecdotal evidence)

---