Gemini Gems Masterclass with the Creator at Google: 3 Gems You Must Build

Tech/Product Summary: “Gemini Gems Masterclass with the Creator at Google: 3 Gems You Must Build”

What “Gemini Gems” are (core concept)

• Gemini Gems = custom versions of Gemini built for a specific use case, designed to solve the LLM problem of “lack of context.”
• Instead of repeatedly re-entering details (role, company strategy, product history, writing style), a Gem stores context via:
  • Instructions (how the model should behave)
  • Knowledge (uploaded documents/data)
• Lisa frames Gems like “master craftsman” specialists versus general Gemini (generalist that requires detailed prompting each time).

Why/what to build: the 3 “must-have” Gems for PMs

1. Writing Clone

   • Helps PMs produce drafts in their personal tone/style.
   • Uses uploaded artifacts like PRDs, emails, and team Slack messages to generate better first drafts faster.

2. Product Strategy Adviser

   • A strategic “thought partner” for key decisions.
   • Can incorporate company strategy docs, market positioning, go-to-market materials, and competitor analysis, among other inputs.

3. User Research Synthesizer

   • Helps synthesize scattered research inputs.
   • Uploads interview transcripts, survey data, and customer support tickets to produce Q&A, key insights, and synthesis.

How to create a Gem (tutorial/workflow)

• In Gemini:
  1. Go to Gems
  2. Click Create a new gem
  3. Provide:
     • Clear instructions (emphasized: detailed instructions → better outputs)
     • Add knowledge (upload relevant context documents)
     • Test and iterate (treat it like a mini product; refine prompts/knowledge over time)

Demo highlights

• Selecting a “product strategy buddy” gem
• Adding company files (e.g., competitive teardowns, strategy docs, roadmaps)
• Running a test prompt, then tweaking instructions/knowledge until satisfied
• Saving the Gem and starting chats under the Gems category

Team sharing

• Gems can be shared internally when teams share context; a PM can act as a power user building Gems for colleagues.

Product design/strategy lessons behind Gemini Gems (feature story)

• Lisa explains Gems emerged from observing that users struggled to discover LLM capabilities reliably.
• Instead of focusing on an “app/GPT store”-style ecosystem, Google emphasized:
  • Personal productivity and team-shared productivity
  • Less emphasis on a proprietary, monetizable “ecosystem,” because instructions are copyable and it wasn’t clear it would form a strong store-like marketplace.
• Core principle: build from first principles about the underlying goal (productivity + shared context), rather than copying competitors’ launch frameworks.

Best practices & common mistakes (review-style advice)

Biggest mistakes

• Vague instructions → be specific and include examples
• Missing key context → a Gem must be personalized with uploaded docs/data
• Creating one generic Gem instead of specialized gems per job
• Not iterating → Gem quality improves through testing and updates

Memory/persistence

• Gems rely on their stored instructions + context files; if data changes, you should update the Gem.

---

Agent/Product Insights (later in the episode): building reliable agents in finance

“Jax” at Zero (agent concept)

• Zero (finance platform for small businesses) builds an AI “financial super agent” called Jax.
• It maps financial workflows (accounting, payments, payroll, etc.) to identify which parts can be automated via agents/AI.
• It uses transaction-level data (invoices, bills, payroll runs) to generate insights and help businesses grow.

Hard lessons for agents (accuracy + reliability)

• Finance is precision-critical: decimal-level correctness matters.
• LLMs aren’t reliable “out of the box” for math/accounting/tax, so Zero uses a hybrid approach:
  1. Domain knowledge + workflow understanding
     • Fine-tuned experience design: which steps require what accuracy and where stakeholder review happens.
  2. Leverage owned data
     • Personalization at the business level
     • Benchmarking/trends across segments (region/subindustry)
  3. Hybrid systems
     • LLMs for multi-agent reasoning/workflow orchestration
     • Programmatic code where deterministic control is needed

Quality measurement/evals

• The discussion references MCP/tool-calling concerns broadly, and Zero addresses them with controlled hybrid architectures plus robust quality measurement/evals, including:
  • quality eval frameworks (“flywheel systems”)
  • human expert annotators (finance domain)
  • LM judges and automated eval metrics

Measuring success for new agent launches (framework)

• Three-phase success ladder:
  1. Baseline quality
     • Does it do the task correctly?
     • Evolve evaluation criteria per use case
     • Use human evaluators + automated judge/eval metrics
  2. Product adoption metrics
     • Usage (MAU/WAU/DAU depending on use case)
     • Retention and engagement
     • Qualitative feedback (e.g., CSAT, customer conversations)
  3. Business impact / monetization
     • Track revenue attribution internally
     • Target ARR/revenue/retention outcomes

“Agent-led growth” perspective

• Lisa partially supports the idea but argues it’s premature:
  • Many customers still need human monitoring and trust-building
  • Adoption will lag; first comes delegation to agents, then agent-to-agent/tool delegation.

---

Product Management & AI PM Career Takeaways (career/analysis)

AI and the PM role

• AI won’t replace PMs because PMs provide product judgment—there isn’t always a single “correct answer.”
• AI will supercharge PM work, especially execution tasks, but PMs still own:
  • product strategy choices
  • taste/judgment
  • deciding what to build and what not to build

PM role evolution

• PM/UX/design becomes more hybrid: PMs are expected to prototype and build early, not only write specs.
• Team structure may shift: PM-to-engineering ratios may compress, enabling smaller teams to ship.
• Hiring filters for AI PMs:
  • core PM skills: strategy, vision, metrics, execution, cross-functional influence
  • additional traits: grit, growth mindset, and fast learning in changing environments

How individuals can break into AI PM roles

• If your company doesn’t offer AI tools/space: it’s still not an excuse—use consumer LLM tools and build side projects.
• To stand out: do real work, not just talk; show fundamentals plus AI enthusiasm via tangible projects.
• Interview readiness:
  • practice heavily and repeatedly
  • communicate product sense and execution under time constraints

---

Main speakers/sources

• Lisa Hang — creator of Gemini Gems (implementation, PM gem ideas, best practices, product strategy story, and AI PM career guidance)
• Podcast host / interview interviewer (the “Jax at zero” segment transitions to Zero leadership; name not explicitly stated in subtitles)
• Mentioned companies/products as context sources: Google Gemini, ChatGPT Custom GPTs, Meta Ray-Ban Story Smart Glass / Meta AI assistant, Zero (finance platform), OpenAI / Claude / Gemini
• Sponsors referenced: Reforge Build