METC Skill Sprint Talks #1 - Inside AI and the Future It Is Building

Main ideas, concepts, and lessons (structured)

1) Event purpose and context (METC Skill Sprint Talks #1)

The session is the first “MTC/METC Skill Sprint Talks” event themed “Inside AI and the Future It is Building.”

The speaker(s) aim to:

• Explain how AI works (especially generative AI)
• Show how AI evolved into today’s systems
• Teach how to implement AI in daily life/work to improve efficiency—particularly for repetitive tasks
• Emphasize that AI’s polished output can be misleading, so users must understand limits and verify results

2) Who’s featured / invited to teach

• A moderator introduces the program and incentives.
• The main teaching is delivered by an expert from Decoding Indonesia (Angel/Putri Angel Saraswati).

3) Core lesson: “AI doesn’t think like humans” (but can sound human)

AI outputs can be fluent, convincing, apologetic, and empathetic—making people assume understanding.

The talk stresses a key distinction:

• Language fluency ≠ human-like understanding or consciousness

Supporting concept: Octopus Test (Emily Bender / Alexander Koller)

An “octopus test” thought experiment is used to explain the “meaning problem”:

• Two humans communicate with an octopus via cables.
• The octopus listens for years, learns communication patterns, and can imitate responses.
• But without real-world understanding of context, it may respond plausibly while still failing meaningfully (e.g., in an emergency scenario).

Lesson applied to AI:

• Success at producing coherent text does not automatically prove grounded understanding like humans.

Additional research mentioned (Anthropic, 2025; large language models)

The talk cites findings suggesting some models use mechanisms beyond simple next-token guessing, such as:

• internal processes that resemble planning in constrained tasks (e.g., rhyming/structured poetry)

However:

• planning is not proof of awareness or human consciousness
• experiments may be model-specific
• tasks may be narrow
• planning ≠ consciousness

4) How AI generates responses (from tokens to output)

The talk explains generation as staged:

• Stage A: Tokenization

  • Input text is split into tokens (could be a full word, part of a word, punctuation, or characters).

• Stage B: Context window / attention

  • The model uses a limited context window (“short memory”).
  • It selects relevant parts of the input.
  • It evaluates semantic relationships between tokens.

• Stage C: Scoring with probability distributions

  • The model computes probabilities over candidate next tokens using patterns learned during large-scale training.

• Stage D: Decoding (choosing the next token)

  • The system chooses a token from the probability distribution using strategies such as:
    • Greedy decoding: pick the highest-probability token each step (safer, less diverse)
    • Temperature sampling: control randomness
      • low temperature → more rigid/focused output
      • higher temperature → more diverse/creative output
    • Top-p / nucleus sampling: choose from the smallest set of tokens whose cumulative probability reaches a threshold
      • e.g., 0.9 → “90% of the probability mass”

• Iteration

  • After one token is chosen, the process repeats until the output completes.

5) Why hallucinations happen—and why trust must be earned

Hallucinations aren’t treated as random “bugs,” but as outcomes of two main factors:

• Pretraining limitations

  • The model learns patterns without truth labels.
  • Rare facts may have weak statistical signals → the model can generate plausible but incorrect statements.

• Evaluation/training objectives

  • Many systems are optimized to produce an answer rather than abstain.
  • If uncertain, they may still output something (since “no answer” can score poorly).

Mitigation strategies mentioned:

• Train the system to recognize uncertainty
• Ask for clarification
• Use retrieval (RAG-style ideas)
• Re-check sources

User responsibility emphasized:

• Fluency/length/professional tone does not guarantee correctness
• The higher the stakes of a decision, the more critical verification becomes

6) “Agentic AI” / AI agents (AI that can act, not just answer)

The talk extends from chat/next-word generation to agents (“agentic AI”):

• An agent is a model embedded in a system with components that enable action toward a goal with minimal human intervention.

Agent system components described

1. Model (language/decision component)
2. Goals (what to achieve)
3. Tools (web search, databases, API calls, file access/editing, etc.)
4. State/Memory (what happened before; prior context, past actions, whether things worked)
5. Feedback loop: observe → choose next action → use tools → read results → repeat

Measuring “agency”: levels of autonomy

From least to most autonomy:

• 1. Suggestion: AI only recommends; user acts.
• 2. Execute with approval: AI proposes actions; user approves (e.g., user presses send).
• 3. Execute within boundaries: AI acts independently but only within strict permissions.
• 4. Plan and recover: AI plans multi-step work and detects failures, then attempts alternatives.
• 5. Delegate and coordinate: AI breaks goals into sub-tasks and coordinates tools/other agents.

7) Prompt Engineering vs Context Engineering (how to control outputs)

The talk argues effective prompting increasingly depends on context, not only prompt text.

Context engineering may include:

• system instructions
• goals
• tools and permissions
• examples
• external data sources
• conversation history and prior action results

Enterprise focus example:

• Real enterprise AI problems often come from data being scattered across systems/divisions and not accessible.
• Solutions mentioned:
  • Microsoft Fabric (data analytics platform combining storage, engineering, and cloud BI)
  • OneLake (central data repository)
• Result: AI/agents answer using organization-specific context, not generic internet knowledge.

8) Future direction: model “teams” rather than one ever-bigger model

Instead of a single large model, the future may resemble a team architecture:

• general-purpose model vs specialist model
• large vs small models depending on task complexity/cost/speed

Rationale:

• not all tasks need the same capabilities
• routine tasks can use smaller/faster models

9) Physical AI / robotics: same concepts, higher safety/monitoring needs

“Physical AI” is intelligence connected to the physical world:

• requires sensors/cameras
• must model environment state
• must plan actions
• must control devices/robots

Why it’s riskier:

• chatbot mistake → wrong report/email
• physical AI mistake → wrong object picked, vehicle misreads environment, scheduling errors

Additional concerns beyond the model:

• latency/performance
• device constraints (battery/efficiency)
• monitoring, simulation
• accountability/responsibility

10) Regulation and compliance timeline mentioned

EU AI regulation is referenced (subtitles noted “EUI/EUI,” clearly referring to EU AI rules).

Timeline and obligations mentioned:

• Feb 2025: “literacy”/requirements begin to apply
• Aug 2025: transparency obligations for providers of general-purpose models
• Aug 2026: broader rule implementation and supervision/enforcement
• a code of practice mentioned for compliance

11) How to survive in the AI environment (human critical thinking)

A survey result is referenced (Microsoft Research + a university; 319 knowledge workers, 936 examples):

• higher user confidence in their own abilities → more critical involvement
• higher trust in AI for a task → less critical thinking (self-report, so not proof of causation)

Three response paths to AI:

1. Avoid it: don’t use AI (but you may miss learning opportunities).
2. Obey it: accept outputs without validation (fast, but reduces judgment).
3. Work with it critically (recommended):
   • use AI leverage while keeping responsibility

Practical behaviors recommended:

• start with small, real tasks
• attempt some thinking without AI first
• collaborate and test AI outputs (create alternatives; look for weaknesses)
• audit: verify facts/sources/logic and evaluate risks/quality

12) Hands-on example: “Can we trust and confident AI?”

A data science mini case study covers prioritizing support tickets using confidence-aware ML.

Scenario:

• A company receives many support tickets daily.
• Some tickets should be prioritized earlier based on escalation risk.

Method flow (as described):

• understand the problem
• understand the dataset
• save data to a “data house”/storage (“e-house”/“warehouse” term used)
• explore patterns (classification)
• generate predictions

Dataset/data handling mentioned:

• uses “Damai’s dataset” (from the live channel)
• prediction target: probability of escalation
• escalation rate described conceptually as a formula-based metric

Modeling:

• uses logistic regression
• not strictly maximum accuracy; example accuracy around 0.6 is mentioned

Confidence analysis and human review rule:

• inspect predictions with confidence values (example: some high-confidence outputs were wrong)
• introduce a threshold:
  • low/uncertain confidence → send to human review
  • high-confidence → auto-prioritize (or route) without full human review
• store results back to “Lake house” for reuse

13) Q&A: user evaluation and technical questions

Key Q&A points covered:

Benchmarking/evaluating LLMs (non-deterministic outputs)

Evaluation methods mentioned:

• task success checks
  • e.g., does generated code run?
  • does a query return correct data?
  • does a spreadsheet compute correctly?
• LLM-as-a-judge
  • a model evaluates another model’s output using a rubric
• combine judges with other metrics

Also consider:

• latency
• cost per request
• availability, rate limits, token usage
• privacy considerations

Long-term memory in AI

• LLMs do not automatically remember everything.
• The context window only includes what’s provided each time.
• A separate memory architecture is suggested:
  • working memory (current context)
  • episodic memory (past chats/events/decisions)
  • semantic memory (stable organizational facts/preferences)
  • procedural memory (instructions/steps/goals)
• Memory is stored outside model weights (databases/document stores/vector DBs).
• Retrieved memory is inserted back into context when relevant.

Why RAG prefers token-based embeddings over word matching (FAQ retrieval)

• user questions may use different wording but the same meaning
• word overlap might be low/none
• embeddings convert queries/documents into vector representations of meaning/patterns
• retrieval compares vectors (semantic similarity) rather than literal word overlap

14) Event mechanics: participation, test, prizes, attendance

Incentives:

• Tokopedia gift cards for winners (initially stated: 10 lucky people, Rp50,000 each).
• Winners selected via giveaway after a post-test.
• Winners notified/distributed via email; estimated delivery within 14 working days.

Requirements:

• attend from start to finish
• fill post-test
• later fill attendance & feedback form (rating 1–5)

---

Speakers / sources featured (identified in the subtitles)

1. Cantika Putri Erbilansah — Program officer, DCoding Indonesia (moderator/host)
2. Putri Angel Saraswati / Sis Angel — Curriculum Developer at Decoding Indonesia (main expert speaker)
3. Emily Bender — referenced in the Octopus Test thought experiment
4. Alexander Koller (Coler) — referenced in the Octopus Test thought experiment
5. Anthropic (2025 research) — referenced for “On the Biology of a large language model”
6. Microsoft Copilot / Microsoft — referenced as tools/services (Copilot used for live draw; Microsoft also funds METC)
7. Microsoft Research + a university — referenced in the survey about knowledge workers (confidence vs critical thinking)

Prize winners’ names were read during the giveaway, but they are not treated as “speakers/sources” since they were not presenting content.