Jak naprawdę działa pamięć AI?

Main ideas and lessons

• AI “memory” is mostly an illusion from the user’s perspective. The model itself has no lasting memory; it only responds based on what is provided in the current interaction (plus any built-in/configured items).
• The key mechanism is the “context window.”
  • This is the maximum amount of information the model can process in a single query (given as tokens).
  • As conversations grow, the context window fills up, limiting how much earlier information can remain available.

Why AI seems to know more over longer chats

• Each new user message is sent as part of a new, independent query.
• That query includes (almost) the entire conversation history so far—up to the context limit.
• Therefore, the model appears to remember because it keeps receiving prior messages again.

Tokens determine the practical limit

• Tokens are fragments of text (not necessarily whole words).
• Example concept: “Hi, my name is Kamil …” is broken into colored token segments.
• Token counts vary by language:
  • English often uses fewer tokens per character than Polish.
  • With larger text volumes, this affects how quickly you hit limits.

The model does not learn during the conversation

• The model structure/weights do not change month-to-month.
• You can provide new information in the conversation, but you cannot retrain the model in-place through normal chat.
• The model has a knowledge cutoff (training end date), so it won’t know events after that unless you provide them in-context.

Method / guidance presented

1) Understand what “context window” means

• Treat the context window as the size of the input the model can consider at once.
• Larger context windows (e.g., GPT-3.5 vs newer models) let users provide much more text per query.

2) Use the chat effectively given how queries work

• Remember:
  • Every message triggers a new independent query to the model.
  • That query must include the conversation history.
• Implication:
  • Longer chats → larger token payload per message → context window fills faster.

3) Track practical limits using tokens

• Use tokenizer/token-counting logic to estimate how quickly you’ll reach limits.
• Account for language differences:
  • English may consume fewer tokens than Polish for similar text length.
• Expect limits to become an even more important constraint as costs and usage limits tighten.

4) Manage “memory” settings intentionally (tool features)

What’s described as “memory” is not magic:

• In products like ChatGPT / cloud tools, saved user info and instructions are stored in account/configuration and then injected into future conversations, consuming context window space.

In settings, configurable items can include:

• Personalization / tone instructions (e.g., “basic stylistic tone” or “don’t use emoticons”)
• Stored memories (facts the system saves; can become outdated)
• Ability to manage memories:
  • Manually add or delete stored memories
  • Review what’s stored periodically to remove outdated info

5) Handle context overflow intentionally

When context becomes too full, systems may:

• Drop older messages, or
• Replace conversation history with a summary

The speaker argues users should still actively manage context because:

• Limits are consumed faster in long conversations (more tokens get resent each turn)
• You don’t fully control what the automatic summary keeps or removes
• Response quality can degrade due to:
  • “Needle in a haystack”: relevant info deep in long content may be missed
  • Confusion from old content: earlier material can mislead generation

6) Recommended user strategy for long chats (practical rule)

• Keep conversations as short as possible or periodically summarize.
• If the conversation feels too long, send a prompt like:

“The conversation has become too long, summarize what we’ve talked about so far, so I can use this summary at the start of a new conversation.”

Then:

• Use the summary as the new starting context
• Review whether the summary is sufficient, contains irrelevant details, or is missing key points
• This restores user control over what goes into the context window

Speakers / sources featured

• Kamil Brzecinski (speaker/instructor)

Mentioned/used as tools or sources in the explanation:

• OpenAI tokenizer (used as an example tool to visualize tokens)
• ChatGPT (GPT chat / browser interface)
• OpenAI API
• GPT-3.5, GPT-5.5
• Cloud (as a category of similar tools)
• Gemini (mentioned as a comparable chatbot)
• Canva (example of an external tool integration)

Also referenced (generic mentions):

• Agent/tooling names: Cloud Code / Codex
• “agent/MD” style configuration files (referenced generically)