Video summary

This Flash Drive has Literally Every File

Main summary

Key takeaways

Educational

Main ideas / concepts

  • A “flash drive” is claimed to appear as if it contains every possible file ever created or to be created.
  • The drive isn’t storing real data. Instead, it behaves like a USB storage device using a computer/microcontroller that simulates a storage device.
  • The drive generates file/folder contents on demand based on the address/path the host computer requests.
  • File browsers can’t practically display/index an astronomically large number of items, so the system uses structure/encoding to represent huge (effectively infinite) file sets with manageable directory complexity.
  • Testing can be done without real hardware by using FUSE to emulate a virtual filesystem on a normal computer.

Demonstration + architecture (how it “works”)

  • When plugged in:
    • The host computer is shown thousands of seemingly random folders, creating the illusion of a massive filesystem.
  • A custom program can:
    • Take a target string (e.g., part of the video script).
    • Compute the exact folder/file path where that content would appear.
  • When the path is opened:
    • The system returns the requested file contents, which match the intended data (e.g., the script itself).
  • Core mechanism:
    • The device is effectively a USB-signal emulator (via a microcontroller), pretending to be a storage device.
    • Every reported file/folder is generated dynamically.
    • No real indexing of “all files” is done; generation occurs when the host requests directory listings or file bytes.

Key limitation and solution strategy

Limitations

  • It’s impossible for the system to:
    • Index the entire “infinite” dataset.
    • Maintain real storage for every file.
  • Practical limits include:
    • File browser performance (practical cap around ~5,000 items before UI slows).
    • Maximum file path length (too many nested folders breaks common path-length limits).
  • For physical-like storage, host computers typically query raw byte ranges at specific finite addresses, which conflicts with an actually infinite structure.
  • Under MTP, there is also:
    • A finite 32-bit identifier requirement for file/folder IDs (may limit theoretical depth on some systems).
    • A maximum file size based on available memory on the microcontroller (mitigated by using an SD card).

Solution

  • Use encoding/representation so the host can “address” a file via path while the device can decode that address into bytes.
  • Choose directory parameters to balance:
    • Too many folders → slow/large paths.
    • Too many items per folder → browser/UI constraints and lookup costs.

Directory encoding methodology (detailed)

Step 1: Start with a simple “numbers as files” idea

  • Create a folder containing files for:
    • Numbers 0 to 100 (easy).
  • Then scale upward:
    • 1,000 → still easy.
    • 10,000 → harder.
    • 1,000,000 → not feasible with naive folder-per-value since it requires too many items and UI/browser limitations.

Step 2: Represent large numbers using digits across nested folders

  • Use decimal digits as a toy model:
    • Create one top-level folder per digit (0–9).
    • Each file stores the current accumulated number.
    • Example intuition:
      • Go into folder “1”, file is “1”
      • Then go into folder “0”, file becomes “10”
      • Then go into folder “5”, file becomes “105”
  • Why this helps:
    • Each extra level adds only one more “digit position” rather than adding another order of magnitude of separate files in one directory.

Step 3: Accept another practical limit—path length

  • Even with digit nesting, you can’t go arbitrarily deep because:
    • File path length has a maximum in common file browsers/OSes.
  • So you must find a workable encoding depth while keeping per-directory item counts reasonable.

Step 4: Optimize directory parameters using alphabet/base choices

  • Improve beyond base 10:
    • Use a larger “alphabet” of symbols/characters (not just digits).
    • Each character corresponds to a sequential number.
  • Tradeoff balance:
    • Increasing alphabet size increases combinations per folder level.
    • But it also increases the number of files/folders that may appear at each level.
  • Constraint/variable framing (as described):
    • Constraints:
      • Keep per-folder file count within a practical maximum (≈ 5,000).
      • Keep path complexity within manageable depth limits.
    • Variables:
      • Alphabet size.
      • Maximum characters per folder name (i.e., how many “digits” are packed into one directory level).
  • Optimization approach:
    • Instead of “non-linear programming,” brute-force search over possibilities.
  • Result stated:
    • Assuming ~5,000 files per folder, the “optimal parameters” were:
      • Alphabet size: 70
      • Max characters per folder: 2

Step 5: Use FUSE to emulate and test filesystem behavior

  • FUSE (File System in Userspace):
    • Acts like a Linux tool letting code implement a fake folder.
  • How it’s used:
    • When an app requests directory contents:
      • FUSE redirects requests to custom code to generate the directory listing.
    • When an app requests a file:
      • Code intercepts the file-open/read operation and generates file contents from the file’s address/path.

MTP-specific protocol method (how the device pretends to be a “USB stick”)

Why normal “raw flash addressing” isn’t enough

  • A typical USB storage model exposes finite byte ranges at finite addresses.
  • Infinite unique files would require either:
    • infinite storage layout, or
    • overlapping identifiers (not allowed).
  • Therefore, real “infinite” structure is not compatible with normal raw-storage semantics.

Switch to Media Transfer Protocol (MTP)

  • MTP behavior (high level):
    • When connected, the host asks what’s in the root directory.
    • Then it asks for contents of specific folders.
    • The device can decide what to return—potentially generating files on the fly.
  • Strategy:
    • Make the USB device emulate an MTP device.
    • When asked:
      • List: generate the folder/file structure (e.g., thousands of folders).
      • Read: decode the requested path/IDs into file bytes.

“Decryption” concept: turning a folder path into file bytes (detailed)

Step 1: Treat folder path as a cipher / number

  • Each folder is assigned a number:
    • From 1 up to the number of possible folders per level (example given: 4,900 choices).
  • The full path becomes like digits of one large number:
    • Imagine one big base-4,900 number where each “digit” corresponds to a folder choice at that level.

Step 2: Convert base-4,900 number into byte values (base 256)

  • Convert that large number:
    • From base 4,900 → base 256
  • Each byte then becomes part of the file content:
    • Since each byte has 256 possible values.

Step 3: Fix the “leading zero” problem using a subtraction adjustment

  • Problem:
    • Numbers can’t have leading zeros.
    • If raw digits are mapped directly, files that should start with null bytes (0) may be impossible.
  • Fix:
    • Before outputting the byte sequence, subtract 1 from each position starting at the least significant end.
  • Effect:
    • It remaps the encoding so that what would have been “leading zeros” in the mathematical number can become valid byte sequences in the output.
  • Example intuition (as given):
    • To represent decimal 010, use an input that becomes a valid byte sequence only after the subtraction transform.

Practical limitations mentioned

  • 32-bit unique IDs required by MTP:
    • Theoretically finite → might limit how many unique folders/levels.
    • Claim:
      • Maximum practical depth is close to a million (beyond typical path-length limits).
    • Windows enforcement:
      • Seems more strictly enforced on Windows.
    • Linux behavior:
      • Testing suggests IDs can be reused and it still works.
  • Maximum file size limited by microcontroller memory holding the large number.
    • Mitigation:
      • Use SD card storage to increase practical file size.

Possible “use cases” ideas (not the core gimmick)

  • Hide content behind an “unlock” path:
    • Only reveals contents if you navigate folders in a specific sequence (secret-code-like).
  • Make the drive appear empty:
    • Reveal a hidden folder only by entering its name.
  • Use the filesystem as input:
    • Since the device can generate data, the speaker suggests it could be used to “play Doom” using folder inputs (joking/meme-level idea).

Instructions / setup resources (what the video suggests)

  • Code:
    • All code is on GitHub, with setup instructions.
  • Hardware purchase:
    • USB sticks/microcontroller-based setup can be found on AliExpress (claimed around $10).
  • Caution:
    • Not beginner-friendly.
    • The creator won’t personally debug issues or maintain the code long-term.
    • Suggestion: use a search engine for problems.

Speakers / sources featured

  • Video narrator / creator (unnamed in the subtitles; the main speaker)
  • boot.dev (sponsor)
  • FUSE (source: “Filesystem in Userspace,” Linux tool used for emulation)
  • GitHub (code hosting referenced)
  • AliExpress (hardware source referenced)
  • MTP (Media Transfer Protocol) (technical standard/protocol referenced)
  • QR code / discount code: “portal runner” (used for 25% off the first payment on boot.dev)

Original video