The Best Local Agentic Coding Workflow (Complete Guide)

Goal / overview

• The video argues that cloud AI pricing is “outrageous” and presents a complete local “agentic coding workflow” that runs privately on your own machine, including:
  • autocomplete
  • agent mode in VS Code
  • terminal agent workflows with Pi
• It emphasizes learning the core concepts so you can adapt setups to different hardware and future model changes, specifically:
  • model size (parameters)
  • context window
  • GPU/VRAM limits
  • quantization
  • MOE (Mixture of Experts)

Core technical concepts (how local models run)

Model size (parameters)

• Models vary by number of parameters (examples mentioned include 862B, 128B, 27B, etc.).
• Very large models often won’t fit locally without special architectures or techniques, so you typically need smaller models.

Context window (maximum tokens considered at once)

• A larger context window helps with longer tasks and can reduce “forgetting.”
• However, increasing context also increases memory usage and may exceed available GPU capacity.

GPU memory (VRAM) + system RAM behavior

• Models load into GPU VRAM first.
• If VRAM is insufficient, the remainder can overflow into system RAM, which can drastically slow generation.
• On Mac unified memory, overflow behavior differs because GPU and system memory are handled differently.

Practical takeaway

• Performance depends heavily on fitting the model (and requested context) into GPU VRAM as much as possible.

Product/tooling guide: LM Studio + model selection

Why LM Studio

• LM Studio is recommended as the easiest UI for local models because it includes:
  • a model browser
  • estimates of how much VRAM a model will use
  • explanations of RAM/VRAM constraints in the interface

Model sources

• Hugging Face is used for discovering models (e.g., trending, filtered by “inference available” / reasoning capability).
• LM Studio can also access models more directly and supports downloading quantized variants.

Key model features the video focuses on

Quantization (Q4/Q6/Q8, etc.)

• Quantization reduces model size by rounding/compressing weights.
• Tradeoff:
  • smaller/faster, but slightly lower quality
• The video suggests starting around Q4 as a “happy sweet spot” (example claim: Q8 → Q4 can roughly halve size across quantization levels).

Capabilities to look for

• Vision (image understanding)
• Tool use (required for agentic coding workflows)
• Reasoning (better “thinking,” typically slower and often larger)

Loading and performance tuning in LM Studio

Setup steps (high level)

1. Download a model from LM Studio / Hugging Face
2. Load it in the LM Studio chat panel
3. Enable “manually choose model load parameters” (advanced controls)

Advanced settings emphasized

• GPU offload: set high/max if the model fits in VRAM to keep execution on GPU
• Context length: adjust to prevent spilling into system RAM
• Reasoning “thinking”: control how much time the model spends reasoning (affects speed)

Measured impact (as described)

• If the model fits in VRAM: very high throughput (video claims ~100+ tokens/sec)
• If it spills into system RAM: ~20–30 tokens/sec (example claim: ~6x slower)
• Extreme context settings can also cause major slowdowns

Big technique for running larger models: MOE (Mixture of Experts)

Why MOE helps

• The video introduces MOE models to overcome VRAM limits:
  • Even if the overall model is large, only parts (“experts”) are active at a time.
  • This can improve performance per hardware constraints.

How MOE is tuned in LM Studio

• Look for model names indicating active parameters (examples mention patterns like “35B … A3B”).
• Use an MOE-related option:
  • “number of layers for which to force MOE weights onto the CPU”
  • Lower values keep more on GPU; higher values push more onto CPU.
• The video experiments to find balance (example: about 3 to 4 as tuned in their test).

Performance comparison shown

• With MOE offload tuning: ~40+ tokens/sec (slower than fully-in-VRAM, but much better than alternatives)
• Without MOE (pushing everything to GPU): can become extremely slow (described down to ~1 token/sec) due to spilling into system RAM

Autocomplete in VS Code: Continue extension (LM Studio as the provider)

Tool positioning

• Continue (VS Code extension) is positioned as the key tool for autocomplete.

Setup highlights

• In LM Studio, ensure:
  • the server is running
  • you use the OpenAI-compatible endpoint URL
• In Continue:
  • configure autocomplete model + roles using the configuration file
  • use a small fast model for autocomplete (example: ~1.5B coder to keep latency low)
  • configure timing controls:
    • autocomplete timeout (video sets ~1,000 ms)
    • debounce (delay after stopping typing)

Configuration details

• Add LM Studio as a provider
• Copy the model’s name/ID from LM Studio
• Set API base to LM Studio URL + /v1
• Define roles (including something like autocomplete)
• Use developer logs to debug latency if autocomplete fails to trigger

Speed requirement

• Autocomplete must respond quickly; if a model is too large/slow, switch to a smaller model or lower context.

Agentic coding in VS Code: Continue “chat” vs “agent mode”

Adapting the same config

• The Continue model config is adapted for agentic workflows:
  • Agent mode: set capabilities like:
    • tool use (file editing/creating, etc.)
    • image input if using vision models
  • Chat mode: may not use tools (agent mode uses tools)

Demonstrated capability

• Agent mode can create/edit files in the project (example: creating test.ts with code).
• The video notes Continue agent workflows can feel buggier than Copilot, but still functional.

Alternative agent experience: GitHub Copilot (beta) with local LM Studio models

Setup approach

• Uses a VS Code insiders/beta feature for configuring local OpenAI-compatible models inside Copilot.
• Steps described:
  • Add models using “OpenAI compatible”
  • Provide:
    • provider name (LM Studio)
    • “API key” can be any value since local models don’t require one
    • capabilities flags: tool calling and vision
    • token limits copied from the LM Studio context window

Tradeoff

• Even with a local model, Copilot still requires internet because Copilot interacts with GitHub infrastructure.

Terminal agent workflow: Pi + Open Coder-style harnesses

Why Pi

• The video switches to Pi for terminal-based agentic coding, described as more suitable for local/offline-style use.

Setup

• Install Pi via a command from pi.dev
• Configure Pi models via its models JSON/YAML file path (location depends on OS)
• Provider: LM Studio
• Base URL: LM Studio URL + /v1
• API type: OpenAI completions endpoint
• Model config includes:
  • context window (copied from LM Studio)
  • reasoning flag (for reasoning-capable models)
  • input mode (text vs text+image)
  • model ID from LM Studio

Demonstrated usage

• Pi can be asked to “describe this codebase.”
• The video emphasizes that local models may be slower but can run offline and are free after setup.

Practical comparisons and examples

Sudoku app generation

• Compares:
  • Qwen 3.6 local model
  • Claude Sonnet
• Both reportedly took ~9 minutes under the same prompt (reasoning/thinking contributes to similar latency).

Bug fixing in a larger codebase

• Example: video editor bug:
  • Claude Sonnet fixed in ~45 seconds
  • Local Qwen model took ~2.5 minutes
• Video attributes the difference to faster/stronger handling of reading and understanding large code quickly (hosted models can outperform locally).

Main speakers / sources

• Speaker: Kyle — “Web Dev Simplified”
• Tools/brands referenced:
  • LM Studio
  • Hugging Face
  • VS Code Continue
  • GitHub Copilot (beta/Insiders)
  • Pi (pi.dev)
• Model concepts referenced:
  • Q quantization
  • MOE (Mixture of Experts)
  • OpenAI-compatible endpoints