I Turned My Spare Phones Into a Wireless GPU for AI Agents (It Actually Works)

Summary

The creator demonstrates Private LLM’s Phone Cluster, a feature that combines multiple phones and tablets to run an AI model that none of the devices could handle alone.

Private LLM runs AI models locally on a phone. It also supports image generation, image understanding, queued prompts, and a model library that shows which models a device can run.

A single phone can comfortably run models of around 7–8 billion parameters, depending on the model and quantization. The demonstrated 26-billion-parameter Gemma 4 model is too large for any of the devices to run individually.

How the Cluster Works

To set up a cluster, the user chooses a powerful phone as the master, sets the other devices as workers, and pairs them by scanning QR codes. All devices must be connected to the same Wi-Fi network.

The app distributes portions of the model across devices according to their available memory and processing power. In the demonstration, an iPhone handles 38% of the model, an iPad 32%, a Samsung S24 21%, and a Pixel 9%.

Rather than running separate models, the devices process the model’s layers in sequence, passing data from one device to the next. The cluster generates about 4–5 tokens per second in the demonstration.

Computer Integration

The app can expose the cluster through an OpenAI-compatible API on the local network. The creator connects it to AnythingLLM on a MacBook and to the Continue extension in VS Code, using the phones to power chat and coding tasks while the laptop serves as the interface.

To connect a computer application, enable network access and start the server in the app, then configure the application with the server address and API key.

Performance and Trade-offs

The cluster works, but generation is slow compared with a powerful single computer. Adding devices also means more Wi-Fi communication.

The main benefit is that the cluster can run a larger model locally across devices that could not run it individually.

Setup Guide

1. Choose a master device.
2. Pair worker devices by scanning QR codes.
3. Connect all devices to the same Wi-Fi network.
4. Load a model.

Main Speaker and Source

The video’s creator and demonstrator is from the orailnoor channel.