Flint 40B A2.5B tested - 16GB Local LLM setup

Summary

Luke’s Dev Lab tests Flint, a model from Recursive Superintelligence. Flint is a 40B-parameter mixture-of-experts model with 2.5B active parameters. The review uses an Unsloth GGUF Q4_K_XL quantization of about 25 GB through llama.cpp. Because the model exceeds the capacity of the 16 GB GPU, the test uses partial GPU offloading.

Setup and speed

The main test setup uses a system with a 16 GB GPU. The reviewer also runs speed tests on an RTX 5090.

At a 128K-token context setting, generation speed was just under 40 tokens per second on the 16 GB system and about 206 tokens per second on the RTX 5090. Overall, the reviewer found Flint notably fast for its apparent capability.

Benchmarks and task tests

• Context and memory retrieval: In a 256K-context retrieval test, Flint found the target information in all three trials at 0% and 25% depth, but only once out of three trials at 50% and 75%. The reviewer considers this a significant weakness, particularly for long sessions and repeated context compression.
• HumanEval Remix: Scored 90/100 on Python tasks, answering all questions.
• Reasoning: Scored 35/48 (73%). Flint handled easy tasks and did reasonably well on difficult ones, but missed some intermediate questions. The reviewer characterized the overall result as below average.
• Kanban frontend app: Eventually produced a functional-looking interface with draggable cards and columns. However, it required many follow-up prompts, troubleshooting, and context use. The reviewer found the process frustrating and the result less impressive than the final interface alone might suggest.
• Sand Physics: Worked well, including interactions among sand, water, and acid, without extra prompts or context compression.
• Dungeon Crawler: Produced a working result with raycasting and dungeon regeneration. It used relatively little context and required no extra prompts. The reviewer notes that these tasks had been public for several months, so possible familiarity from training data is uncertain.
• Blender: Created a star-lantern model with generally good detailing and lighting. It needed one follow-up prompt to provide the requested screenshots, but was completed without context compression. The reviewer considered this one of Flint’s strongest results.
• Godot game: Failed on a much more complex 3D game task. The game was largely unusable, the response was cut off, and repeated context compression led to a loop the reviewer could not break.

Overall assessment

Flint’s main strength is speed, with promising results on Python, Sand Physics, Dungeon Crawler, and especially the Blender task. Its main weakness is reliability over long or complex sessions: retrieval was inconsistent in the middle of long contexts, and repeated compression coincided with a severe failure on the Godot task.

The reviewer advises caution when using Flint for extended agent workflows unless the surrounding system has strong context management. The model may be better suited to shorter tasks, simpler frontend work, or some 3D-modeling tasks.

Guides, tools, and products mentioned

• Testing setup: The reviewer runs local models with llama.cpp and says configuration details are available on the Luke’s Dev Lab website.
• Abacus AI ChatLLM: A sponsored segment describes a service for accessing multiple AI models, generating images and video, conducting research, creating presentations, and using an agent workspace for development and related tasks. These are promotional claims from the video, not results of the Flint tests.

Main speaker/source: Luke’s Dev Lab, the video’s reviewer and tester.