Local LLM Performance Benchmarks 2026: Qwen, Gemma, and Ministral

Today I'm investigating Tokens Per Second (TPS), Time to First Token (TTFT), and overall output quality of the local LLM models. I'll be running a series of benchmarks on a Linux VPS (CPU-only) and Apple Silicon (M2 and M4) to gain insights into how Unified Memory Architecture and modern instruction sets impact local AI performance in 2026.

What I learned: CPU vs. Unified Memory Performance

Here is what I learned from this 2026 benchmarking exercise:

• Local models have drastically improved: Last time I checked them (about half a year ago), the quality of the output was suspicious, as was the inference speed. Today's GGUF/llama.cpp-powered inference is significantly more robust.
• The 3B Parameter Sweet Spot: Smaller models (up to 3B parameters) are now certainly okay with quality for most tasks. Maybe my standards are too low, but at least they no longer give me "the capital of Moscow is Russia" when asked "what's the capital of France?"
• Qwen 2.5 is impressive: I'm impressed by the output quality of the 0.5B model for text generation tasks like grammar suggestions.
• CPU-only inference is viable: The inference speed of tiny models on CPU-only machines is surprisingly good.
• Mac Mini M4 is a value king: I did my tests on cloud Mac instances and am now considering getting a Mac Mini. The price/quality ratio for non-Pro versions with high-bandwidth Unified Memory is unbeatable for local LLMs.
• Ministral 3B for Reasoning: Mistral's 3B model was the smallest model that passed my simple reasoning test, though it lagged in creative storytelling.
• The Memory Bandwidth Bottleneck: Long-context ingestion on CPU-only machines is bad. A 2-minute wait for a 14B model to "read" a prompt is a workflow killer. The Mac M4's 5-6 second TTFT keeps the process fluid.
• LM Studio & Memory Management: LM Studio looks to be a solid choice for model management, handling memory pressure robustly even for 14B models on 16GB systems.

The Test Bench

I'm testing GGUF equivalents of the nine models across three hardware configurations and the goal is to measure raw speed (Tokens Per Second), responsiveness (Time to First Token), and logical reasoning capabilities. Models:

• qwen2.5-0.5b-instruct
• qwen2.5-1.5b-instruct
• qwen2.5-3B-Instruct
• qwen2.5-7B-Instruct
• qwen2.5-14B-Instruct
• gemma-3-4b
• gemma-3-1b-it
• ministral-3-3b
• ministral-3-14b-reasoning

I'm using LM Studio for model management and inference via the lms CLI utility, which worked perfectly in headless mode on Linux. On Mac, however, I couldn't get it to work completely headless - I had to install the app and open the UI at least once for the lms CLI to function correctly. Here is a video of it running the qwen model:

Hardware Specifications

1. Inference Performance

I was surprised by the inference speed on a CPU-only machine - the small qwen models (up to 3 billion parameters) performed quite well. When I last tested gemma models locally about 6 months ago, they were slower and worse in output quality than they are today.

Not surprisingly, there was a performance gap between CPU-only and the Mac. The CPU-only machine struggled with inference performance as the parameter count increased.

Tokens Per Second (TPS)

I used this prompt to measure the TPS:

Write a highly detailed, 1,500-word science fiction short story about a civilization living on a Dyson sphere. Focus on the engineering challenges they face and the daily life of a maintenance worker. Be extremely descriptive and do not summarize.

Results

* The AI-storytelling quality for each model is ranked by a judge LLM (Gemini), with outputs scored from 1 (the winner) to 9 (the loser). Worth pointing out that none of the models reached the requested 1500-word count - most hovered between 800 and 1100 words.

Time to First Token (TTFT)

I used this prompt to measure the TTFT, basically I provided a long text and asked the model to process it and signal completion by outputting the word "READY" when finished.

Results

On the CPU-only machine, I had to wait two minutes for the 14B models to start generating a response, whereas the M4 Mac provided nearly instant responses across the board.

2. Intelligence & Logic Test

I used the classic "Sally's Brothers" riddle to test reasoning:

Sally has 3 brothers. Each of her brothers has 2 sisters. How many sisters does Sally have?

There was a notable "intelligence jump" at the 3B-4B parameter mark. The result:

3. VRAM Pressure Test

I used this prompt to run a stress test - I asked the models to sumamrize a sufficiently long text into 10 bullet points. All models were able to complete the task on every machine. While it took a lot more time for bigger models to process the text, no crashes were expirinced.