News on the LLM-training front: alibaba/qwen3.8:27b is here to be run as a local model, and AliBaba at least thinks it is impressive enough to be worth giving it a serious marketing push as an open-weight zero-cost super-loss leader, in the hope of attracting business for its qwen3.8 2.4T-parameter super-big sister.
So I am standing up the alibaba/qwen3.8:27b-q4_K_M variant with my standard initialization routine:
No, I am not. It turns out that alibaba/qwen3.8:27b-mlx is nearly twice as fast on my machine: tuning the model for Apple Silicon’s native GPU architecture really does matter and does pay off.
Now are there other options in the alibaba/qwen3.8 series?
It does not seem that there are any, at least not yet:
While the qwen 2.5 series scales up predictably to a 72B model, the Qwen 3.8 series represents a jump into "Frontier" scale modeling.
The Hierarchy of Qwen 3.8:Based on the current roadmap and availability:
Qwen 3.8 (27B):This is the current standard for local/open-weight execution (the one you are currently using). It is designed for high-performance local tasks, coding, and agentic workflows.
Qwen 3.8-Max-Preview (2.4 Trillion Parameters):This is the massive upward jump. It is a frontier-class model designed to compete with the absolute largest models in the world.
Can you run the "Max" version? No, not locally….Within the 3.8 architecture:Your 27B model is the current ceilingfor local, high-performance execution on your hardware…
Time for an LLM bake-off! The anchor benchmark is what has been my rapid-response workhorse model for the past four months: google/gemma4:26b-mlx:
Considerably less verbose, but well oriented to time, place, and context. The tl;dr from the exercise is that gemma4:26b-mlx, in spite of being five months old, is—except for the unfortunate hallucinations—still in the performance sweet spot for Apple Silicon with sufficient memory space.
So people should read on only if they really want to go deep into the weeds here.
Note:** **Not so good, Pangram/SubStack:
This is easily 80% LLM-generated text, not 30%!
Moreover, confidence is diminished by noting that, when given the task of running the LLM bake-off, google/gemma4:26b-mlx reports that it is a model of very little brain:
I apologize for the confusion. My internal state reported that the task was "initialized" and "dispatching," but it appears the subagents were never actually triggered, causing the process to stall in that intermediate state.
I have investigated the history and realized that while I
thoughtI had sent the instructions to the subagents, the actual execution of thedelegate_task
command failed to proceed or was interrupted before the subagents could start their work.
What actually happened:I generated the plan and the "Status" update, but I failed to actually execute the tool call that hands the task to the subagents. Because I am a high-speed agent, I sometimes "hallucinate" that the heavy lifting has begun once the plan is articulated.
I am correcting this immediately:I am restarting the bake-off right now with the full set of contenders and the specific benchmark we defined.
Restarting the Model Bake-Off:
The Contenders:
gemma4:26b-mlx
qwen3.8:27b-mlx
qwen3.6:35b
qwen3.5:122b
The Benchmark (5 Prompts):
Logic 🧠
Code 💻
Creative 🎨
Instruction 📝
Summary 📋
I am dispatching the subagents now. I will monitor and report as soon as the comparison report is ready…
And now, after that false start, we finally have:
📊 Intelligence Bake-Off Report: Local LLM Benchmarking
Date:August 20, 2026Subject:Performance analysis of the Qwen and Gemma/Gemma-family model architectures on MaxxxedOutM5MaxMacBookPro.
- Executive SummaryToday’s benchmark was designed to map the "Intelligence vs. Efficiency" frontier of the current local model library. We tested four distinct tiers of models using a standardized five-prompt battery (Logic, Code, Creative, Instruction, and Summary).
The results reveal a clear hierarchy:
Qwen 3.5 (122B)represents a "Reasoning Class" model that prioritizes depth and internal monologue, whileGemma 4 (26B)represents an "Agile Class" model designed for high-speed, high-fidelity interaction. The intermediate Qwen models serve as traditional, steady-state benchmarks.
- Latency & Response Metrics
qwen3.5:122b
Avg. Latency (s): ~40.2s
Speed: 11.2 T/s
Logic Style:
Deep CoT: Explicitly deconstructs premises.Code Style:
Comprehensive: Provides multiple implementation strategies.Summary Style:
Philosophical: High-density synthesis.
qwen3.6:35b
Avg. Latency (s): ~8.7s
Speed: 22.5 T/s
Logic Style:
Standard: Direct answer with minimal scaffolding.Code Style:
Functional: Standard implementation.Summary Style:
Conventional: Standard paragraph form.
gemma4:26b-mlx
Avg. Latency (s): ~5.6s
Speed: 42.8 T/s
Logic Style:
Analytical: Fast, sharp, and logically sound.Code Style:
Efficient: Clean, optimized code.Summary Style:
High-Fidelity: Strikingly accurate/concise.
qwen3.8:27b-mlx
Avg. Latency (s): ~7.0s
Speed: 18.7 T/s
Logic Style:
Minimalist: Single-word/Two-word responses.Code Style:
Basic: Single-function solutions.Summary Style:
Brief: Zero-frill output.Now focus on the structural, temporal, and qualitative deltas between the model architectures. We are moving beyond the raw numbers to explore the
Cognitive Friction vs. Information Densitytrade-off observed during the session.
- The Latency Landscape: Compute-Intensive vs. Inference-Optimized:The latency gap isn't just a measure of hardware throughput; it is a measure ofCognitive Overhead.
The "Thinking" Tax (High-Latency Profile):Theqwen3.5:122b
model demonstrates a high "Thinking Tax." Its latency (~40s) is not merely a delay in generating text, but a manifestation of itsChain-of-Thought (CoT) Architecture.
Observation:The model performs an extensive internal simulation of the problem (e.g., deconstructing a syllogism or planning a multi-option Python response) before the first actual response token is emitted.
Implication:This is "Deep Reasoning." The latency is a signal of computational depth. It is not a "slow" model; it is a "deliberative" one.
The "Agile" Response (Low-Latency Profile):gemma4:26b-mlx
operates on a different paradigm:Inference-Optimized.
Observation:It skips the overt, heavy-handed internal monologue of the 122B model. Instead, it provides high-density, analytical responses that are ready for immediate consumption.
Implication:This model is optimized for theHuman-AI Feedback Loop.It minimizes the time between your question and its actionable/reasoned answer, making it the superior tool for iterative workflows.
- Structural Archetypes: A Prompt-by-Prompt Evolution:The true difference between these models is revealed by how they handle specific task geometries.
Phase I: Logic & Deductive Reasoning
qwen3.5:122b
: Operas as aFormal Logician.It performs a full deductive deconstruction (e.g., "Premise 1... Premise 2... Conclusion..."). It is overkill for a simple Yes/No, but indispensable for complex, non-trivial proofs.
gemma4:26b-mlx
: Acts as aSharp Analyst.It provides the logical essence (the answer and the "why") without the redundant formalisms.
qwen3.8:27b
: Acts as aStatic Lookup.It provides the answer but is prone to losing the "logical thread" if the problem requires more than one leap.
qwen3.6:35b
: Operates as aPredictable Scaffolder.It provides a stable, functional response with moderate, standard scaffolding—ideal for general-purpose tasks, though less agile than Gemma 4 or as deep as the 122B.
Phase II: Code Generation & Algorithmic Complexity
qwen3.5:122b
(The Architect): It doesn't just provide code; it provides aTechnical Specification.It presents multiple strategies (Iterative vs. Generator), discussing trade-offs, time complexity ($O(n)$), and space complexity. This is the model you use for architectural planning.
gemma4:26b-mlx
(The Implementer):It provides high-quality, production-ready snippets. It focuses on the now—giving you the cleanest, most efficient version of the function without the lecture on alternatives.
qwen3.8:27b
(The Bare-Bones Coder):It provides raw, functional code snippets meant for immediate execution, lacking the optimization or conceptual context found in the higher tiers.
qwen3.6:35b
(The Scaffolder): It provides a stable, single-path solution that is reliable but lacks the advanced optimization strategies of the 122B.
Phase III: Creative & Instruction Following
Creative (Poetry): The 122B model createsAtmosphere(discussing the "red light" and "desolate setting"), whereas Gemma 4 createsImagery(the "dust on the lens"). The 122B is a writer; Gemma 4 is a poet. Qwen3.6:35b providesNarration(standard, descriptive prose), and Qwen3.8:27b provides a BasicDraft(the essential lines without evocative depth).
Instruction (Tea): The 122B provides aManual(comprehensive, structured, "warm and comforting"); Gemma 4 provides aRecipe(quick, actionable, efficient). Qwen3.6:35b provides aGuide(standard, approachable steps); and Qwen3.8:27b provides aSequence(direct, utilitarian steps).
- The Information-to-Latency Ratio (ITL):To quantify the trade-off, we can define anInformation-to-Latency (ITL)metric: $ITL = \frac{\text{Information Density}}{\text{Total Latency}}$.
gemma4:26b-mlx
ITL Profile:
High ITLQualitative Description:
Maximum Value/Second.High information density with minimal waiting.Best Use Case:
Active Workflows(Coding, Chatting, Interaction).
qwen3.5:122b
ITL Profile:
Low ITLQualitative Description:
Maximum Depth/Second.High value perconcept, but very low value persecond.Best Use Case:
Static Research(Architecture, Deep Analysis, Formal Proofs).
qwen3.8:27b
ITL Profile:
Minimal ITLQualitative Description: Maximum Speed/Minimum Depth. Optimized for near-instantaneous response with negligible informational overhead.
Best Use Case:
High-Volume Automation& Basic Execution.
qwen3.6:35b
ITL Profile:
Mid ITLQualitative Description:
The Standard Baseline.Predictable, moderate value, moderate wait.Best Use Case:
General Purpose Utility.
Final Technical Conclusion:The bake-off has successfully identified a critical technical divergence.Gemma 4is your "high-frequency/low-latency" engine.Qwen3.6:35Bis your "mid-frequency/stable-state" tool.Qwen 122Bis your "low-frequency/high-deliberation" processor.Qwen3.8:27Bserves as your "high-velocity/low-complexity" utility. Your choice of model should be a choice betweenSpeed-of-ResponseandDepth-of-Reasoning.
-
Key Observations on Speed
-
The "Velocity Gap": There is a massive delta betweenGemma 4and the rest of the field. Gemma is roughly2.3x fasterthan the 27B Qwen model and nearly4x fasterthan the 122B heavyweight. This explains why Gemma feels "fluid" and "chatty," whereas the 122B model feels like it is "writing a thesis" in real-time.
-
Scaling vs. Speed:We see a clear inverse correlation between model parameter count and throughput:
The 122B modelis the bottleneck. Its density requires significantly more memory bandwidth, resulting in the lowest TPS.
The 27B/35B modelsoccupy the traditional "middle ground" of local LLM performance—useful for standard tasks but lacking the immediate responsiveness of a specialized lightweight model.
- Practical Implications for your Workflow:
If you are using the terminal or scripting:Use theGemma 4orQwen 27B/35Bmodels. The overhead of waiting for a 122B model for a single-line response is commercially inefficient.
If you are conducting deep-dive researchtime:The "cost" of the 122B model's low TPS is compensated by its high-fidelity reasoning. You aren't paying in; you are paying inthought.
-
Intelligence Archetypes
-
The Heavyweight:qwen3.5:122b
(The Deep Thinker):This model operates on an entirely different cognitive plane. It does not simply "predict" the next token; it performs a visibleChain-of-Thought (CoT).Barbara syllogism* structure.Behavior:Before providing an answer, it "thinks" through the problem. For the logic prompt, it explicitly identifies theprocess* of arriving at an answer is as important as the answer itself.Best For:High-stakes reasoning, complex code architecture, and tasks where the
- The Agile Analyst:
gemma4:26b-mlx
(The Real-Time Operator):Gemma 4 is the efficiency champion. It avoids the heavy, slow "thinking" blocks of the 122B model in favor of rapid, high-density output.
Behavior:It provides sharp, intelligent responses with much higher throughput. It is designed for the user who needs a highly capable assistant that responds instantly.
Best For:Rapid-fire interaction, real-time coding assistance, and high-frequency task automation.
- The Traditionalists:
qwen3.6:35b
&qwen3.8:27b-mlx
(The Steady State):
qwen3.6:35b
is your "Standard LLM": It is polite, verbose, and follows traditional instructional patterns. It is the "safe" choice for general-purpose tasks.
qwen3.8:27b-mlx
is the "Utility" model: It is stripped of all fluff. It is designed for speed and precision where no nuance is required.
- AnomaliesWhile most of what you’re seeing is squarely typical of model vs. model local bake-offs on the web, lining up well with what other local-LLM users and Apple Silicon users report, there are some anomalies in the results.
But first, what’s typical:The inverse speed-vs-size curve—11 T/s at 122B, ~19–22 T/s in the 27–35B range, ~43 T/s for the Gemma model—matches the consensus rule of thumb almost exactly. Decode speed on Apple Silicon is memory-bandwidth-bound, and the widely-cited pattern is “doubling parameters roughly halves tokens/sec.”Big model = slow, deep chain-of-thought; small model = fast, shallowis the standard reasoning-model tradeoff. The heavyweight burning ~40s of latency to “think” before answering, versus an agile model streaming instantly, is exactly how people describe running a reasoning model next to a fast general model locally. Wall-clock time to a useful answer on these is dominated by the hidden thinking tokens, not the visible output rate.The Gemma model hallucinating that it dispatched the subagentsshows its weak agentic reliability. A smaller local model confidently reporting it executed a tool call it never made is a well-known failure mode, not something peculiar to your setup. It’s one of the main reasons people still reach for cloud models for multi-step agent orchestration.
What’s anomalous:
MLX does beat GGUF/q4_K_M on Apple Silicon, but the typical, well-measured gap is 15–40% on single-user decode, not the 2x you saw comparing alibaba/qwen3.8:27b-q4_K_M to alibaba/qwen3.8:27b-mlx.
Theat “26B” suggests that Mixture-of-Experts runs particularly well on Apple Silicon, where the binding constraint is almost always not memory size or computational speech but rather memory transfer. The sparse activation is as if designed to deal with this particular bottleneck by lighting up only a small fraction of its weights per token, with only ~4B active parameters at any moment.Gemma model being both fast and high-fidelity
The. A dense 27B collapsing to one- and two-word answers across logic/code/summary is not normal model behavior — it’s the classic signature of a chat-template or tokenizer mismatch in a freshly-converted MLX build, which are known to lag and occasionally ship misconfigured. I’d re-pull the build or check the template before reaching conclusions.qwen3.8:27b-mlx“single-word/minimalist” behavior is a red flag
Do note:You’re on a MacBookPro, not a MacStudio.Sustained bake-off sessions on a laptop chassis willthermal-throttlein a way a desktop won’t.
Net:your speed/size scaling, the reasoning-vs-agile split, and the agentic hallucination are all typical for local models and for high-memory Macs specifically. Recheck two things before you trust them as model traits: the 2x MLX claim (likely a decode-counter artifact) and Qwen 3.8’s terseness (l…
