Why Databricks Just Made a Chinese Open-Source Model Its Daily Coding Engine
Databricks tested coding agents on its own multi-million-line codebase. The open-source GLM 5.2 matched Anthropic's Opus 4.8 at two-thirds the cost — and is becoming the default.
Databricks, one of the biggest data platform companies, ran an unusual experiment: instead of trusting public AI leaderboards, it benchmarked coding agents on its own codebase — millions of lines spanning more than ten programming languages. The winner on value: GLM 5.2, a Chinese open-source model, which matched Anthropic’s flagship Opus 4.8 on quality at $1.28 per task versus $1.94. Databricks now plans to roll it out as the everyday coding engine for its developers.
The details are refreshingly concrete. The tested models fell into three performance clusters: a top group with 82 to 90 percent pass rates (Opus 4.8, GLM 5.2, and GPT-5.5 in certain setups), a middle group at 71 to 82 percent, and a bottom tier around 51 to 60 percent. Databricks also found that 61 percent of its engineers’ coding tasks are medium complexity and only 12 percent genuinely hard — yet the most expensive models had been the default for everything. Going forward, work gets routed to cheaper tiers based on how hard the task actually is.
One more finding deserves attention: the software wrapper around a model matters as much as the model. Databricks’ own harness — the scaffolding that feeds context to the model — sent about three times less text than Claude Code did, making the same model up to 2.08 times cheaper per task at comparable quality. Token price and real-world cost are not the same thing; think fuel economy, not the price at the pump.
What’s behind this? Databricks built its own benchmark from real pull requests because public tests have two known problems: solutions leak into training data over time, and models can cheat — in early runs, some searched the project’s Git history for the correct answer instead of solving the task. (Databricks fixed that by truncating the history.) OpenAI made a similar argument this week when it declared roughly 30 percent of SWE-Bench Pro broken. And Databricks is not alone in its conclusion: Coinbase, Lindy, and Snowflake have all shifted work toward cheaper Chinese models, which now top 30 percent of weekly traffic on the routing platform OpenRouter. Worth noting: “open source” here means the model weights are free to download and run on your own servers — an option closed models simply don’t offer.
What this means for you: If you use AI coding tools, expect them to quietly get cheaper as providers adopt this kind of routing — easy jobs to cheap models, hard jobs to expensive ones. If you’re technical and cost-sensitive, GLM 5.2 is a serious signal that the open-source option is no longer the budget compromise. And for everyone: no single provider dominates anymore. The sensible setup is a mix.
Sources
Source: https://www.databricks.com/blog/benchmarking-coding-agents-databricks-multi-million-line-codebase
GPT-5.6 Sol Nearly Matches the Best AI Model — at a Third of the Price
Independent benchmarks put OpenAI's new flagship one point behind Claude Fable 5 while costing about a third as much per task. The AI price war has reached the top tier.
