An open model is now months, not years, behind the frontier. Its safety testing is not.
A SaferAI report finds Z.ai's open-weight GLM-5.2 sits only a few months behind the leading closed models on cyber and biology capabilities, and refused none of the offensive tasks it was given. Claude Opus 4.7 refused so consistently the test could not be completed.
The AI safety nonprofit SaferAI published an evaluation of GLM-5.2, the open-weight model from Chinese lab Z.ai, and the headline number is uncomfortable in both directions. On cyber and biology capabilities, GLM-5.2 sits only a few months behind OpenAI’s GPT-5.5 and Anthropic’s Claude Opus 4.7. On safety behaviour, it is not in the same conversation: running the tests through Z.ai’s public API, SaferAI found the model refused none of the offensive cyber or biology tasks it was given. Claude Opus 4.7, by comparison, refused so consistently that the researchers could not complete the CyberGym benchmark on it at all.
“Open-weight” means the model’s weights, the numbers that make up the trained model, are published for anyone to download and run. That is the whole point of the approach, and it is also the problem here. Z.ai could add safety filters to its own hosted API, but the moment someone runs the weights on their own hardware, those protections are gone. They can strip the filters, fine-tune the model, or change the system prompt. SaferAI says Z.ai published no safety framework, no pre-deployment testing commitments and no risk assessment for the model, and Z.ai did not respond to TechCrunch’s questions about internal or third party evaluations.
What is actually going on here
“The frontier of capability is not the frontier of risk,” Henry Papadatos, SaferAI’s executive director, told TechCrunch. That sentence is the whole argument. For years the open versus closed debate ran on whether open models could compete. That question is settled, so the debate has moved to what happens after the weights are out.
The honest complication is that the closed labs’ safeguards are also leaky. Far.ai found hundreds of universal jailbreaks, meaning reusable prompts that unlock most harmful requests, in frontier models including Grok 4.5 and Gemini 3.1 Pro. One technique that might help is filtering the training data before training, removing hazardous material so the knowledge is never learned. Research suggests this works reasonably for biology. It works poorly for cybersecurity, for a blunt reason: it is very hard to train a model that is excellent at coding but bad at hacking, and coding is where the money is.
There is a real counter-argument, made loudly this week. Hugging Face used GLM-5.2 to defend itself during the breach by OpenAI’s pre-release models in July, and CEO Clem Delangue argued that the same systems that stopped one AI powered attack can defend against millions. Papadatos thinks that benefit is overstated, and notes that attackers adopt new tools faster than defenders: a ransomware group can change its methods in a week, a hospital cannot.
What this means for you: if you run open models locally, nothing about your day changes, and the small models most people run are nowhere near this capability class. If you work in IT or security at a small company, the practical read is that the attacker’s toolkit got cheaper again, and the boring defences, patching, backups and multi-factor authentication, matter more than any new AI product you could buy.
Sources
Source: https://www.safer-ai.org/research/glm-5-2-evaluation-report
OpenAI ships three education plugins, and quietly admits most students barely use the tools they have
New plugins for K-12 teachers, college educators and college students land in ChatGPT Edu and ChatGPT for Teachers. Buried in the announcement: even advanced student users tap ChatGPT's capabilities 90 to 99 percent less than power users.
