I ran a controlled test of GLM-5.2 as a code reviewer — and the prompt mattered more than the reasoning effort.
## The setup
– 2 TypeScript backends on Bun + Hono + Drizzle + SQLite
– Test suite written first. 26 planted bugs introduced after.
– Same reviewer: GLM-5.2 in Kilo Code CLI.
– Only variables changed: 3 prompt framings x 3 reasoning effort levels.
## What I found
– **Round 1 (16 easy bugs):** GLM-5.2 caught 13-15 every single run. SQL injection, password hash leaks, missing auth, CSV formula injection, pagination off-by-one, bulk-op bugs — all caught. Prompt framing barely mattered.
– **Round 2 (10 hard product bugs):** caught 4-7. Wildly unpredictable run-to-run.
– **Local bugs** (one-function mistakes, never-firing permission guards, backwards version checks) caught reliably.
– **Cross-route product rules** (archived tasks leaking into search, exports, overdue list) kept getting missed.
– **Strict “block or approve production PR” framing pushed the model into a security checklist** and away from the planted product bugs.
– **Casual + consistency-focused prompts scored better** because they kept the model focused on behavior.
– **Reasoning effort (low/medium/high) made far less difference than prompt wording.**
## Frontier comparison (one pass, no tuning)
– **GPT-5.5:** wrote out a table of which endpoints filtered deleted vs archived rows in one pass.
– **Opus 4.8:** only model to state the exact intended rule on the reopen-a-finished-task bug.
– **GLM-5.2 best run:** 7/10. Ceiling is there. Unpredictability is the gap.
## Practical takeaway
– Lean on GLM-5.2 for single-function bugs.
– Ask for behavior, not “be strict”.
– Use frontier for cross-route consistency.
– Never trust one pass on high-stakes PRs.
Your prompt beats your reasoning effort. Stop paying for more reasoning. Pay for better wording.
#GLM52 #CodeReview #LLMEval #AIEngineering #DevTools #SoftwareEngineering #PromptEngineering #LinkedIn #AICodeReview #EngineeringLeadership
Video Source
About the Author
