Full Report
The author of this post was writing a Go implementation of ML-DSA, a post-quantum signature algorithm done by NIST last summer. After 4 days of trying to create the implementation, the code was rejecting some valid signatures. They tried debugging it for several hours but were unable to resolve the issue. So, they asked Claude Code to check it out and left their computer for a bit. The prompt explains what the code does and the issue they were dealing with. They granted it access to run the tests and implement the changes, as well as access to the source code for reading. They topped it off with ultrathink to make it go hard on the problem. To their surprise, an issue popped up! AI excels at well-scoped tasks like this one. The issue was subtle in the math. They had merged HighBits and w1Encode into a single function for using it within Sign. This function was used in Verify(), which had already produced the high bits. So, they were effectively taking the high bits twice. Claude found the issue immediately without using any exploratory tool use! Was this a fluke? They had two bugs prior to this that took an hour to debug. One was around incorrectly hardcoded constants. The other was an encoding being 32 bits instead of 32 bytes. In both cases, it was able to identify the issue through extensive debugging and multiple runs. Still, this was faster than the author of the post! I love seeing use cases of AI and the prompts used. It helps me utilize the tooling better. Thanks for the article!
Analysis Summary