AI in Bug Reports: When to Use LLMs and When Not To

Transcript

Uh so hey everyone um yeah I I'm really going to reinforce a lot of what people have talked about with AI uh already but from a practical perspective um and uh what you can do really as an individual researcher. So um I'm Alejandro Munos McDonald uh unsafe call on Twitter and I'm a security researcher and lead smart contract triager at Unifi. Uh I read bug reports all day every day uh which means I see both the brilliance and the chaos of web 3 uh security. So today I want to talk about uh LLM usage in bug reports. Uh so why does this matter? Um at the end of the day a good bug report really only does two things. It it tells you something true and it tells you it

clearly. Um early LLM generated reports failed at both. So when chat GBPT first hit uh we were flooded with invalid reports calling out re-entry, potential rounding issues, critical thefts of funds uh from admin functions. Uh very often they're vague with phrases like may lead to um could potentially cause uh might result in with no concrete attack. So if you read a lot of these AI reports, uh they start to turn out more like web 3 mad libs where they're randomly jumbling nice sounding words together. And uh I think what this came from was very heavy waiting towards checklist style uh prompting for vulnerabilities um and not reasoning further than

initial prompts. So this is a trap that new researchers would fall into when they rely on LLMs to generate reports. Um, if you don't understand the bug yourself, uh, you can't tell when the model is actually confidently making stuff up, uh, you don't know when to push back, you don't know what to prompt more for, it's not enough to say something that sounds right for these reports, uh, which are what LLMs are trained to do. Uh, they need to something say something that is right. So, uh, recently they've improved a little bit more in, uh, specificity and moved from like the checklist style prompting and I see more reports that have described, uh, functionality specific to the protocol quite well. Uh, but the biggest weakness hasn't changed

and that's uh, self-verification. So, LLM don't understand uh, often chain state or lack the ability to reason on their own about the true existence of an attack. So um I ran a survey on AI usage um and uh the results were kind of interesting. Uh so 100% of respondents uh said that they were using LLMs uh already in their bug hunting process. Um but only 30% said that they had ever been paid for a bug where AI helped generate the vulnerability. That seems still quite high. Um but AI isn't being used or LLM isn't being used as a magic vulnerability generator by everyone yet.

and uh they're mainly using it to do things like uh learn faster uh do analysis of the code uh vulnerability or pattern or test generation or or just clean up their reports. Uh so not surprisingly the people actually getting paid for AI generated vulnerabilities were actually more senior researchers. So people who are much better at contextualizing the the prompts and can curate uh uh results with a much higher chance of getting paid. Um so I asked also asked uh what were some of the complaints about AI and uh there are quite a few um there is uh some of the examples are false positives uh limited ability to fit large code bases in uh its context no understanding of real world chain data overconfidence none of

this is surprising but uh it's still very real um so what what do people want to see uh well they want to see code uh AI models that can analyze code better models that can understand specific chain architectures and uh uh protocols and the big one is not just uh generate an explanation of vulnerabilities but also check their own work. Um so what we need for that is better standardization of code uh intent and its specifications uh which many people were talking about today um and LLM that can reason better on their own and better tooling that uh can be integrated into agentic workflows um to validate its assumptions. uh we don't need more AI that just writes

reports. Uh we need AI that can verify its own results. Um so if you don't have the infrastructure in place for the LLM to do this on its own, uh you as the security researcher will need to be the contextualizer and the adversarial thinker uh against its output. Right? So um how long before uh we get fully AI auditors? So for audits around 70% of security researchers feel that AI will be able to perform at the level of a human auditor within the next 5 years.

Uh 20% said one year. Bug bounty is a little more cautious answer. Uh still more than half said 5 years or less. Um but if you want value out of LLMs right now um you can use them to learn a new protocol. uh understand unfamiliar code faster uh set up and generate initial frameworks for PC's and also generate leads and identify some entry points for further review. So LLM can help you level up your own skills. Uh and once you become a better researcher uh only then are you really able to take advantage of LLM and vulnerability finding uh for contextualizing your prompts and and valid validating its output. Uh so I'm going to leave you with this. Uh if AI is ever going to independently audit

code at the level of a human researcher, it has to learn something that humans already know and that is how to doubt itself. Yep. Thank you. And uh thank you to these respondents to the uh survey. Awesome.