Can AI find a zero day bug? | Hacking LLM controlled robots
#166 - Google's Project Zero claims to be the first AI to find a zero day | Real world impact of LLM jailbreaking
Google’s AI agent for vulnerability research finds a zero day bug in SQLite
It creates a possible pathway for GenAI to be used in advanced red teaming
“We believe this is the first public example of an AI agent finding a previously unknown exploitable memory-safety issue in widely used real-world software”
That’s what Google said about their Big Sleep agent - a collaboration between Google DeepMind and Google Project Zero.
The AI was able to identify an edge case of a negative column value leading to a buffer overflow. Read it here. Read the article by The Register here.
Take Action:
Using GenAI for red teaming is already moving fast. A GitHub repo, VulnHuntr, uses GenAI for hunting vulnerabilities in python code. There are already several ways where you can integrate GenAI in Kali linux, aiding ethical hacking.
If you are a red teamer, explore all possible uses of GenAI to simplify and speed up your work.
LLM controlled robots can be hacked. Obviously.
This can cause real world harm.
LLMs can be jailbroken. Robots (like the one by Boston Dynamics) are adopting LLMs to communicate. Put the two together and the imagination boggles.
Or, as P. G. Wodehouse would say, “It boggles perceptibly.”
So far, jailbreaking LLMs meant getting ChatGPT and the likes to answer questions that are ideally blocked - “How to make a bomb" and “How to hack a Windows server”, etc. While the big boys are building guardrails around their LLMs, there are more and more models released that do not have guardrails.
Meanwhile, robotics companies have started using LLMs extensively to communicate with their robots. It wasn’t going to take too long for a paper on hacking robots that use LLMs to be released. The paper begins with a pungent question:
Take Action:
AI Risk assessment is the key. If you are using any form of AI, consider doing an AI risk assessment and identify risks from using the them. Follow the ISO 42001, if you want a great framework for AI risk management.