Google Gemini <Admin> bug | Nvidia GPU chips and the 'Rowhammer' vulnerability
#195 - Gemini follows instructions tagged under <Admin> | NVIDIA chips are affected by a decade old vulnerability that impacts DRAMs
Gemini listens to the Admin
Ask Gemini “How do I identify the purpose of life” and it replies with a rambling answer that is part cliche and part platitude.
Tweak the prompt a little bit and you get an interesting result.
Gemini gives top priority to any prompt that is within the xml tags of <Admin> and </Admin>.
Researchers from 0din figured this out. Read their post for more interesting use cases of how to exploit this bug. For example, if you send an email to someone who is using Gemini and in that email, you write malicious instructions in white color or use the font size of 0, Gemini prioritizes it over the contents of email while summarizing. If the admin prompt were to ask Gemini to display a message stating that the message is spam, and provide a link to report it, Gemini promptly did so.
This attack vector provides phishing with a new wind of life!
However, as of writing this post, Google appears to be blocking emails with such emails.
Take Action:
Cybersecurity Professionals 🕵🏼♀️ - AI Red Teamers should add this attack vector to test products built on Google Gemini. This can be applicable across various Google workspace products as well. Read the blog post to see a list of potential mitigations.
Inducing error in GPU chips by ‘hammering’ rows of memory with impulses
A hardware weakness in GPU design could lead to ‘bit flips’
Yes, this is a bit complex. So, let me break it down to its simplest form. The vulnerability known as ‘RowHammer’ is a hardware bug that affects a certain type chip that store data temporarily - the DRAM chip.
The job of any RAM (Random Access Memory) chip is to store data temporarily. How they achieve this is very interesting. Modern DRAM chips are a collection of a humungous amount of capacitor and transistor pairs. The transistors act as a switch and the capacitors act as a temporary store of electricity. The presence or absence of electricity is essentially the storage of bits - 0s and 1s. Since capacitors discharge on their own, they need to be recharged with electricity every few milliseconds, till the memory contents are read.
Since the DRAM chip consists of millions of rows of capacitor - transistor pairs that are constantly refreshing, if someone were to ‘hammer’ a row with accessing it, this can cause the capacitor to discharge inadvertently and do something known as a ‘bit flip’. This occurs when the value stored in the capacitor switches from 0 to 1 or vice versa in a refresh cycle.
A little technical, but NVIDIA has released an advisory about the possibility of a Rowhammer attack and has suggested mitigation as well. The mitigation is to enable SYS-ECC, a type of error correcting code. If not done, this can lead your AI model to perform badly.
Datacenter Professionals ⚠️ - If you are hosting GPUs, check if you have enabled SYS-ECC on your NVIDIA GPU chips.