Finding secrets in obscure places | What is GenAI being misused for?
#152 - A security researcher looks in places others haven't and finds secrets | As usual Google has the answers to how GenAI is being misused
You can still find published secrets
You just need to know where to look
Just when you thought you had all the secret scanning mechanisms in place - you have trained your developers to not hardcode access keys, drilled it into their brains to use environment variables and you had purchased the nifty secret scanning tool from your favorite repo provider - this article comes along to tell you that there’s more to secrets being exposed than code repos and public cloud buckets.
This article from the Wired shows how security researcher Bill Demirkapi looked for secrets in obscure places - one of them being the retro hunt feature of VirusTotal. This allows the searcher to look for data using YARA rules. Bill was able to find around 15000 secrets using this little feature.
Another interesting thing that Bill did was look up dangling subdomains. What are dangling subdomains? Read this Microsoft article for a simple explanation. In short, when a resource behind a subdomain is de-provisioned, but the corresponding DNS entry is not removed, an attacker can use that pointer and point to their own malicious url, leading to a subdomain takeover. This diagram explains it well:
Take Action:
If you are looking for secrets that you might have uploaded, add the retrohunt search to your scanning.
Train your security researchers to ensure files uploaded to VirusTotal and similar such sites do not contain confidential data or metadata
Review your DNS records frequently to ensure that dangling subdomains are not left behind
How is GenAI being misused?
Google Deepmind has published a paper about it
Multimodal generative AI (GenAI that can generate images, videos, text, etc.) can be misused in multiple ways. This research by Google and Jigsaw traced GenAI based misuse instances for a period slightly over a year and published their results. In the screengrab of the graph above, you will see that the top 6 misuses are not related to the AI system, but the use of the AI system.
In fact, the top 5 uses seem to be related to disinformation.
Impersonation - pretending to be someone else
Scaling and Amplification - using GenAI to scale your messages using GenAI to mass email people
Falsification - Fabrication, falsification, etc.
Sockpuppeting - creating fake profiles for specific purposes
Appropriated Likeness - changing the expressions of a person, etc.
Only after all these disinformation related misuses do we get to porn. NCII stands of Non consensual intimate imagery. This is your celebrity porn videos etc.
Then, we have the first attack on the AI system - prompt injection.
Read the full paper here.
Take Action:
If you are creating or deploying AI systems, remember that the AI impact assessment is far more important than the AI risk assessment. Conduct a thorough impact assessment to understand what the possible misuses of the AI system might be and identify mitigations for the same.
Consider implementing AI governance standards such as NIST-AI-RMF or ISO 42001