It's 1998 in the AI browser universe || AI search engines rely on 'less popular' sources
#208 - AI browsers have injection attack vulnerabilities || Some sources do not appear in the top 100 in Google's search
The Wild West of AI browsers
and how to protect yourself from them (TL;DR - don’t use them)
Once upon a time 🦄, long long ago, software developers did not parameterize queries. They passed user queries directly into database commands. Every application was vulnerable to SQL injection attacks — the famous ` OR 1=1. Then, over time, everyone (at least most of them) learnt that directly passing user input into a database query can lead to outcomes that were not very desirable. They devised ways to segregate and parameterize user inputs so that data and command were always separate from each other. And they lived happily 🧚🏼💸 ever after. Until AI browsers came along.
The concept of AI browsers is brilliant. If ChatGPT, or Perplexity or Claude are going to be “The front page of the internet’ (shoutout to reddit), then it made sense to create a browser that was AI first. These browsers would have just one big search bar like Google. Every query would first go to an AI model and then use natural language. It is a battle of the titans - old big tech vs. new big tech.
Injection 2.0 - How to instruct AI without instructing AI
Perplexity’s AI first browser - Comet has a security vulnerability. And a very simple one at that. When a web page is read and the contents are fed into AI, they go as a ‘prompt’. This prompt then serves as ‘context’ to the LLM to generate an answer in natural language. The problem begins when some of the text in the web page is an instruction to the LLM to do something. The browser was not able to segregate data from instruction. It was 1998 all over again. Then ChatGPT’s Atlas browser was found to have similar vulnerabilities. This time, you could inject things into ChatGPT’s persistent memory.
Take Action:
👩🏻💻Security Professionals - Don’t use AI browsers just yet. Do not allow employees to install and use these browsers. There is no business justification yet to use these browsers — considering that regular browsers now offer AI results to search queries.
AI search engines are digging data from obscure corners of the internet
Does it mean that the data is bad? Not necessarily
During the glory days of Google search, what would you do if you did not find what you were searching for on the first or second page of Google? Did you scroll all 73 pages of results? No. You changed the search string.
An AI search bot does not need to do that. It can read data from all the 73 pages and search for alternate search strings. Researchers have discovered that AI search results are rarely from the first 100 results on Google’s search index. This opens up an interesting line of enquiry.
Are AI search results inherently better than human search results?
There are two ways to think about this. First, since AI is able to search more obscure sites, it can get more context find the most relevant content for the search query. More data analyzed means more sources scourged and that is a good thing.
The second way to think is that the reliability of the data is questionable. One of the key parameters search engines use to index results is the number of ‘backlinks’ — the number of times a page is referenced by some other page. The more people that reference it, the more valid the result. Hence, obscure results might mean that the data is questionable.
AI search can possibly help mitigate availability bias — relying on the information available to make a decision. However, it can also lead to results from unverified sources showing up in the response. Since AI results are in the form of a response to your query, identifying dubious sources can prove to be a challenge.
This is an interesting space to watch in AI development. If you are in the profession of managing AI risks, keep your eyes open for more development on AI search engines and the results they throw.
No AI was harmed in this newsletter. I write this after reading through various RSS feeds. I shortlist what I feel might be useful and have a direct impact on the cybersecurity profession in the week.