I "Vibe Consulted" a Gen AI policy. Here are the details [PART 1]
#183 - A series on how I used Gen AI to create a policy for how to use Gen AI ✨ for consulting
A step by step guide on how you can use Gen AI for cybersecurity GRC
Can Gen AI replace your traditional, garden-variety cybersecurity consultant in creating policies?
Answering the question involves thinking like a consultant 😂 !!. It involves setting up a process with clear objectives and measurable results.
Like everyone else, I have been wondering if Gen AI can take our jobs. What better way to work on this and use the consultant’s approach - create a method, follow it and observe the results.
Ironical, eh? It takes a consultant’s thinking to see if Gen AI can replace a consultant.
I am going to touch on a rather taboo topic. A consultant should never let on how easy it is to create a security policy (or any policy) using off the shelf 📚 AI tools. But like the masked magician, I labored through it, so that you can see for yourself how it's done. It also forces consultants like us to up our game and provide the value we promise when we sell our services.
The method
I'm a consultant at heart. I look for a method to the madness. Getting on to ChatGPT and experimenting with various prompts is what comes naturally. But, that’s not how I can get the right results. I would rather set up a -ahem- process. First step, figure out the objectives, the rules, restrictions and the method.
The Objectives
On first thinking it seemed very simple - create a Gen AI policy using Gen AI. But as I started thinking about it, it got complicated. I did not want to create a generic Gen AI policy. The policy should be specific to a company. And come to think of it, that was not the only objective!
The time required to create this policy should be significantly lesser than what it would take to do the same task without Gen AI. Duh! Of course that’s true.
The most significant part of the objective would be to generate a policy that is fit for purpose and implementable in said organization. The quality of the policy should be good enough to define the justify the huge price tag we charge as consultants.
So, here are the three objectives summarized:
Create an organization specific Gen AI usage policy.
Measure the time taken to ensure that I am actually saving time and not spending more than I would without Gen AI.
Ensure the policy is organization specific and fit for purpose - subjective to measure.
The Rules
With the objectives clearly set, I wanted to set clear rules for doing this. I know I'd never put a client's private information into free AI tools if I were helping them as a consultant. So, I set a few rules for myself:
The Gen AI tool should not compromise the company's privacy. It should not use the data that I am feeding it to learn. I don't want parts of my client's policies appearing as answers on ChatGPT!!
Trial and error cannot be done on a live client project. I have to use a dummy project - probably create a dummy organization or experiment on my own organization.
Once I ensure data privacy, I are allowed to put company internal data into the AI system. I figure this would be necessary if I want to generate good results. I will set up the AI to ask questions and I will answer them, just like a consultant fixes up an obscene number of 'catch-ups' and ends up asking harrowingly basic questions over and over again.
The Tools
Call me old fashioned but I cannot think without a method. It comes from decades of writing “Our Approach” slides for doing various GRC stuff.
Since our first rule is that privacy should not be compromised, I know I cannot use the free versions of tools. I should exclusively use Gen AI tools that clearly mentions that they do not data for learning. I have an embarrassingly large share of Gen AI subscriptions, but many of them are ‘personal’ subscriptions - they use chat data for AI training. I do not want to buy a whole set of corporate subscriptions for this blog post - the board of directors are not very pleased about such expenses, right?
The solution is APIs. Most LLM companies do not learn from API calls. So, this seems to be a good starting point.
I started with OpenRouter.
For those of you who have not used OpenRouter, it is the 'unified interface for LLMs'. It allows access to multiple LLMs. You do not have to subscribe to each one separately. I use OpenRouter extensively anyway, so I have the setup ready. (save for the Gemini that comes bundled with our Google Workspace subscription)
I checked the privacy settings of OpenRouter, made a few changes (open source models were allowed to learn from my prompts, which I disabled) and finally reached a setting like this:
OpenRouter has a decent web based chat, just like ChatGPT or the others, but what is the fun in that? My experiments with truth should also be fun.
I have been using AnythingLLM for a while to experiment with all kinds of GenAI stuff. I have connected an OpenRouter API key in Anything LLM. It seems that the setup is already in place. I also use Fabric-AI, which I quite like, but I use it exclusively through a CLI terminal.
AnythingLLM it is!
I created a space called "Vibe Consulting".
Whew! The setup is complete. All that is left is to do the work 😊. Watch this space for how I go about building a Gen AI policy using Gen AI!
Will definitely be looking out for the results