AI for Compliance Automation - Compliance AI Platform
Disclaimer: All data, records, policies, users, security events, compliance findings, and other information presented in this project are entirely fictitious and have been created solely for demonstration and experimentation purposes. No real organizational data, proprietary information, or personally identifiable information (PII) is included. This project is intended to showcase the design and implementation of an AI-powered compliance platform and should not be interpreted as representing the security posture, policies, or operations of any real organization.
Ensuring that an organization is safe from cyberthreats requires strong policy and compliance guidelines and employees who are knowledgeable on these standards. Guaranteeing that all of the people and parts within the organization are complying is a challenging task that can be remediated with artificial intelligence. A large language model (LLM) can be particularly helpful with answering questions and compiling data. To combat the challenges of compliance security, I have put together a Compliance AI Platform that can perform a variety of compliance tasks that can assist users with their inquiries and provide clearer transparency among access and auditing.The platform uses Retrieval-Augmented Generation (RAG) to help the LLM find the specific data that it needs.This can significantly reduce the amount of time needed for a user to get the understanding that they need when searching for specific information. With the power of AI, an organization can significantly enhance its security through better access to compliance and policy data using the Compliance AI Platform.
The application has ten pages in total. The home page provides a broad overview of compliance metrics, recent assessments, security alerts, and key findings. It acts as a synopsis, giving the user a rundown of the ongoing security and compliance posture.
The next page is Policy QA, which assists with retrieving policy data. Users can ask the system a policy question and receive a verified response.
The Control Testing page is responsible for evaluating whether a security or compliance control is running successfully. When a control ID is provided, a status of pass/fail/warning assessment, evidence, and a recommendation are provided.
The Compliance Data page provides the user with a place to look over the evidence that backs the compliance activities, specifically with access reviews and policy exceptions. These tables inform the user of details such as who has access to what, what exceptions are allowed for policy requirements, and what has been approved.
The Audit Log page keeps a record of access information within the system. The log records who accessed system resources, when the access occurred, and what actions were performed. The AI Workspace page provides users with a chatbot for specific queries and requests, such as questions about policies or to create compliance assessments. RAG is utilized to assist the LLM in finding responses.
The Assessment History page lists the results of previously completed AI control assessments. The tables details when the evaluation happened, what was specifically examined, and what the outcome was.
The Evidence Explorer page allows users to browse and investigate compliance evidence that the organization has gathered. The page displays all of these records in a table and allows the users to scroll through each record to observe more details.
The System Health page presents how well the Compliance AI platform itself is performing. To create a clear understanding of how effectively the platform is running, the page displays the status of the database connection, data pipeline, vector store, and AI model availability.
The Report Generator page allows the user to create a report summarizing the organization’s compliance readiness, detailing the compliance findings, evidence, and AI assessments in a consolidated compliance report.
The architecture of the Compliance Evidence Pipeline is split into three major components: frontend, backend, and AI layer. The frontend sets up the user interaction for the tool. Streamlit dashboard is used to create the interface that allows the users to view and interact with the compliance evidence pipeline through its many pages. To access the platform, the user must sign into the interface with a username and password. The level of access that they have is determined by a specific role. The interactive reporting also assists the user in gaining insight on the organization’s compliance findings. The backend is responsible for storing and managing the data that is displayed in the tool. PostgreSQL is used for database storage to record data such as users, hosts, access reviews, and policy exceptions. SQLAlchemy models are utilized to transform this data so that it can work with Python. To capture the policy information, the data pipeline uses text files that go into detail on the specific policies and chunks, embeds, and stores the vectors so that specific responses can be retrieved. This moves us to the AI layer. Once the policy information has gone through this process, the RAG system can parse the vector storage for the specific response that it needs. The large language model that it is utilized (ChatGPT in this case), receives the request and finds the specific policy areas where it can locate the correct response. These three components working together allows for a smooth user experience when gathering data on compliance policy.
The RAG pipeline is what gives the Compliance AI Platform its AI abilities and makes searching for and putting together policy and compliance data easier. This is possible thanks to RAG’s ability to ingest the policy documents, embed the information, and retrieve this information when needed. Using langchain, the chunking breaks down the data into smaller pieces, creating a smoother process for the LLM when it is gathering the information that it needs for its responses. Metadata is also included to ensure that the source that the AI got the data from is not lost. These chunks are stored using a Chroma vector database, stashed in smaller pieces so that they can be easier found if needed by the LLM. When a user prompts the LLM with a request, the request is embedded and turned into a numerical vector. Chroma seeks through the vector database to correlate the vector to each chunk that is stowed. Once Chroma has found applicable chunks, a prompt is set up that contains the user’s request, the located chunks, and any instructions for how the model should respond. This prompt is sent to the LLM so that it can create a response that is sent back to the user. Through this method, RAG is able to simplify and improve the process of how an LLM responds to requests, allowing for better and faster compliance visibility within an organization.
Artificial intelligence can assist cybersecurity in several ways, and one significant approach to this is utilizing large language models and their ability to respond to specific requests and generate specific data insights. This project demonstrated the incredible potential for the use of AI within cybersecurity compliance visibility and policy comprehension. Through the use of retrieval-augmented generation, AI processes can be enhanced through its assistance to the LLM’s ability to get the data chunks that it needs to respond to specific requests. This ability opens the door for a variety of helpful applications from compliance reviews to policy querying to auditing logging. New advances in AI conveys new opportunities to improve cybersecurity processes through more than just automation, but through establishing a full platform that security professionals can rely on.
Code for project is available here: https://github.com/jack-gage-sec/cyber-ai-rag/tree/compliance-ai-platform