Mentor - Patrick Kelly
Apprentices:
Akerke, James, Manda, Phil and Tyler
Develop a comprehensive application that acts as a sanitizing layer and front end for user inputs and outputs to and from various AI Large Language Models (LLMs) with restful APIs. This application must ensure data privacy, security, and integrity, using a secure web API as middleware or a proxy interface for popular Generative AI and Large Language Model applications and APIs. The challenge includes incorporating multi-platform support (Web, Android, iOS) with secure and robust user authentication to the Front End, API, and Generative AI APIs, with secure vaulting and tokenization of Generative AI Authentication Credentials and API keys and Personal Access Tokens(PATs), data tokenization, optional participation in anonymized data analysis for research purposes for users, and compliance with the latest security standards.
The solution involves creating an open-source, secure web API that preprocesses and sanitizes inputs/outputs between users and AI LLMs, with the goal of covering as many GenAI and LLM Top Risks as possible that a user or consumer of GenAI is exposed to when interacting with Generative AI and LLM applications and models. This API serves as a middleware and optional front end as a Generative AI chat proxy with the goal of providing the best protection available for consumers of GenAI, ensuring data privacy and security. The application will support user registration, multi-factor authentication, team/organization management, and SSO with leading identity providers. It will need to provide secure vaulting and tokenization of Generative AI Authentication Credentials and API keys and Personal Access Tokens(PATs) for that users will need to obtain themselves with documentation available to the users if they need help obtaining the Generative AI API authentication credentials, keys, and tokens. The solution will build its own LLM from the data it collects, tokenizes, anonymizes, analyzes and learns from data the application receives as raw input, pre-processes, receives as raw output, and post-processes. The solution leverages modern, secure web and application development frameworks and languages, focusing on scalability and security. The architecture integrates with Cloud Native Computing Foundation projects for a resilient, scalable system.
In the initial development phase, our focus will be on addressing five critical security risks out of the top ten LLM security threats as defined by OWASP. These targeted threats are:
Prompt Injection: This threat involves manipulating the model's behavior by injecting malicious prompts. To mitigate this risk, our solution will implement stringent input validation and sanitization techniques, ensuring that only safe and appropriate prompts are processed by the model. This will help prevent unauthorized control over the model’s outputs.
Insecure Output Handling: Improper handling of model outputs can lead to security vulnerabilities, such as leaking sensitive information or producing harmful content. Our application will incorporate robust output sanitization mechanisms to ensure that all outputs are checked and sanitized before being delivered to the user, thus preventing the propagation of insecure or harmful data.
Model Denial of Service (DoS): This involves overwhelming the model with excessive requests to disrupt its normal operation. To protect against DoS attacks, our system will include rate limiting, request throttling, and monitoring mechanisms to detect and mitigate suspicious activity, ensuring the model remains available and functional.