Secure Your Own Copilot on Azure

Embarking on the journey of Building your Own Copilot on Azure in my previous article, your new GenAI Copilot offers tremendous assistance with a variety of tasks, but it’s important to be aware of the potential hazards. A primary concern is avoiding exposure of sensitive information that may be leaked through RAG with natural language processing to produce text from user inputs. It could unintentionally disclose private details like names, addresses, emails, phone numbers, passwords, proprietary data, or personal views. Such breaches can damage your credibility, lead to legal troubles, or break privacy regulations.

We need to secure the solution end-to-end and it starts from designing the right guardrails and developing with best practices in mind. Through many discussions with architects around the region, many were not able to comprehensively secure their new GenAI application.

To build your foundations for a secure solution, this is a guide to begin safeguarding your Copilot on Azure.

Note: Threats are constantly changing; you’ll need to continue protect against new threats beyond the scope of this article.

Start with the Right Foundation

The Cloud Adoption Framework (CAF) consists of validated guidance, industry best practices, and resources that support the alignment of cloud strategies with business objectives to speed up cloud adoption. The CAF offers a thorough and uniform method for planning, governance, migration, management, and innovation on Azure.

A primary advantage of utilizing the CAF is its assistance in creating and deploying a secure cloud solution that adheres to compliance and regulatory standards. Security discipline within the CAF includes discussions on these subjects:

• Security principles and mindset: Adopt a security-focused culture that enables your team to create secure Azure solutions.
• Security baseline: Set a fundamental level of security controls and policies for your cloud environment and its resources.
• Security governance: Define and implement security roles, responsibilities, and protocols throughout your organization and cloud operations.
• Security operations: Establish procedures to monitor, address, and recover from security incidents and threats in the cloud.
• Security tools and services: Utilize Azure’s native and third-party security features to safeguard your cloud assets and data.

Implementing the Azure OpenAI Landing Zone reference architecture during the CAF Ready phase speeds up the development of scalable and secure solutions. You may not need to start with the Enterprise Scale Landing Zone you could start small with scale in mind sets you up with a better foundation to grow your solution. It comprises various Azure services such as Azure API Management to serve as a central API hub and DDoS protection for existing backend services. Configuration with an Application Gateway as a Web Application Firewall (WAF) offers additional security, guarding against typical web assaults like SQL injection or XSS, and can be tailored for needs. Azure Managed Identities gives applications their own identities to connect to resources, enabling them to authenticate with any Azure service compatible with Microsoft Entra authentication.

Defense in Depth

Securing a cloud-based system is essential and entails multiple aspects of protection, encompassing Application, Data, AI, Network, Identity Access, and Threat Protection. Below is a brief examination of each, you could further explore the topics with Microsoft Copilot.

1. Application: Security at this level includes safeguarding applications hosted on Azure, like web, mobile, services. It spans protecting the code, access controls, settings, deployment strategies, alongside overseeing their operational efficiency and stability. “Shift Security Left” with DevSecOps tooling such as GitHub Advanced Security.
2. Data: Pertains to securing stored and processed data on Azure, covering databases, documents, or blobs. Strategies involve data encryption at rest, in use and during transfer, setting access controls, masking and authorizations, and managing data backup and recovery processes. Microsoft Purview is a comprehensive set of solutions that helps organizations govern, protect, and manage data across their entire data estate.
3. AI: Covers securing AI models and related services hosted on Azure, including Azure AI services and Azure Machine Learning. This ensures that AI models are shielded from unauthorized access, abuse, or tampering, and maintains their dependability, equity, and accountability. A significant threat to AI security is jailbreaking, where the attempt is made to bypass or exploit the core model via crafted harmful inputs. To combat such threats, Prompt Shields have been introduced for mitigating direct attacks; they utilize real-time grounded detection to pinpoint and block suspicious input before impacting the primary model. Azure AI’s Groundedness Detection strictly serves to identify unsubstantiated assertions employing a language model with specialized training in Natural Language Inference (NLI) to determine if statements are validated, contradicted, or unrelated to the key document. Refer to the Azure AI tools to build secure and trustworthy GenAI Apps.
4. Network: Infrastructure needs to be secured, through defining the Azure resources and services, such as virtual networks, load balancers, or VPNs. It involves isolating and segmenting the network, applying firewalls and network security groups, and enforcing secure communication protocols. The Landing Zones from CAF serves as good reference for securing your network infrastructure.
5. Identity Access: Security of the identities and access management of the users and applications that interact with Azure, involves authenticating and authorizing the users and applications, applying role-based access control, and auditing and logging the activities. Multi-factor authentication (MFA) is a security feature that requires users to provide more than one piece of evidence to verify their identity when they sign in to Azure or other online services. It can help prevent identity leaks, which are unauthorized access or use of someone’s identity information, by making it harder for attackers to impersonate them or compromise their accounts.
6. Threat Protection: Detection and response to potential threats and attacks on Azure, such as malware, phishing, or DDoS, involves monitoring and analyzing the security events and alerts, applying preventive and remedial actions, and reporting and escalating the incidents. Microsoft Sentinel is a cloud-based SIEM and SOAR solution that can ingest, query and analyze data from various sources such as the Log Analytics Workspace. The Copilot for Security enriches and builds on the security data, signals and existing incidents and insights sourced from Microsoft Defender XDR and Microsoft Sentinel, to increase productivity of your SecOps team.

Tests & Evaluation

An effective approach to test and assess your copilot solution involves utilizing the Azure Well-Architected Framework, encompassing a comprehensive set of practices and recommendations to construct solutions on Azure that are secure, reliable, efficient, and cost-effective. The framework is structured around five fundamental pillars: security, reliability, performance efficiency, cost optimization, and operational excellence, each accompanied by design principles, best practices, and evaluative questions for refining your solution.

The Azure AI Studio serves as another robust avenue for testing and evaluating your copilot solution—an integrated platform tailored for machine learning and artificial intelligence development. This environment offers extensive features and services that support the creation, training, deployment, and management of AI solutions. One of its key components is the Responsible AI toolkit, designed to provide insights into the fairness, interpretability, and robustness of your AI models through various analytical metrics and visual tools, refer to AI-assisted safety evaluations in Azure AI Studio.

Conclusion

This piece has outlined the process of designing a secure conversational AI solution, your own Copilot on Azure. It has also detailed how to assess and evaluate your copilot solution with the Azure Well-Architected Framework and Azure AI Studio tools. Adhering to these guidelines allows you to develop engaging and smart chat applications that can enhance customer satisfaction and business results.

So why in did the Retail Copilot leak customer’s data? Authenticating users is the first step, but a lot of times this gets overlooked because people aren’t really used to authenticating themselves when they’re chatting with bots!