RAG Data-Leakage and Access-Control Testing

• CTO of a multi-tenant SaaS
• Head of Platform Engineering
• CISO
• AI Search or Copilot Product Lead

RAG lets your assistant answer from customers' own documents, tickets and records. It also creates a new path to that data: a query, an embedding search and a prompt. If permissions are not enforced at retrieval time, or if caches and logs mix tenants, one customer's question can surface another customer's content.

We test your RAG feature for data leakage and access-control gaps from the outside in and the inside out. We seed each test tenant, role and user with unique canary documents, then use test personas to query through every entry point: chat, search, API and integrations. Any canary that appears in an answer, citation, cache hit, log or trace where it should not is a finding. We also review the retrieval code path, index design and permission sync, test what happens when access is revoked or a document is deleted, and check whether uploaded documents can carry instructions that change the assistant's behaviour. Findings map to OWASP LLM risks for sensitive information disclosure and vector and embedding weaknesses.

• Canary documents seeded per tenant, role and user to detect leakage
• Persona-based queries across chat, search, API and integration entry points
• Permission revocation and deletion tests for stale chunks
• Cache, citation, analytics and trace leakage checks
• Indirect prompt injection tests through uploaded or synced documents
• Review of retrieval code paths, index design and permission sync

1. 01Map the data pathWe map ingestion, chunking, indexes, permission sync, retrieval, caching and logging for the RAG feature.
2. 02Seed canariesUnique canary documents are added for each test tenant, role and user.
3. 03Query as each personaTest personas query through every entry point and try to reach content they should not see.
4. 04Change permissionsWe revoke access, delete documents and move users between groups, then query again.
5. 05Detect and reviewCanary hits in answers, citations, caches and traces are flagged and confirmed by a tester.
6. 06Report and retestFindings come with fixes to retrieval filters, index design or sync, and are retested after changes.

1. Data path review
   • Ingestion and chunking review
   • Index and namespace design review
   • Permission sync review
2. Test setup
   • Canary document generator
   • Test tenants, roles and personas
3. Test execution
   • Entry point drivers for chat, search and API
   • Permission change scripts
4. Detection
   • Canary detector across answers and citations
   • Cache, log and trace scanner
5. Reporting
   • Tester review
   • Findings with leakage paths and fixes

A tester confirms every canary hit and traces the leakage path before reporting it. Your engineers decide on the fix, such as per-tenant indexes or retrieval-time permission checks.

• Vector stores such as pgvector, Pinecone, Weaviate, Qdrant, Elasticsearch or OpenSearch
• Connected sources such as SharePoint, Google Drive, Confluence or Jira
• Identity provider and permission service
• Tracing tools such as Langfuse or LangSmith
• CI pipeline for canary regression

• Synthetic canary content only, never real customer documents
• Testing limited to test tenants in staging
• Canaries removed after testing
• Findings shared only with named contacts
• Read-only access to logs and traces

• Maps to OWASP Top 10 for LLM Applications categories for sensitive information disclosure and vector and embedding weaknesses.
• Evidence of tenant isolation supports obligations on access to personal data under GDPR and the DPDP Act 2023.

• SaaS & AI Product Companies

  LLM Application Penetration Test (OWASP LLM Top 10)

  A scoped pentest of your LLM feature and the app around it: prompt injection, data disclosure, output handling, excessive agency and system-prompt leakage, with reproducible evidence.

• SaaS & AI Product Companies

  Continuous AI Security Regression in CI

  Run versioned prompt-injection, jailbreak and data-leakage suites in CI on every prompt, model, tool or index change, and block releases that regress past agreed severity thresholds.

• Capital Markets & Wealth Management

  Data-Leakage Testing for Internal Research Copilots

  Test internal research and RM copilots for leakage across information barriers, research entitlements and client data, including prompt injection through ingested documents.

• Legal

  Confidentiality and Privilege Leakage Testing for Legal AI

  Test your legal copilot, contract tool or client chatbot for cross-matter leakage, privilege exposure, prompt injection hidden in documents and weak access controls, with evidence and fixes.