Safety Testing for Student-Facing AI Tutors

• CTO (Edtech)
• Head of Product, Learning Apps
• Trust and Safety Lead
• Academic Quality Head

Coaching institutes and edtech companies are adding AI tutors that solve doubts from a photo, explain concepts in Hinglish and chat with students late at night. Most users are teenagers under exam pressure. A tutor that can be talked into unsafe content, gives a confidently wrong derivation or reveals another student's data creates a safety incident, not just a bug.

We run a structured red-team of your tutor through the channels students use: app chat, image upload and voice mode where you offer it. Multi-turn playbooks in English, Hindi and Hinglish probe age-inappropriate content, handling of self-harm and distress, jailbreaks through role-play, academic-integrity bypasses, prompt injection hidden in uploaded question images, retrieval of other students' records and system-prompt extraction. A solution-accuracy set checks answers against faculty-verified solutions. Each test is scored PASS, RISK or FAIL with the full transcript as evidence, then reviewed by our security lead and your academic or safety reviewer. You get prioritised fixes and a re-test.

• Multi-turn adversarial playbooks in English, Hindi and Hinglish, by text and voice
• Child-safety checks for unsafe content, distress handling and escalation to humans
• Academic-integrity tests against exam-mode and assignment-mode policies
• Indirect prompt injection through uploaded images, PDFs and question text
• Cross-student data leakage and access-control tests on tutor memory and records
• Solution-accuracy benchmark against faculty-verified answers for each subject

1. 01Scope and policiesWe agree the tutor's channels, age bands, content policy, integrity rules and the data it can access.
2. 02Build playbooksAttack and accuracy scenarios are written for your subjects, languages and student personas.
3. 03Run testsPlaybooks run against a staging tutor, automated and by hand, across chat, image upload and voice.
4. 04Score resultsEach conversation is scored PASS, RISK or FAIL by rules and an evaluator model.
5. 05Human reviewOur security lead and your academic or safety reviewer confirm every RISK and FAIL.
6. 06Report and re-testYou receive evidence, mapped risks and fixes, and we re-run the failed tests after changes.

1. Test design
   • Policy and age-band profile
   • Attack playbook library
   • Faculty-verified solution sets
2. Execution
   • Chat and API test runner
   • Image and document injection payloads
   • Voice-mode test runner
3. Evaluation
   • Rule-based checks for unsafe content and personal data
   • Evaluator model for policy adherence
   • Answer comparison against verified solutions
4. Review
   • Reviewer console for RISK and FAIL results
   • Severity rating and fix recommendations
5. Reporting
   • Evidence pack with transcripts
   • Mapping to OWASP LLM Top 10 and NIST AI RMF
   • Re-test tracker

Every RISK and FAIL is confirmed by our security lead and your academic or trust-and-safety reviewer before it goes into the report. Decisions on launch and acceptable risk stay with your leadership.

• Tutor staging environment and API
• Mobile and web app builds for manual tests
• Issue tracker such as Jira
• CI pipeline for regression runs
• Content moderation service, if you use one

• Tests run only on staging with synthetic student accounts, never real student data
• Harmful test prompts kept in a restricted library and never published
• Self-harm scenarios written with care and reviewed by your safety lead
• Findings shared only with named contacts
• Re-tests reuse the same playbooks so results are comparable

• Helps you evidence safeguards for children's data under the DPDP Act 2023, including the restrictions on tracking, behavioural monitoring and targeted advertising directed at children.
• Findings map to the OWASP Top 10 for LLM Applications and NIST AI RMF; for UK users, the ICO's Age Appropriate Design Code may also be relevant.

• Education & Coaching

  Question Bank Tagging and Generation

  Turn years of DPPs, module books and past papers into a clean, syllabus-tagged question bank, remove duplicates and draft new variants that faculty check before any student sees them.

• Education & Coaching

  Online Test Proctoring Assistance

  Flag possible malpractice in remote tests from webcam frames and browser events, so a trained proctor reviews short evidence clips and decides, rather than software disqualifying students.

• SaaS & AI Product Companies

  Voice Agent Adversarial Testing

  Adversarial test calls to your voice agent in English, Hindi and Hinglish: identity bypass, data disclosure, off-script promises and speech-layer flaws, with PASS, RISK and FAIL evidence.

• SaaS & AI Product Companies

  RAG Data-Leakage and Access-Control Testing

  Test whether your RAG pipeline leaks documents across users, roles or tenants through retrieval, caching, citations or logs, using seeded canary documents and role-based test personas.