Deterministic evaluation

Empirical Benchmarks & Quality Scorecards

We do not evaluate our models using subjective LLM-as-a-judge scores. Every release gate is asserted against frozen unit test suites, ground-truth financial cases, and symbolic evaluators.

Release Gate CI

Gayatri AI Platform — 1,252 Unit & Integration Tests

Enforced on every GitHub Actions commit across 43 release phases. Ensures complete data isolation, scoped RAG boundary enforcement, and offline stability.

100%

1,252 / 1,252 PASSED

Verification Phase	Test Target & Subsystem	Enforcement Rule	Status
Phase 01	Auth & Tenant Isolation	JTI token revocation & cross-tenant request rejection	Passed
Phase 06	Scoped RAG Authorization	Zero cross-course embedding leakage; strict boundary check	Passed
Phase 16	Socratic Turn Orchestrator	Zero final answer string disclosure in student response buffer	Passed
Phase 22	Network Guard & Privacy	Zero socket calls outside 127.0.0.1; zero cloud telemetry	Passed
Phase 25	Memory Capacity & Footprint	RSS memory stays strictly under 1GB RAM on CPU workloads	Passed
Phase 43	Production Readiness Gate	All 1,252 regression tests pass concurrently in ~90s	Passed

Financial Adjudication

Gayatri AI BFSI Workbench — 12/12 Target Metrics Passed

Evaluated against 11 multi-document ground-truth case packs (6 Personal Loans, 5 Health Claims, 40 Grievances) with 100% rule agreement.

100%

12 / 12 TARGET METRICS

Evaluation Metric	Target Threshold	Measured Result	Evaluation Method
Loan Underwriting Precision	100% agreement	100% (6/6 cases)	Arithmetic check & reducing EMI vs gold standards
Claim Adjudication Agreement	100% agreement	100% (5/5 cases)	Policy exclusions, sub-limits & tariff reconciliation
Policy Citation Accuracy	≥ 90% accuracy	94.2% citation precision	Clause-level BM25+TF-IDF hybrid RAG verification
Unanswerable Abstention	100% abstention	100% strict abstention	30-question unanswerable query benchmark
Grievance Escalation Detection	100% detection	100% escalation catch	RBI dispute taxonomy with maker-checker gates
Audit Ledger Integrity	Zero broken links	100% valid SHA-256 chain	Cryptographic hash-chain traversal verification

Fine-Tuned SLM

Gayatri Chemistry Tutor — 305 Domain Benchmarks

Evaluated on Class 11–12 NCERT Chemistry (Thermodynamics & Inorganic Chemistry) with symbolic equation normalization.

100%

305 / 305 DOMAIN TESTS

NUMERIC TOLERANCE

Zero String Fallback

Evaluates chemical thermodynamic calculations within strict physical error bounds.

EQUATION BALANCE

Stoichiometric Normalization

Validates reaction coefficients, phase states, and enthalpy signs deterministically.

MISCONCEPTION DETECTION

Concept Trap Diagnosis

Identifies standard enthalpy sign confusion and guides students back to first principles.

Transparent Verification

Reproduce Any Benchmark Locally

All benchmark datasets, pytest execution scripts, and gold ground-truth files are open and available in our GitHub repositories.

Visit Gayatri-Education on GitHub →