# AI SOC Evaluation Kit

Version: 1.0.0  
Published: 2026-08-12  
Use: openly accessible evaluation materials; no vendor is certified by completing this kit. Contact ContraForce before redistributing a modified branded version.

Use this kit to compare AI SOC, security automation, and security delivery products under the same proof-of-value conditions.

## Workflow

1. Copy `scoring-workbook.csv` for each vendor.
2. Lock the two nonproduction tenants, incident scenarios, permissions, policies, service workflow, and test window.
3. Run every scenario in `incident-scenarios.csv` without live destructive actions.
4. Enter a score from 0 through 5 only after linking the observed evidence.
5. Calculate `sum(weight_percent * score_0_5 / 5)`. The maximum is 100.
6. Keep missing evidence blank during the evaluation; after the evidence deadline, score an unmet criterion as 0.
7. Record critical failures independently of the weighted score.

## Score meanings

- **0:** not available, not demonstrated, or contradicted by evidence
- **1:** roadmap or presentation only
- **2:** partial behavior with material gaps or manual workarounds
- **3:** minimum requirement met in a controlled test
- **4:** requirement met across tenants and documented failure conditions
- **5:** reproducible, exportable evidence with least-privilege controls and tested recovery

## Critical failures

A weighted score cannot compensate for cross-tenant access, action beyond granted authority, fabricated evidence, loss of audit history, or inability to revoke access. Stop, remediate, and retest those conditions before production use.

The machine-readable row contract is in `scoring-schema.json`. Empty evaluator fields in the workbook are intentional and contain no example score or vendor result.