Search
Close this search box.
SOCReporting Guide

FAQ: What records should we keep from AI model evaluations?

Keep the purpose of the evaluation, the system and version tested, the relevant configuration, the approved test set or its controlled reference, results, and the reviewer’s decision. Explain the intended use and limitations so a strong result in one setting is not mistaken for evidence about every setting. Track issues through correction and any retesting. Protect test data and sensitive outputs appropriately. A useful record lets another qualified person understand what was evaluated and why the team accepted the result. A single score without context is difficult to use when the product, provider, or business use changes.

Related: SOC 2 audit and compliance hub.

Further reading: NIST Generative AI Profile.

Additional FAQs

Can employees put customer data into an AI tool?

Only after the organization has approved that use and confirmed the relevant data, contractual, security, and privacy requirements. Check the provider's applicable terms and settings, including retention, training use, access,

Get Our Emails

SOC Reporting Guide