A useful AI summary should shorten the path to the original evidence. It should not replace that evidence with a confident story. Review the numbers, identity claims, uncertainty and proposed action separately. A summary can quote the correct failure percentage while drawing an unsafe conclusion about whether a sender is legitimate.
Establish the denominator and observation window
Retain the report set, source identifiers, time interval, timezone and deduplication method used for the summary. Distinguish observed messages from all messages sent by the organisation. Reports may arrive late or cover different periods; absence of a report is not evidence of absence of traffic. RFC 9990 describes aggregate reporting and its evaluation and security considerations.
Ask the summary to state exactly which count is being divided by which total. Separate raw authentication results, aligned results and receiver disposition. Those fields answer different questions. Do not rename every non-rejected message a DMARC pass or every failure an attack. A legitimate flow can fail alignment, and authenticated mail can still be abusive.
Check each material claim
Build a claim table with the summary sentence, evidence pointer, calculation or inference and reviewer decision. Numerical claims should be recomputable. Identity claims need evidence of the organisation's particular sender, not merely a recognizable provider name or IP range. Recommendations need a stated business impact and an owner who can validate the mail flow.
RFC 9989 distinguishes the relevant authenticated identifiers and their alignment with the From domain. A generic statement that SPF passed is not enough to conclude DMARC passed. A summary should preserve that distinction, including cases where an aligned DKIM signature provides the pass despite an SPF failure.
Check three claims against stated inputs
Suppose a hypothetical deduplicated report population for one specified interval contains 900 DMARC-pass observations and 100 DMARC-fail observations. The observed failure rate is 100/(900+100), or 10%. A candidate summary saying zero failures contradicts those inputs. A claim that every sender is business-approved remains unsupported if the account and owner evidence is absent; it is not disproved solely by the failure count.
A recommendation to publish reject immediately also needs its own readiness, impact and authorization evidence. Correcting the percentage cannot supply those missing decisions. These are hand-worked claim comparisons, not a supplied claim table, deterministic validator or observed model output. For an actual review, retain the exact sentence, source pointer, calculation or inference, and reviewer disposition for each material claim. Inspect omissions and tool effects as well as explicit errors.
Review omissions and action bias
Look for low-volume high-impact senders, forwarding paths, unknown ownership, missing reporting periods and failed test messages. A summary that mentions only the largest source may conceal a payroll or password-reset failure. Ask what evidence would change the recommendation and whether that evidence has actually been collected.
Use a second reviewer for consequential recommendations and keep the exact input and output versions. NIST's AI RMF provides a voluntary framework for governing, measuring and managing AI risk; this claim-table method is an application, not a mandated NIST test. Reject unsupported certainty rather than editing its tone to sound more cautious while retaining the same unsafe action.