Define the outcome before selecting a metric
An authentication project may improve control over legitimate sending identities without proving that more recipients saw or read messages. DMARC evaluates aligned authentication and policy context; it does not establish every aspect of delivery, recipient engagement or business value. A measurement plan should therefore name the outcome being assessed before it selects a dashboard metric.
Useful outcomes can include aligned authentication for a bounded route, fewer particular SMTP rejections, receiving-server acceptance or a separately measured placement result. They are not interchangeable. A higher DMARC pass rate can coexist with unchanged delivery problems caused by other factors. Conversely, delivery can improve while authentication remains unchanged. Keep the mechanism and the outcome in separate evidence fields. RFC 9989, section 2.4 and 4.4
Choose the delivery stage precisely
Sender applications and providers expose different stages. An accepted API request can precede queueing and attempted delivery. A temporary deferral is not a final rejection, and receiving-server acceptance is not proof of inbox placement. A human opening a message is another distinct event with its own measurement limitations. Avoid using the single word delivered to cover all these stages.
The retained SendGrid event documentation provides a concrete example: processed indicates that the message was received and is ready to be delivered; delivered concerns the receiving server; deferred and dropped describe other distinct outcomes. Those definitions support interpreting that provider’s events, not a universal schema for every platform. Preserve the exact provider definition and version used in the analysis. SendGrid Event Webhook Reference
If the available data stops at receiving-server acceptance, state that limit. Do not rename it inbox placement because that makes the outcome sound more commercially compelling. The measurement should remain useful even when it cannot answer every question a stakeholder would like to ask.
Keep the before and after populations comparable
Define the included streams, recipient providers, domains, message categories, time windows and counting units. Preserve retry and multi-recipient handling rules. A comparison between a holiday campaign and a quiet transactional period may reflect population changes rather than a technical intervention. Likewise, removing a problematic stream from the denominator can make a rate look better without improving that stream.
Use consistent event definitions and collection methods across the comparison. Record any instrumentation change, missing interval, late report or excluded artifact. If a provider starts reporting more detail after the intervention, the apparent trend may partly reflect visibility rather than behavior. Keep dataset versions and extraction methods so another analyst can reproduce the result.
Do not manufacture a baseline when none exists. A current snapshot can describe current observations but cannot prove improvement over an undocumented past state. If the historical data is incomplete, present a bounded comparison or establish a prospective baseline rather than estimating convenient numbers.
Record competing explanations
List material changes that could affect the outcome: sending volume, recipient mix, content, list hygiene, complaint patterns, provider routing, reputation, seasonality and application changes. The purpose is not to claim every variable can be controlled, but to prevent an authentication fix from receiving automatic credit for all subsequent movement in delivery metrics.
Distinguish temporal association from a supported causal conclusion. A before-and-after chart can show that one measured outcome changed after a configuration correction. It cannot, by timing alone, establish that the correction caused the entire change. Where a controlled design is feasible and authorized, define it carefully; otherwise disclose the observational nature of the comparison.
Keep unsupported financial extrapolations out of the result. More accepted messages do not automatically equal additional sales, prevented fraud or a particular return on investment. Those claims require their own data and analytical assumptions. No uplift, savings or prevented-loss estimate is invented in this manuscript.
Measurement worksheet and acceptance criteria
The worksheet should include the intervention description, exact change time, affected routes, before and after windows, event stage, numerator, denominator, source systems, query version, exclusions and external influences. Add separate authentication and delivery columns so a reviewer can see whether the observed identity correction and delivery outcome concern the same scope.
For authentication, retain aligned results rather than substituting raw SPF pass. For delivery, retain the actual SMTP or provider event evidence and distinguish temporary states from final outcomes. If the source cannot support a transaction-level join, describe the grouped relationship and uncertainty instead of implying exact correspondence.
Acceptance requires reproducible counts from the declared dataset version, stable definitions and a conclusion no broader than the evidence. The report should disclose whether the comparison is descriptive or supports a stronger inference. A real operational review also needs approval of any data access or test design; the worksheet itself does not authorize exports or message sends.
Fictional diagnosis: alignment improves while acceptance shifts
Imagine a fictional receipt route whose custom signing configuration is corrected. A proposed comparison would show aligned authentication separately from receiving-server acceptance. During the same period, the organization also changes recipient targeting and sending volume. No actual messages, counts or rates were collected for this example.
If both metrics later improve, the analyst can describe the observed changes within the measured population, but should not attribute the entire acceptance change to authentication without addressing the other changes. If alignment improves and acceptance does not, the authentication work can still have achieved its scoped technical objective. It should not be called a failure merely because a different delivery problem remains.
If the after period excludes a troublesome recipient provider, the population must be restated or the comparison corrected. Quietly dropping that provider would create an artificial improvement. The useful conclusion names the included routes, event stage and uncertainty, not a universal claim that DMARC guarantees delivery.
Original evidence and release status
The catalogue requires a before-and-after measurement with stable population, delivery stage and external influences recorded. This draft supplies the methodology and a fictional example only. No baseline, production event export, controlled trial or measured improvement is claimed.
All six release gates remain open. Before operational publication, obtain authorized source data, validate the counting rules and have a qualified reviewer inspect the population and causal language. A stakeholder summary should preserve the distinction between authentication coverage and delivery outcomes even when brevity is important. Honest limits make the result more actionable by showing which problem was actually improved and which still requires investigation.