Define the work before measuring the saving
Operator time saved is a comparison between two ways of completing equivalent work, not the difference between a long anecdote and a short demonstration. For email authentication, a useful unit might be investigating one unfamiliar sending stream and producing a reviewed recommendation. Opening a dashboard is not equivalent to finishing that investigation. A fast suggestion that another person must reconstruct can increase total effort while making the first interaction look impressive.
This article proposes an evaluation method, not an EmailAuth productivity result. No customer tasks were timed, no automation was demonstrated, and no performance improvement is claimed. Source review is dated 8 November 2026. Product capabilities, operator populations and study conditions would need to be recorded separately before this design could support a business claim.
Fix the acceptance boundary
Describe the starting evidence, allowed tools, expected output and stopping condition. An investigation could begin with an approved report extract and end with a documented owner attribution or an explicit escalation. Include an unresolved outcome when that is the correct decision. Requiring every case to end with an approved sender would reward unsupported confidence rather than competent handling of uncertainty.
Set quality criteria before seeing timings. Examples include using the correct customer, retaining evidence provenance, distinguishing authentication from alignment, and avoiding an unauthorized production change. RFC 9989 supplies the protocol distinction; it does not establish how quickly a particular operator can apply it. A reviewer should evaluate the result independently of whether the manual or assisted workflow produced it, where practical.
Compare work that could reasonably be compared
Choose a representative mix of routine, ambiguous and failed cases. Record the number of operators, their relevant experience, task complexity and the period sampled. Do not give the assisted group clean, complete evidence while the baseline group must recover missing documents. If the proposed tool changes evidence preparation, count that preparation as part of the intervention rather than hiding it outside the timer.
NIST's randomized block design guidance explains controlling important nuisance factors and randomizing others. Applied here, an evaluation might group comparable cases by difficulty and balance workflow order across operators. That is a proposed study choice, not a claim that this article ran a randomized experiment. Repeating the identical case can create a memory advantage; use matched cases or disclose the learning effect.
Count human attention across the whole lifecycle
Record hands-on effort for intake, investigation, review, correction, approval, verification and closure. Add setup, training, integration maintenance and failed-run recovery to the period-level ledger. Define whether interruptions and context switching count, then apply that rule consistently. If two people attend the same ten-minute review, that consumes twenty person-minutes even though only ten clock minutes pass.
Keep elapsed time separate. Waiting for a colleague, an external provider or another reporting interval affects completion time without necessarily consuming continuous operator attention. Background processing is also not automatically human labor. A Halo time-tracking guide describes action-linked and quick-time entries; those are possible recording mechanisms, not proof that the entries capture every activity or measure causal savings. Never treat billable time as a complete substitute for actual effort.
Preserve mistakes and incomplete runs
Maintain a case ledger with workflow assignment, start conditions, effort components, outcome, review findings and exclusions. If a task is abandoned, retain the time already spent and its reason. If a reviewer repairs an incorrect attribution, count the repair and classify the error. Reporting only successfully completed assisted cases while retaining all baseline failures produces a biased comparison.
Choose a follow-up window for reopened incidents and later corrections. State when that window ends and what remains unknown. A recommendation that appears correct at initial review may create downstream work. Do not change the inclusion criteria after discovering an inconvenient result. If an exclusion is genuinely necessary, publish the rule, the count and a sensitivity analysis that shows how the conclusion changes when excluded cases are retained.
Calculate effort without turning it into fictional revenue
A proposed period comparison is baseline human effort for the defined workload minus assisted human effort, including its attributable setup and maintenance. Use consistent units and disclose how shared costs are allocated. A per-case difference needs a clear denominator, especially when the workflows complete different numbers of acceptable cases. Report completion and error rates alongside effort, rather than letting a shorter average conceal more unresolved work.
Show sample size, typical effort and spread. A median, relevant percentiles and the range can reveal whether a few difficult cases dominate the result; a statistical interval needs a method appropriate to the study design. Do not multiply one favorable demonstration by every future domain. Released capacity is not automatically cash saved, additional revenue or a staffing reduction. Those claims require separate economic evidence and approved assumptions.
Keep approval controls inside the measurement
The assisted workflow must retain the same required human decisions as its comparator. Removing a DNS approval merely to improve the timing changes the safety conditions and invalidates the intended comparison. OWASP's excessive-agency guidance supports limiting functions, permissions and autonomous actions. Reviewing a recommendation does not itself grant permission to modify DNS, establish credentials or share customer evidence externally.
The exact commissioning boundary remains open: Describe a testable workflow and its approval boundary. Verify any EmailAuth capability before presenting it as available. The required original evidence is also outstanding: Measure comparable tasks with errors, human review and variability retained. A future evidence packet should contain the approved protocol, de-identified case ledger, quality assessments, calculation method and limitations. Until those exist, describe the evaluation plan and leave the outcome unmeasured.