[ Agentic automation ]

Test an agent's sender attribution against ambiguous infrastructure

Test sender attribution on shared and indirect infrastructure, rewarding correct abstention instead of confident guesses.

By the EmailAuth research team · 2 min read

[ Key points ]

  • Test sender attribution on shared and indirect infrastructure, rewarding correct abstention instead of confident guesses.
  • Report both answered-case accuracy and population coverage; abstention preserves uncertainty but still creates investigation work.
On this page

Sender attribution connects an observed mail path to an accountable service. It is not the same as recognizing an IP owner or finding a familiar provider name. Evaluate an agent against deliberately ambiguous examples before relying on its sender inventory or allowing it to recommend new authorization.

Define the label you need

Separate infrastructure provider, customer service instance, business owner and approval status. A shared provider address may support the first label while leaving the remaining three unknown. A DKIM domain identifies signing responsibility within the protocol; it does not by itself identify the purchasing department or current approval. RFC 6376 describes the signing identity and verification semantics.

Build a gold set using controlled samples, account-specific provider configuration and service-owner decisions. Retain disagreements as unresolved labels. Do not use the agent's earlier guesses as ground truth, and do not reward correct provider recognition as correct business-service attribution.

Include cases that force abstention

Consider five hypothetical case descriptions: shared infrastructure with two plausible services; a forwarding path without origin-account corroboration; a provider-only clue; a service supported by account configuration, a matching controlled sample and an accepted owner; and conflicting owner records. Under a proposed conservative rule requiring a unique service instance and corroboration, only the fourth supports attribution. The other four need specified missing evidence or conflict resolution.

This is an illustrative rule, not a sender-identification engine or executed classifier. Real architectures may need additional routing, gateway and account evidence. DKIM authentication can survive some forwarding while the connecting IP changes; it does not necessarily survive message modification. Address correlation alone cannot explain every indirect path.

Measure errors that matter

Count false attribution separately from abstention. A false attribution can authorize the wrong sender or hide a compromised integration. An abstention creates investigation work but preserves the uncertainty. Report correct attribution among answered cases alongside coverage across the complete set; a system that answers only one easy case should not appear universally accurate.

If an imagined system answered only the corroborated case correctly and abstained on the other four, answered-case correctness would be 1/1 while answer coverage would be 1/5, or 20%. Those assumed counts do not establish production precision or absence of future errors. A single easy answered case cannot support a broad accuracy claim. For an actual evaluation, retain concrete inputs, expected labels and observed outputs so a reviewer can challenge both the answer and the label rationale.

Connect evaluation to operations

Route unresolved cases to a service owner or mail administrator with the missing evidence listed. Do not add SPF permissions, delegate a DKIM selector or relax DMARC merely because a provider seems plausible. RFC 9989's alignment result answers an authentication question, not a business authorization question.

Re-test after changes to the agent, source enrichment, parser or provider mapping. Include negative examples from real investigations when authorized and minimize personal data. Keep a versioned label rationale so an improved score cannot result merely from redefining ambiguous cases as known. No EmailAuth attribution capability is implied by this test design.

Primary reading: RFC 6376 DKIM, RFC 9990 DMARC Aggregate Reporting, RFC 9989 DMARC.

[ Takeaway ]

Report both answered-case accuracy and population coverage; abstention preserves uncertainty but still creates investigation work.

[ By role ]

What this means for you

Choose your role for practical next steps.

IT / Mail Admin

Corroborate the instance

Provider recognition is only an initial clue.

Next steps

  1. Compare controlled samples.
  2. Check account configuration.
  3. Retain conflicting records.

CISO / SecOps

Prioritize false attribution

Wrong authorization can be more costly than abstention.

Next steps

  1. Review negative cases.
  2. Audit labels.
  3. Version evaluations.

Founder / Non-technical

Budget for unresolved cases

Safe automation still needs accountable humans.

Next steps

  1. Track investigation workload.
  2. Assign service owners.
  3. Reject unsupported certainty.

[ FAQs ]

Frequently asked questions

Does an IP owner identify the sending service?

Not reliably, especially on shared infrastructure.

What counts as ground truth?

Corroborated service-instance evidence and an accountable owner decision.

Should abstention count as a failure?

Track it separately from incorrect attribution and explain its investigation cost.

Can forwarding change the observed IP?

Yes. The connecting host may be an intermediary.

Does the five-case example measure an agent?

No. It illustrates a proposed conservative rule and hypothetical 1/5 answer coverage. Actual accuracy requires observed outputs on a defined evaluation set.

Does a familiar DKIM domain establish procurement approval?

No. Signing identity and business authorization are different.

Why report coverage with precision?

A classifier can avoid errors by answering very few cases.

When should the benchmark be rerun?

After relevant model, parser, enrichment or mapping changes.

[ Sources ]

Sources

  1. RFC 6376 DKIM

    Sections §2.5 · §3.1 · §3.5 · §6.1.2

  2. RFC 9990 DMARC Aggregate Reporting

    Sections §3.1 · §3.1.1 · §7 · §8

  3. RFC 9989 DMARC

    Sections §4.4 · §5.3.5 · §5.4 · §7.4 · §11

[ Glossary ]

Glossary

Abstention
An explicit decision not to assign a service label because required corroboration is missing or conflicting.
Answer coverage
The proportion of evaluation cases for which the system supplies an attribution rather than abstaining.

Enterprise provenance

Why top financial institutions and governments retired their email security dashboards.

Traditional email authentication platforms sell you a dashboard and ask your engineers to do the labour. In high-stakes environments, manual oversight means delayed enforcement, configuration drift and unmonitored shadow domains.

EmailAuth is the agentic email provenance platform. Our agents enforce p=reject, keep third-party senders aligned and move your domains to full RFC 9989 compliance, the new DMARC standard,, with human-in-the-loop auditability.

Governments Large enterprises Tier-1 financial institutions

SaaS · Dedicated tenancy · Air-gapped on-prem · BYOK encryption

Knowledge Base