[ MSP, APIs & integrations ]

Design Microsoft Sentinel ingestion for authentication evidence

Build a bounded report pipeline into Azure Monitor before treating authentication data as usable Sentinel evidence.

By the EmailAuth research team · 3 min read

[ Key points ]

  • Build a bounded report pipeline into Azure Monitor before treating authentication data as usable Sentinel evidence.
  • Distinguish API acceptance, queryable evidence and analytics execution; sum represented message counts rather than counting aggregate rows.
On this page

Microsoft Sentinel analysis is only as reliable as the data that reaches its workspace. Start with a controlled pipeline that turns untrusted report files into normalized, tenant-scoped observations. Sending an attachment directly to a logging endpoint is not a substitute for validating its contents, preserving provenance and checking that the intended rows remain queryable.

Define the Azure boundary

Microsoft's Logs Ingestion API guide describes JSON ingestion into a Log Analytics workspace, with a data collection rule (DCR) defining the input and destination. The table must exist first. A transformation can map the input schema to the destination schema. Obtain the endpoint, DCR immutable identifier and stream name from the actual deployment rather than copying values from an example.

The same guide distinguishes a DCR ingestion endpoint from a data collection endpoint (DCE). A DCE is needed for the documented private-link case or when the DCR lacks an ingestion endpoint. It also documents access scoped to the DCR and the Monitoring Metrics Publisher role. Have the Azure owner approve the identity, scope and destination. This article does not provision identities, grant permissions or assert a native EmailAuth connector.

Preserve meaning during normalization

Use separate fields for report period, ingestion time, tenant identifier, reporting organization, report identifier, row identity, policy domain, source IP and message count. Authentication outcomes and policy disposition are different observations. Preserve the raw-file hash and an access-controlled evidence reference without copying the entire report into every event.

RFC 9990 defines aggregate-report semantics. One row may represent multiple messages. Summing rows instead of the count field produces a misleading volume. Deduplicate the report before expanding it into rows, and keep any correction or replay linked to its earlier import. Do not present missing reports as proof that no mail was sent.

Reject unsafe inputs before export

Bound attachment size, expanded bytes, execution time and queue depth. Use an XML parser configured to reject external resource resolution and unnecessary declarations. OWASP's XML guidance explains why parser configuration must be explicit. Put rejected inputs in a restricted quarantine with a reason and retention policy; a parser error is not an authentication failure.

Test invalid compression, excessive expansion, malformed XML, prohibited external-resource resolution, invalid message counts and incorrect tenant association before accepting data. Choose size, nesting, element-count and runtime limits from the supported schema and capacity tests, not arbitrary demonstration values. Keep parser rejection separate from a DMARC authentication result. No parser implementation, sample dialect or executed security test is supplied here.

Verify the complete path

Suppose one accepted report row represents seven messages. A repeated identical report should not increase that analytical total under the chosen deduplication policy; the expected total remains seven, not fourteen. This is hypothetical count arithmetic, not an observed ingestion result. In a real test, query the destination table by tenant and report identifier and compare both row count and summed message count with the retained normalized input. Test a transformation that drops a field or row so acceptance cannot be confused with complete evidence.

Record request acceptance separately from destination visibility and analytic-rule execution. Maintain a durable retry queue, classify permanent schema failures separately from transient service failures, and reconcile uncertain submissions before replay. Alert on ingestion lag and quarantine growth. These design checks do not prove Azure ingestion, Sentinel detections or email delivery without observed end-to-end evidence.

[ Takeaway ]

Distinguish API acceptance, queryable evidence and analytics execution; sum represented message counts rather than counting aggregate rows.

[ By role ]

What this means for you

Choose your role for practical next steps.

IT / Mail Admin

Test the transformation

A valid request can still yield unusable analytical data.

Next steps

  1. Compare source and destination counts.
  2. Retain raw hashes and replay identifiers.
  3. Quarantine malformed input.

CISO / SecOps

Separate trust boundaries

Incoming reports and export credentials create distinct risks.

Next steps

  1. Approve narrowly scoped access.
  2. Test cross-tenant rejection.
  3. Review retention and log access.

Founder / Non-technical

Fund evidence reliability

Budget for validation and reconciliation, not merely a connector.

Next steps

  1. Assign an Azure service owner.
  2. Require a queryable acceptance specimen.
  3. Review ingestion gaps as service risks.

[ FAQs ]

Frequently asked questions

Is the Logs Ingestion API a DMARC parser?

No. Normalize and validate the report before sending JSON that matches the DCR input.

Must every deployment use a DCE?

No. Use the documented DCR endpoint where available; private-link and older-DCR cases may require a DCE.

Does this guide supply a tested DMARC parser?

No. Validate the actual parser’s schema handling, resource limits, external-resource restrictions and rejection paths before ingestion.

Why keep report and ingestion times?

They distinguish the observation period from pipeline delay and help avoid false trend changes.

Should seven messages become seven rows?

Not necessarily. An aggregate row can represent seven messages; retain and sum its count.

Does API acceptance prove a Sentinel detection ran?

No. Check destination visibility and the intended analytics separately.

How should rejected XML affect pass rates?

It should not be counted as failed authentication. Track it as unavailable or rejected evidence.

Is an EmailAuth integration verified here?

No. This is a source-backed design with hypothetical examples; live tenant acceptance and end-to-end testing remain required.

[ Sources ]

Sources

  1. Microsoft: Logs Ingestion API in Azure Monitor

    Sections Basic operation · Configuration · Endpoint · Data collection rule · REST API call

  2. RFC 9990: DMARC Aggregate Reporting

    Sections §3.1 · §3.1.1.4 · §3.1.2 · §3.5.1 · §3.5.4 · §4 · §7 · §8

  3. OWASP: XML External Entity Prevention Cheat Sheet

    Sections Introduction · General Guidance

[ Glossary ]

Glossary

DCR
Azure data collection rule describing an input stream and its destination.
Ingestion lag
Elapsed time between receiving evidence and making it usable in the destination.

Enterprise provenance

Why top financial institutions and governments retired their email security dashboards.

Traditional email authentication platforms sell you a dashboard and ask your engineers to do the labour. In high-stakes environments, manual oversight means delayed enforcement, configuration drift and unmonitored shadow domains.

EmailAuth is the agentic email provenance platform. Our agents enforce p=reject, keep third-party senders aligned and move your domains to full RFC 9989 compliance, the new DMARC standard,, with human-in-the-loop auditability.

Governments Large enterprises Tier-1 financial institutions

SaaS · Dedicated tenancy · Air-gapped on-prem · BYOK encryption

Knowledge Base