Report exports become unreliable when a time filter, page boundary and late arrival are treated as unrelated details. A result that looks complete may omit records sharing a timestamp or count the same report twice. Write down the selection rules before building the export loop.
Choose the time being filtered
Aggregate reports have an evaluation period; the collector also has a receipt time. RFC 9990 defines the aggregate-report structure, including its date range. An export filtered by receipt time answers what arrived during the window, not necessarily what messages were evaluated during it. Name the chosen field and preserve both meanings when available.
Use UTC timestamps with an explicit boundary convention. The hypothetical receipt-time example below uses a half-open window: include start, exclude end. Adjacent daily exports then do not overlap solely because they share midnight. For a provider with different documented semantics, translate deliberately and deduplicate rather than pretending its end boundary is exclusive.
Follow the service's continuation mechanism
Do not guess that every endpoint uses page numbers or that a short page always ends the result. GitHub's pagination documentation demonstrates following Link headers where supplied. Other services use opaque cursors. Follow the actual contract, retain the active filter and stop only under its defined completion condition.
Ask whether pagination observes a stable snapshot. If new arrivals can change page membership, offset paging may skip or repeat records. Use stable ordering and snapshot-bound cursors where supported. For a proposed design, a stable receipt-time plus identity order and snapshot/filter-bound cursor can make continuation auditable. Use the provider’s full identity key as the tie-breaker; a report ID alone need not be unique across reporters. These are design recommendations, not verified vendor guarantees.
Test boundaries with known inputs
Assume a fixed snapshot for one reporter and tenant, filtered by receipt time for a UTC day. Report 0 arrived one second before the day and has 10 observations. Reports 1 and 2 arrived at noon and have 30 and 20 observations; report 3 arrived at 18:00 and has 50. Report 4 arrived exactly at the next midnight and has 50. Under an include-start/exclude-end rule, select IDs 1, 2 and 3: 30 + 20 + 50 = 100 observations. With the assumed page size of two and stable identity tie-breaker, those selected records occupy pages of two and one.
These are invented input facts and expected arithmetic, not an export or executed pagination test. Verify both identities and totals: substituting out-of-window report 4 for report 3 would preserve 100 but select the wrong set. This example filters whole report receipt times; it does not infer individual message timestamps from an aggregate report interval. A collection cut-off or historical-period query needs its own documented rules.
Preserve evidence of completeness
Store the original filter, version, continuation sequence, row count, distinct identities and final status with the exported data. If a page fails, label the export incomplete and resume or restart under the provider's supported mechanism. Never show a partial result as a complete monthly metric.
Review empty windows, changed filters, repeated cursors and cursor expiry. An expired cursor may require a fresh snapshot and reconciliation rather than continuing from a guessed page. For historical evaluation-period reports, allow a collection cut-off and explain late corrections. The reader should be able to tell both which observations were included and which expected sources were absent.