An API evaluation should answer whether a customer can reliably retrieve the intended evidence, not whether a sample request returns HTTP 200. Define the workflow first: for example, export one customer's received aggregate reports into a private evidence store without losing pages or mixing tenants. Avoid assuming that any named product offers this interface until its current contract is verified.
Pin the contract and the boundaries
Record the provider, endpoint, API version, authentication method, tenant boundary, fields, pagination rules and documented limits. Separate the identity of a report from the time it arrived and the period it covers. Specify whether the API returns raw reports, normalized rows or derived findings. Those are different evidence populations and need different reconciliation rules.
GitHub's API version documentation illustrates explicit version selection with X-GitHub-Api-Version. That is a concrete example of versioning practice, not evidence that GitHub exports DMARC reports. For an actual evaluation, record those details in a reviewable contract note before testing. This guide supplies no vendor contract, endpoint or product capability.
Test provenance as well as fields
For a normalized row, retain reporter, report ID, original period, receipt time and source fingerprint when available. Preserve the original artifact under approved retention controls. RFC 9990 is the primary reference for aggregate-report structure. A convenient JSON schema cannot turn reported counts into proof of delivered messages or recover information the receiver never supplied.
Ask how the service represents partial ingestion, malformed reports, corrected data and missing collection periods. A silently skipped page can make a plausible metric wrong. Request a sample that includes known errors and compare it with independently counted input, not merely another screen powered by the same export.
Exercise failure paths deliberately
Test a multi-page result, an empty result, duplicate input, two records sharing a timestamp, a revoked credential, a cross-tenant request and a transient error. Follow documented continuation links or cursors. For rate limiting, record status, relevant headers and retry policy. GitHub's best practices recommend respecting rate-limit signals and avoiding uncontrolled concurrency; exact limits still belong to the actual service contract.
Suppose page one is stored but page two fails. An HTTP 200 for the first request cannot justify a complete export; retain the partial status and use the provider’s supported continuation or restart process. Separately, a successfully authenticated request for another authorized test tenant must still be denied when the caller lacks that grant. These hypothetical cases test different requirements: completeness and authorization.
For a retry test, distinguish a documented rate-limit response from a permanent permission failure. GitHub’s rate-limit documentation explains that it can report rate limiting with 403 or 429, so do not classify every 403 as permanently non-retryable without reading its error context and documented headers. Apply the actual provider’s bounded wait and retry rules. No virtual clock, restart test, vendor call or live access-control check was executed here.
Decide what is acceptable
Create an evaluation record listing each required behavior, the evidence obtained and remaining uncertainty. Treat missing tenant isolation or unrecoverable data loss as stop conditions. A lack of production credentials need not prevent documentary evaluation, but it does prevent claiming that live access controls were tested.
Agree the next step with the service owner: obtain a sandbox, implement a guarded adapter, or narrow the workflow to supported exports. Retain the pinned contract and test fixtures so a provider change can be assessed later. Keep product-specific claims limited to documented and observed capability; no EmailAuth API integration is verified by this guide.