DMARC Stats

Ingest DMARC aggregate reports (RUA) from email, store normalized results in a database, visualize via built-in dashboard or Grafana.

• Backend: Quarkus (Java 25), Gradle, Hibernate + Panache, Flyway
• Modes:
  • Scheduled poller (pull from IMAP mailbox on a cron)
  • Trigger on arrival (POST raw .eml to /api/ingest — ideal for mail server webhooks / Sieve / procmail style)
• DB: H2 (dev) / PostgreSQL (prod)
• Frontend: Zero-dependency single-file dashboard in /frontend
• Observability: Grafana + Postgres pre-provisioned

Project Layout

dmarc-stats/
├── *.eml                           # Your sample report emails (Google + Outlook)
├── backend/                        # Quarkus application (all Java/Gradle code)
│   ├── src/main/java/dk/kofoed/dmarc/...
│   ├── src/main/resources/db/migration/
│   └── ...
├── frontend/
│   ├── index.html                  # Self-contained dashboard (Tailwind + Chart.js via CDN)
│   └── grafana/                    # Provisioning + sample dashboard JSON
├── docker-compose.yml
└── README.md

Quick Start (Local Dev)

1. Prerequisites

   • Java 25+
   • (Optional) Docker for full stack

2. Easiest: one-command self-contained demo (embedded H2, no Postgres needed)

   cd backend
   ./gradlew demoUberJar
   java -jar build/dmarc-stats-0.1.0-SNAPSHOT-runner.jar
   

   This builds a single ~46 MB executable jar (dev profile + H2) that bundles the two sample reports. The reports are imported automatically on startup.

   Then open frontend/index.html in a browser (or use the APIs on port 8080).

3. Normal development (recommended for daily work)

   cd backend
   ./gradlew quarkusDev
   

   The app starts on http://localhost:8080. Sample reports are auto-loaded on first start if the DB is empty.

4. Open the dashboard

   • Option A (easiest): open frontend/index.html directly in a browser (it talks to localhost:8080)
   • The dashboard has a "Load sample data" button and file upload for more .eml

5. Useful API endpoints

   • POST /api/ingest (application/octet-stream) — send a raw .eml
   • GET /api/reports
   • GET /api/stats/summary?days=30
   • GET /api/stats/daily
   • GET /api/dev/load-samples — force reload the two test emails
   • GET /q/health

   Example curl from project root:

   curl -X POST --data-binary @'[Preview] Report Domain kofoed.dk Submitter enterprise.protection.outlook.com Report-ID 01ab5df40e9b49c0a8ce3574417074cc.eml' \
        -H 'Content-Type: application/octet-stream' \
        http://localhost:8080/api/ingest
   

Scheduled Job vs. Trigger on Arrival

Scheduled (pull): Set in application.properties (or env):

dmarc.mail.enabled=true
dmarc.mail.host=imap.yourcompany.com
dmarc.mail.user=dmarc-reports@...
dmarc.mail.password=${DMARC_MAIL_PASSWORD}
dmarc.mail.poll-cron=0 */10 * * * ?
# optional: dmarc.mail.processed-folder=Processed

The poller looks for subjects containing "Report domain" / "Report Domain" and ingests them. Processed messages are either moved or marked SEEN.

Trigger on arrival (push / webhook):

• Many mail systems (Google Workspace, Microsoft 365, self-hosted) can deliver a copy of the DMARC report email via HTTP POST of the raw message.
• Point the webhook at POST http://your-host/api/ingest with Content-Type: application/octet-stream (body = full RFC822 email).
• This gives near real-time ingestion without polling.

Both modes are supported simultaneously.

Production (PostgreSQL)

• Use %prod profile (or set QUARKUS_PROFILE=prod)
• Configure datasource (or env vars QUARKUS_DATASOURCE_*)
• Set dmarc.mail.* as needed
• Run Flyway migrations automatically (quarkus.flyway.migrate-at-start=true)

Example env for container:

QUARKUS_DATASOURCE_JDBC_URL=jdbc:postgresql://db:5432/dmarcstats
QUARKUS_DATASOURCE_USERNAME=dmarc
QUARKUS_DATASOURCE_PASSWORD=...
DMARC_MAIL_ENABLED=true
...

Docker Compose (recommended for demos / staging)

From project root:

docker compose up --build

• Postgres on 5432
• App on 8080 (waits for DB)
• Grafana on 3000 (admin/admin)

Grafana will automatically have:

• "DMARC Postgres" datasource
• "DMARC Overview" dashboard (import uid dmarc-overview if needed)

You can also point any Grafana at the Postgres dmarc_report + dmarc_record tables or the dmarc_stats view.

Database Schema

• dmarc_report: one row per received aggregate report (deduped by org_name + report_id)
• dmarc_record: the individual <record> rows (source IP, count, policy disposition, dkim/spf results, identifiers)
• dmarc_stats: convenience view joining the two

Common queries for Grafana / BI:

• Volume by day / by reporter
• Pass vs fail rates
• Top source IPs causing issues
• Policy enforcement (disposition) trends

Development Notes

• To add more parsers or fields: look in parser/EmailParser.java + DmarcXmlParser.java
• Idempotency is based on (org_name, report_id) — re-sending the same report is a no-op
• Quarkus dev mode: live reload on Java + resources changes
• Tests live in backend/src/test/...

For AI agents and future contributors: please read AGENTS.md first. It captures the non-obvious packaging, Flyway/H2, sequence naming, and sample-loading decisions made during the original implementation.

Adding Tests / More Samples

Copy additional .eml files into backend/src/test/resources/dmarc-samples/ and they can be loaded via the dev endpoint or tests.

License

MIT or whatever you prefer for internal tooling.