Analyze a docket
Configure rulemakings manually to generate Unity Catalog Delta tables, date windows, and custom ingestion parameters. For broad topic monitoring, use the Watchlist or the Discovered Rulemakings panel.
Docket registration
Stage 2 sequentially fetches one detail page per comment from regulations.gov, capped at ~1000 req/hour by api.data.gov. Bottleneck stage: parsing.
Orchestration Settings (Databricks Cloud)
Tip: Submit Analysis Job sends a pipeline trigger command directly to your hosted Databricks instance and navigates to the tracking dashboard to monitor progress.
configs/dockets.yaml snippet
- docket_id: "EPA-HQ-OAR-2021-0317"
source: "regulations_gov"
topic_id: "oil_and_gas"
agency_id: "EPA"
title: "EPA Standards of Performance for New, Reconstructed, and Modified Sources: Oil and Natural Gas Sector Climate Review"
date_window:
start_date: null
end_date: null
ingestion_mode: "full"
expected_scale: 396
processing_status: "configured_awaiting_run"
notes: "Exact normalized-text hash baseline found 7 duplicate clusters across 16 comments from 396 parsed rows."Pipeline commands
.uv-test-venv\Scripts\python.exe scripts\run_ingestion.py --docket-id EPA-HQ-OAR-2021-0317 .uv-test-venv\Scripts\python.exe scripts\run_embedding.py --docket-id EPA-HQ-OAR-2021-0317 --backend databricks .uv-test-venv\Scripts\python.exe scripts\run_clustering.py --docket-id EPA-HQ-OAR-2021-0317 --clustering-mode vector_search
For production-scale runs, use the Databricks workflow task order from the end-to-end runbook: load sample tables, embed, cluster, export dashboard data.
Coverage policy
Analyzed
Appears in primary browsing with semantic clusters and validation receipts.
Baseline only
Appears with exact-hash metrics and an explicit semantic clustering next step.
Ingestion ready
Appears as a workflow or template, never as a zero-result dashboard.
Template topics
Supported agencies
Tip: clicking a chip pre-fills the form by finding a known docket for that agency or topic (e.g./legacy/analyze?agency=SECautofills the SEC digital-asset-custody docket).