Analyze a docket
Configure rulemakings manually to generate Unity Catalog Delta tables, date windows, and custom ingestion parameters. For broad topic monitoring, use the Watchlist or the Discovered Rulemakings panel.
Docket registration
Stage 1 paginates the source API and writes each page to a FUSE-backed Delta path; per-page write cost grows with the table size. Bottleneck stage: ingestion.
Orchestration Settings (Databricks Cloud)
Tip: Submit Analysis Job sends a pipeline trigger command directly to your hosted Databricks instance and navigates to the tracking dashboard to monitor progress.
configs/dockets.yaml snippet
- docket_id: "17-108"
source: "ecfs"
topic_id: "telecom"
agency_id: "FCC"
title: "FCC Restoring Internet Freedom Proceeding (Net Neutrality Deregulation)"
date_window:
start_date: null
end_date: null
ingestion_mode: "full"
expected_scale: 4993
processing_status: "configured_awaiting_run"
notes: "500-comment controlled slice validated on Databricks Serverless with Vector Search clustering and export."Pipeline commands
.uv-test-venv\Scripts\python.exe scripts\run_ingestion.py --docket-id 17-108 .uv-test-venv\Scripts\python.exe scripts\run_embedding.py --docket-id 17-108 --backend databricks .uv-test-venv\Scripts\python.exe scripts\run_clustering.py --docket-id 17-108 --clustering-mode vector_search
For production-scale runs, use the Databricks workflow task order from the end-to-end runbook: load sample tables, embed, cluster, export dashboard data.
Coverage policy
Analyzed
Appears in primary browsing with semantic clusters and validation receipts.
Baseline only
Appears with exact-hash metrics and an explicit semantic clustering next step.
Ingestion ready
Appears as a workflow or template, never as a zero-result dashboard.
Template topics
Supported agencies
Tip: clicking a chip pre-fills the form by finding a known docket for that agency or topic (e.g./legacy/analyze?agency=SECautofills the SEC digital-asset-custody docket).