Analysis request
Fiduciary Investment Advice Rule
req_1ye3jd47h / created 6/1/2026, 6:46:59 PM
Databricks Jobs modesubmitted
Rulemaking Metadata
Docket IDDOL-2010-0050
Agency IDDOL
Topiclabor
Data Sourceregulations_gov
Expected Scale~5000 comments
Date WindowFull Historical Ingestion
Notes / Reviewer Context
"Smart batch ingestion. Labor/finance rule, useful for economy topic without duplicating CFPB payday."
Status & Control Plane
Live pipeline progressAuto-refreshing every 10s - no manual sync needed.
Elapsed / Projected total
under 1 min / ~5.5 hr
1. IngestACTIVE
0 / 5,000
2. Parse
0 / 5,000
3. Embed
0 / 5,000
4. Cluster
0
5. Export
0
Pipeline is running normally. Databricks compute is warming up (Serverless cold start typically takes 2-4 min). The first row counts will appear shortly. You can leave this page open or come back later - progress is auto-saved.
Databricks Workspace Integration
Active Databricks Run ID: 929767762278492
Command-Generation Mode
If you want to run this pipeline locally on your system instead of hosted Databricks, run the following sequence in your terminal:
.uv-test-venv\Scripts\python.exe scripts\run_ingestion.py --docket-id DOL-2010-0050 .uv-test-venv\Scripts\python.exe scripts\run_embedding.py --docket-id DOL-2010-0050 --backend databricks .uv-test-venv\Scripts\python.exe scripts\run_clustering.py --docket-id DOL-2010-0050 --clustering-mode vector_search
Command-generation mode allows running comment ingestion and clustering locally via python scripts, writing directly to your local delta lakehouse.