[project-defiant]

Summary

The 2026-08-28 GWAS Catalog summary statistics PICS run was moved from the root of gs://gwas_catalog_sumstats_pics into a dated run folder, so the release is preserved before the DAG runs on gentropy v4.0.0 with the new credible set schema. As for the top hits (#220), this allows comparing the results and rolling back.

gs://gwas_catalog_sumstats_pics/prod/20260828T115527/
├── config/gwas_catalog_sumstats_pics.yaml   # DAG config of the run
├── docs/                                    # README.md + DAG svg
└── output/{study_index, study_locus_window_based_clumped, study_locus_ld_clumped, credible_sets}

The timestamp is the time of the first object written by the run (the study index, 2026-08-28T11:55:27Z).

This PR points the repo at the new location:

• gwas_catalog_sumstats_pics.yaml: output_path is now gs://gwas_catalog_sumstats_pics/prod/20260828T115527/output. It is identical to the config uploaded to the run folder; the rest of the config (sync_date: '20260802', GENTROPY_REF: v3.4.0-rc.3) is the one of the run.
• pis/config.yaml: the GCSS PICS study index and credible sets are copied from the run's output/. The data is the same v3 release: pis moves to the v4 outputs once the credible set validation reads CredibleSet and CredibleSetVariant.
• Docs: the sumstats PICS section moves to docs/datasources/gwas_catalog_data/gwas_catalog_sumstats_pics/README.md. The new README:
  • lists the inputs of the run: the gentroutils sync of 2026-08-02, the harmonised summary statistics and the gnomAD LD index
  • describes the four steps, with gentropy links pinned to v3.4.0-rc.3
  • has schema tables of the four outputs, read from the parquet files, with a description per field and the columns with no values marked "Empty"
  • The DAG svg is regenerated and moves next to the README. The GWAS Catalog README links to the new page and gets a changelog entry.
• Makefile: upload-gwas-catalog-sumstats-pics-readme uploads only the sumstats PICS docs to $(BUCKET)/docs, and fails when BUCKET is not set (make upload-gwas-catalog-sumstats-pics-readme BUCKET=gs://gwas_catalog_sumstats_pics/prod/20260828T115527). It is not part of update-bucket-docs. The GWAS Catalog upload no longer copies the full docs to the sumstats PICS bucket.

The bucket is already in this state. The data was copied, and the config and docs were uploaded; the files in this PR are identical to the uploaded copies.

After merge

• Delete the old folders at the root of the bucket, still read by pis on main until this is merged: study_index/, study_locus_window_based_clumped/, study_locus_ld_clumped/, credible_sets/ and docs/.

Notes

• With output_path pinned to the run folder, running the old DAG again would overwrite this release. The next run moves the DAG to StagingDagConfig on gentropy v4 with its own run folder.

Test plan

• uv run pytest tests/test_dag_validation.py tests/test_staging_models.py tests/test_staging_dag.py (orchestration): 73 passed
• uv run pytest tests (pis): 10 passed
• make -n upload-gwas-catalog-sumstats-pics-readme fails without BUCKET and resolves to the run folder with it
• README, svg and config in the PR are byte-identical to gs://gwas_catalog_sumstats_pics/prod/20260828T115527/
• the copied outputs match the root folders: same bytes and the same crc32c of every object
• study_index/*.parquet (1 file) and credible_sets/*.parquet (23 files) globs in pis/config.yaml match files in the run folder

🤖 Generated with Claude Code

[Copilot]

🔵 Needs a closer look

The change's correctness hinges on the real GCS bucket state—verified data copies and lingering old roots—which automated review cannot inspect, so a human quorum vote is required.

No open issues 🎉

What changed in this PR

This PR relocates the 2026-08-28 GWAS Catalog summary statistics PICS release out of the root of gs://gwas_catalog_sumstats_pics into a dated prod run folder (gs://gwas_catalog_sumstats_pics/prod/20260828T115527/), preserving the v3 release before the DAG is re-run on gentropy v4.0.0 with the new credible-set schema. It repoints the repository (DAG config, pis inputs, docs, and Makefile) at the new location, mirroring the top-hits migration done in #220. The bucket is reportedly already in this state, so these changes bring the repo into alignment with the archived release.

Changes:

• Point the gwas_catalog_sumstats_pics DAG output_path and pis input sources at the new prod/20260828T115527/output/... location.
• Split the sumstats PICS documentation into a dedicated gwas_catalog_sumstats_pics/README.md (with input list, step descriptions pinned to gentropy v3.4.0-rc.3, and output schema tables) plus a regenerated DAG SVG, and add a changelog entry.
• Add a BUCKET-parameterised upload-gwas-catalog-sumstats-pics-readme Make target, update build-dag-svgs to the new SVG path, and stop rsyncing full docs to the PICS bucket.

File	Description
pis/​config.yaml	Repoints GCSS PICS study index and credible-set sources to the new prod/.../output/ paths.
orchestration/​src/​orchestration/​dags/​config/​gwas_catalog_sumstats_pics.yaml	Sets output_path to the dated run folder; propagates to all derived step paths.
orchestration/​Makefile	Updates build-dag-svgs SVG path, adds upload-gwas-catalog-sumstats-pics-readme (requires BUCKET), removes the full-docs rsync to the PICS bucket.
orchestration/​docs/​datasources/​gwas_catalog_data/​README.md	Replaces the inline PICS section with a link to the new sub-README and adds a changelog entry.
orchestration/​docs/​datasources/​gwas_catalog_data/​gwas_catalog_sumstats_pics/​README.md	New detailed README documenting inputs, DAG steps, and output schemas.
orchestration/​docs/​datasources/​gwas_catalog_data/​gwas_catalog_sumstats_pics/​gwas_catalog_sumstats_pics.svg	New DAG SVG relocated next to the README, with cluster node ids fixed.
orchestration/​docs/​datasources/​gwas_catalog_data/​gwas_catalog_sumstats_pics.svg	Old SVG removed from its previous location.

Note: the correctness of this change ultimately depends on the actual GCS bucket contents matching the new paths, which cannot be verified from the repository alone.

_{Review effort: Balanced}

[project-defiant]

Closing: the freeze is in main through #226, and #221 is merged. #243 is now based on main.