[polrus]

What

intermediate/chembl_molecule gets ten columns for every molecule, read from the ChEMBL dump the step already restores. They are for the tractability work; no published output reads them yet.

Column	Type	From
mwFreebase, alogp, qedWeighted	Float64	compound_properties
aromaticRings, heavyAtoms, numRo5Violations	Int32	compound_properties
structuralAlerts	list of {alertId, alertName, setName}	compound_structural_alerts, structural_alerts, structural_alert_sets (all five sets)
structureType	String	molecule_dictionary
naturalProduct	Boolean	molecule_dictionary (natural_product = 1)
moleculeClass	String	the ChEMBL molecule_type; a Protein of 1,500 Da or less (PEPTIDE_MAX_MW) is Peptide, and a missing type is Unknown, as in drugType

How

• TABLES reads two more molecule_dictionary columns and four more tables from the same dump. There are no new sources, so config.yaml and the DAG don't change.
• A new add_compound_properties runs after process_molecules, which is unchanged. It left-joins on the ChEMBL id, so every molecule is kept. Molecules with no properties row get nulls, and molecules with no alerts get an empty list.
• The joins use validate='1:1', so a duplicated molregno or id fails the step instead of duplicating molecules.
• New schema molecule_properties_schema in pts/schemas/drug_molecule.py. The existing schemas don't change.

Effect on existing outputs

None:

• drug_molecule selects drug_molecule_schema.
• chemical_probes reads id.
• pharmacogenetics reads id and name.
• clinical_report builds its drug label lookup from id, name, synonyms, tradeNames and crossReferences.

Checked on real data (do/platform-2610-1, ChEMBL 37)

I ran the new step on that run's intermediate/chembl_molecule, with the five tables restored from its input/chembl/chembl.tar.gz.

• Rows: 2,921,148, the same ids, and the 11 existing columns identical.
• drug_molecule rerun on the result: identical to the run's output/drug_molecule (19,179 rows, same schema, all columns).
• Parquet size goes from 986 MB to 1,030 MB. Restoring the four extra tables takes about 25 s locally, and the new step takes about 8 s.
• Nulls: mwFreebase 0.67%; the other properties 3.31%. ChEMBL doesn't compute them for most peptides.
• Alerts: 1,674,254 molecules have at least one (MLSMR 1.41M, Dundee 1.12M, BMS 297k, Glaxo 219k, PAINS 70k).
• moleculeClass: 15,533 Peptides (MW 180–1,499.83) and 7,266 Proteins, 600 of them with no MW. Every other class matches drugType.

Notes

• ChEMBL's natural_product flag is set on some synthetic drugs (for example imatinib, CHEMBL941), so naturalProduct should be read with care.

Tests

• 12 new tests in tests/test_chembl_molecule.py, including the 1,500 Da edge, a Protein with no MW, a null type and duplicated keys.
• uv run pytest -q: 2,816 passed, 3 skipped. ruff and ty are clean.

🤖 Generated with Claude Code

[d0choa]

Thanks for this, the PR description and the real-data check are really helpful. I'm requesting changes for the inline comments, but before going further I'd like to understand the scope better.

Are we planning to use all the fields that come in here? Do you envision showing them on the drug profile pages? And are all of them relevant to the tractability assessment, or are some just nice to have? Each column adds to the intermediate and is something we have to maintain and keep correct across ChEMBL releases. I'd rather add only the ones with a clear consumer, and bring in the others when something needs them.

[d0choa]

== 1 turns ChEMBL's default into a definite False. As far as I know ChEMBL only curates natural_product for drugs, so for the ~2.9M other molecules a 0 means "not curated" rather than "not a natural product". A consumer would read False as a confident negative when the true answer is unknown. Any other value ChEMBL might use would also silently become False.

Together with the imatinib case you mention in the notes, I'm not sure how reliable this flag is. Are we planning to use naturalProduct for anything? If not, I'd leave it out. If we are, it should probably be null outside the curated set.

[d0choa]

Can you explain the rationale for the Peptide class? Why do we need it, and what will consume it? It isn't a ChEMBL class, and the 1,500 Da cut-off isn't documented anywhere. Where does that number come from?

There are also a couple of consistency issues with the rule as written:

• It uses each molecule's own mw_freebase. A salt or child form with no compound_properties row stays Protein while its parent becomes Peptide, so one drug family can carry mixed classes.
• The ~600 Proteins with no MW never get reclassified, even if they are short sequences.

If we keep it, deriving the class from the parent molecule would avoid the mixed families.

[d0choa]

Suggested cleanup: process_molecules drops molregno, so this joins molecule_dictionary (2.9M rows) in a second time just to map chembl_id back to molregno, then joins back onto the molecules on a string id. If _molecule_preprocess kept molregno and dropped it only at the final select, properties and alerts could join straight onto the molecules. That removes the second molecule_dictionary pass, the id-keyed join and its separate validate='1:1' failure mode.

[d0choa]

This repeats the molecule_type.fill_null('Unknown') rule that already produces drugType in _molecule_preprocess. If the two copies drift, moleculeClass will no longer match drugType outside the Peptide override, which is the equivalence the PR relies on. Could moleculeClass be derived from drugType instead?

[d0choa]

This docstring still says molecule_schema is the chembl_molecule intermediate, but the step now writes molecule_schema plus molecule_properties_schema, so no single schema describes the output. Could we either define the intermediate's full schema (e.g. {**molecule_schema, **molecule_properties_schema}) or update this docstring?

[d0choa]

This comment sits above a select that only renames columns. The Decimal → Float64 conversion happens much later, in .cast(molecule_properties_schema), so _molecule_class compares a still-Decimal mwFreebase against PEPTIDE_MAX_MW. Casting the decimal columns in this select would make the comment true and mean the 1,500 Da comparison runs on the final type.

[d0choa]

This hunk and the compound_structures schema one above (line 407) are formatting-only changes to existing tests. Could you drop them from this PR? They make the diff and the blame bigger without changing behaviour.