Skip to content

fix: link only the libc variant a linux platform asks for - #3041

Open
u9g wants to merge 2 commits into
aspect-build:mainfrom
u9g:fix-libc-constraints
Open

u9g wants to merge 2 commits into
aspect-build:mainfrom
u9g:fix-libc-constraints

Conversation

@u9g

@u9g u9g commented Sep 28, 2026 •

Copy link
Copy Markdown

Fixes #3040.

npm_translate_lock built each optional dependency's select() from its os and cpu alone, so on linux a package that ships glibc and musl builds linked both (for example @rollup/rollup-linux-x64-gnu and -musl, or @anthropic-ai/claude-agent-sdk-linux-x64 and -linux-x64-musl, about 235 MB each).

Changes

  • pnpm.bzl reads libc from pnpm-lock.yaml packages, beside os and cpu.

  • _collect_dep_constraints collects it, and it reaches npm_import_links_rule as a new internal deps_libcs attr, the same way deps_oss and deps_cpus do.

  • _to_deps_attr puts dependencies that declare os, cpu and libc in their own select(). On linux they are keyed on new @aspect_rules_js//platforms/pnpm:linux_<cpu>_<libc> config settings over the existing //platforms/libc constraint. Other OSes keep <os>_<cpu>, since pnpm only checks libc on linux.

  • Each such dependency is also keyed on linux_<cpu>_unconstrained. A platform that doesn't set //platforms/libc (its default is unconstrained) still links every variant, so nothing changes unless a platform asks for glibc or musl.

  • docs/pnpm.md gets a short "Platform-specific packages" section showing how to set it.

  • pnpm.to_bazel_constraints picks each dependency's select() group and conditions, which replaces the if/elif chain in _to_deps_attr. A dependency that declares libc without os or cpu is grouped with the libc dependencies too: a missing os or cpu allows all of them, as in pnpm, and aix folds into linux so that conditions never overlap.

  • os, cpu and libc are normalized to lists when the lockfile is read, since package.json allows a string as well ("libc": "glibc" in sass-embedded).

Test plan

  • New os_cpu_libc_constraints_test in pnpm_test.bzl; parse_pnpm_lock_tests expects the new libc field.
  • New constraints_test, more cases in os_cpu_libc_constraints_test, and a parse test for scalar and list os/cpu/libc.
  • examples/nextjs/snapshots/next_links_defs.bzl regenerated; //nextjs:write_next_links_defs_test passes on bazel 7.7.1.
  • snapshots/rollup_links_defs.bzl regenerated: the linux_* entries move into the libc select().
  • cquery on //:.aspect_rules_js/node_modules/rollup@4.55.2's direct deps, with a throwaway platform on the host:
    • no libc set: rollup-linux-x64-gnu and rollup-linux-x64-musl, as before
    • //platforms/libc:glibc: rollup-linux-x64-gnu only
    • //platforms/libc:musl: rollup-linux-x64-musl only
  • bazel test //npm/... //platforms/...: the 125 tests that load pass. A few targets under the root package didn't load where I ran this, because it couldn't fetch the example packages that come from GitHub commit tarballs; none of them involve this change.
  • buildifier (check + lint) and prettier 3.1.0 are clean on the changed files.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@CLAassistant

CLAassistant commented Sep 28, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@jbedard

jbedard commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Please fix the failing snapshot tests which will also provide a diff of the real results of this change helping the review process

":.aspect_rules_js/node_modules/@rollup+rollup-win32-ia32-msvc@4.55.2": "@rollup/rollup-win32-ia32-msvc",
},
"@aspect_rules_js//platforms/pnpm:win32_x64": {
":.aspect_rules_js/node_modules/@rollup+rollup-win32-x64-gnu@4.55.2": "@rollup/rollup-win32-x64-gnu",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the same fix for windows? 😨

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not with the data we have. @rollup/rollup-win32-x64-gnu and -msvc both declare only os: [win32] and cpu: [x64] in the lockfile, with no libc or other field to tell them apart, so pnpm installs both on win32 x64 as well. Choosing one would need a mapping from package names to toolchains that pnpm doesn't have, so I've left windows as is.

Comment thread npm/private/npm_import.bzl Outdated

for k, v in deps.items():
if k in deps_oss and k in deps_cpus:
if k in deps_oss and k in deps_cpus and k in deps_libcs:

@jbedard jbedard Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we have k in deps_cpus and k in deps_libcs but not k in deps_oss? Is that impossible and libc constraints only exist if os+cpu constraints also exist?

Either way it makes me wonder if this 3-if/else-block is no longer the ideal way to structure this code. I'm not sure if you have a better idea or if it's necessary at the time but please give it a thought...

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible, yes: npm doesn't require os or cpu alongside libc, though every native package I've seen declares all three. Before this fix, such a dependency fell back to the os/cpu-only branch and the libc was dropped.

I replaced the if/elif chain with pnpm.to_bazel_constraints(oss, cpus, libcs), which returns the select() group and the conditions for one dependency. _to_deps_attr now just buckets dependencies by group. Any dependency that declares libc goes in the libc group. A missing os or cpu means all of them, as it does in pnpm, so cpu: [x64], libc: [musl] becomes darwin_x64, win32_x64, … plus linux_x64_{musl,unconstrained}. aix maps to the linux constraint, so it's folded into linux to keep the conditions within one select() from overlapping. Covered in constraints_test and os_cpu_libc_constraints_test. The rollup snapshot is unchanged by the refactor.

@jbedard jbedard left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated Next.js links snapshot also needs to be updated. The current CI failure shows examples/nextjs/snapshots/next_links_defs.bzl still expecting the old linux_<cpu> selects while the generated output now contains linux_<cpu>_{glibc,musl,unconstrained}:

https://github.com/aspect-build/rules_js/actions/runs/36410446225/job/109039252901

The failure suggests running:

bazel run //nextjs:write_next_links_defs

--codex

Comment thread npm/private/pnpm.bzl Outdated
resolution = package_data["resolution"],
cpu = package_data.get("cpu", None),
os = package_data.get("os", None),
libc = package_data.get("libc", None),

@jbedard jbedard Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libc needs normalization here because it is valid as either a string or a list. npm's documented example uses "libc": "glibc", and pnpm preserves that scalar form; this repo already has examples in e2e/pnpm_lockfiles/v101/pnpm-lock.yaml (and v110/v120). As written, a scalar flows into deps_libcs, which is later passed to attr.string_list_dict() (or would be iterated character-by-character by _resolve_pnpm_constraint_values), so valid projects such as ones using sass-embedded will fail repository evaluation. Could we normalize a scalar to [value] and add coverage for both forms?

--codex

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. os, cpu and libc are now normalized to a list when the lockfile is read (_constraint_list in pnpm.bzl), so the scalar form used by sass-embedded in e2e/pnpm_lockfiles/v120 works. I did os and cpu too because package.json allows a string for them as well. The new parse test covers both the scalar and the list form.

@u9g
u9g force-pushed the fix-libc-constraints branch from fe10b72 to 28fa927 Compare November 3, 2026 04:43
u9g added 2 commits November 3, 2026 04:43
npm_translate_lock built each optional dependency's select() from its os and
cpu alone, so on linux a package with glibc and musl builds linked both. It
now reads libc from the pnpm-lock.yaml as well and, on linux, keys those
dependencies on new linux_<cpu>_<libc> config settings over
//platforms/libc. A platform that does not set //platforms/libc matches the
unconstrained setting and links every variant, as before.

Fixes aspect-build#3040
- pnpm keeps a package's [os], [cpu] and [libc] as written, and package.json
  allows a string as well as a list (sass-embedded uses "libc": "glibc").
  Normalize all three to a list when reading pnpm-lock.yaml.
- Replace the if/elif chain in _to_deps_attr with pnpm.to_bazel_constraints,
  which returns a dependency's select() group and conditions. A dependency
  that declares [libc] with only one of [os] and [cpu] now goes in the libc
  group too: a missing [os] or [cpu] allows all of them, as in pnpm, and aix
  folds into linux so the conditions in one select() never overlap.
- Regenerate examples/nextjs/snapshots/next_links_defs.bzl.
@u9g
u9g force-pushed the fix-libc-constraints branch from 28fa927 to 0b6e263 Compare November 3, 2026 04:43
@u9g

u9g commented Nov 3, 2026

Copy link
Copy Markdown
Author

Rebased on main. I regenerated examples/nextjs/snapshots/next_links_defs.bzl with bazel run //nextjs:write_next_links_defs, and the diff is what this change is meant to do: @next/swc-linux-*-gnu and -musl move from linux_<cpu> to linux_<cpu>_{glibc,musl,unconstrained}. //nextjs:write_next_links_defs_test passes locally on bazel 7.7.1, as does bazel test //npm/private/test:all //platforms/... (118 tests). The review fixes are in a separate commit, 0b6e263.

@u9g
u9g force-pushed the fix-libc-constraints branch from 0b6e263 to 3c121b7 Compare November 3, 2026 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: npm_translate_lock ignores libc, so glibc and musl variants are both linked on Linux

4 participants