Skip to content

Restore value-safe Android resource compilation avoidance - #50

Open
jbarr21 wants to merge 3 commits into
uber/android/8.8.0from
jb/resource-tracking-safety
Open

jbarr21 wants to merge 3 commits into
uber/android/8.8.0from
jb/resource-tracking-safety

Conversation

@jbarr21

@jbarr21 jbarr21 commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Restore resource-level compilation avoidance without reusing bytecode containing stale final Android resource IDs.

  • Read generated R field metadata directly from input jars without loading classes or running initializers.
  • Fingerprint field descriptors, finality, constant values, and owner-class shape so changes to inlined resource IDs and styleable indexes invalidate compilation.
  • Continue avoiding compilation for runtime-only non-final value changes and array-content changes.
  • Preserve other used classes, jar-scoped metadata keys, and independent dependency-level proof that an R jar is unused.
  • Fall back conservatively and symmetrically during cache lookup/update when provenance is missing, incomplete, or unsupported.

This targets uber/android/8.8.0 and builds on the already merged KSP metadata-context fix (#49).

Rollout requirement: Android must select both this Bazel fix and the companion rules_kotlin field-provenance fix before enabling resource-level invalidation.

Test Plan

Completed local validation:

  • ActionCacheCheckerTest: 87/87 cases pass, covering final-ID renumbering, styleable arrays/indexes, same-mtime replacements, incomplete metadata, mixed packages, unused jars, and tool-input invalidation.
  • Real Android/Robolectric replay on jbarr-j2k: the old Bazel reproduces stale final string-ID and styleable-index failures. Fixed Bazel plus the local rules_kotlin archive passes ten mutation/restore/no-op phases, totaling 20 real test executions.
  • Repeat with materialized Java dependency reports: another 20/20 test executions pass.
  • Full Helix first unused-string edit after a cache-restored seed:
    • Class-level policy: 1773.137s.
    • Resource-level policy: 1431.238s; Kotlin workers 658 → 5, KSP workers 477 → 4.
    • Resource-level policy plus --noexperimental_inmemory_jdeps_files: 803.213s; Javac workers 5659 → 6.
    • Combined reduction: 969.924s, or 54.701% of baseline wall time.
    • No-op control: 24.685s with zero compiler workers.

The 54.7% result requires both fork fixes and the consumer Java-jdeps setting; it is not attributable to this PR alone. This is one affected incremental edit, not a clean-build or fleet-wide estimate. The KSP fix was held constant.

Revert Plan

Return Android consumers to class-level invalidation before reverting this PR.

Revert this resource-tracking change while retaining the separate KSP metadata-context fix. No cache expunge or data migration is required.

Summary:

Restore resource-level local compilation avoidance without reusing bytecode containing stale final Android resource IDs.

- Read generated R field metadata from the actual input jar without loading classes or running initializers.
- Include descriptors, finality, and ConstantValue data for inlined IDs and styleable indexes; ignore runtime-only non-final values and array contents.
- Preserve owner-class shape and other used classes, scope metadata keys to each jar, and use symmetric conservative fallback for incomplete/unsupported provenance.
- Preserve independent dependency-level proof that a resource jar is unused. Missing usage information still keeps the jar required.

This builds on the already merged KSP metadata-context fix (PR #49) and targets uber/android/8.8.0. Android must not enable the resource-level policy until both this change and the companion rules_kotlin field-provenance change are available.

Test Plan:

- ActionCacheCheckerTest: 87 cases, zero failures, including final-ID renumbering, styleable arrays/indexes, same-mtime replacements, missing metadata, mixed packages, unused jars, and tool-input invalidation controls.
- Real Android/Robolectric replay on jbarr-j2k using locally built Bazel 8.8.0 and a local rules_kotlin release archive: the old Bazel reproduces stale string-ID and styleable-index failures. The candidate passes ten mutation/restore/no-op phases (20 real test executions); runtime-only array and unused-R-jar changes avoid Kotlin compilation.
- Repeat all ten fixed phases with Java dependency reports materialized: 20/20 test executions pass.
- Full-Helix incremental unused-string addition: class policy 1773.137s versus resource policy 1431.238s (19.282% faster); Kotlin workers 658 to 5 and KSP workers 477 to 4. This is one affected incremental build, not a clean-build/fleet estimate. Native binaries differ only by a resource-mode guard unreachable in class mode; the rules archive is identical.
- Full-Helix combined configuration (resource-level tracking plus materialized Java jdeps): 1773.137s to 803.213s, saving 969.924s (54.701% of baseline wall time). Java workers 5659 to 6, Kotlin 658 to 5, and KSP 477 to 4. Restore 752.845s; no-op 24.685s with zero compiler workers. All phases and source/configuration/artifact/restoration audits pass. This is one first resource edit after a cache-restored seed on j2k, not a fleet/clean-build estimate; the KSP fix is held constant.

Revert Plan:

Return Android consumers to class-level invalidation before reverting this PR. Revert this PR, leaving the separate KSP context fix intact. No cache expunge or data migration is required.
Summary:
- Use the same conservative class/full-jar representation on local cache PUT and GET when the global resource list includes fields absent from this input jar.
- Avoid perpetual misses caused by storing class hashes but validating a partial field map.
- Add repeated-hit controls for mixed resource packages and cache coverage for runtime arrays paired with inlined final indexes.

Test Plan:
- New mixed-package regression fails at the initial healthy-hit assertion in both UNUSED_RESOURCES and UNUSED_CLASSES_AND_RESOURCES before the fix.
- Full ActionCacheCheckerTest class passes all 82 cases with symmetric fallback, including used-field removal, final-ID renumbering, same-mtime jar replacement and KSP tool invalidation.
- Rebuilding the native Bazel candidate and rerunning the real Android/Robolectric ID-shift replay before the full-app recovery measurement.

Revert Plan:
Revert this change and keep resource-level avoidance disabled; the prior asymmetric fallback can repeatedly invalidate unchanged mixed-package consumers.
Summary:
- Let successful dependency-path provenance keep truly unused R jars out of
  resource-mode fingerprints, without dropping explicit or unknown inputs.
- Cover all three policies and preserve conservative no-usage-info behavior.

Test Plan:
The regression fails in both resource modes before the guard removal and
passes in class-only mode. The corrected ActionCacheCheckerTest suite
executes 87 cases with no failures on the isolated j2k bootstrap base.

Revert Plan:
Revert this commit; unused R jars will conservatively re-enter resource-mode
fingerprints, reducing compilation avoidance without enabling stale IDs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant