Skip to content

Cache the changed build files instead of the public folder - #664

Merged
regdocs merged 1 commit into
frappe:developfrom
gi-yt:asset-store
Nov 11, 2026
Merged

regdocs merged 1 commit into
frappe:developfrom
gi-yt:asset-store

Conversation

@gi-yt

@gi-yt gi-yt commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

The asset cache used to store only apps///public, so any build output written elsewhere in the app was lost when a build was restored from the cache. It also skipped the steps bench build runs after bundling.

Build side (get_cached_app.py):

  • Touch a per-app time marker before build_assets, then list every path under apps/ changed since the marker with find -cnewer.
  • Skip node_modules (directories and the public/node_modules symlink) at any depth. Dependencies already come from the earlier yarn install, and the link is recreated on restore.
  • Pack the list into added_files_.tar.gz inside the image, using --no-recursion so listed directories don't pull in unchanged contents.

Restore side (get_cached_app.py):

  • extract_assets now overlays the changed files onto the app directory entry by entry instead of replacing public/ wholesale. Unchanged files are left alone, and existing symlinks are replaced, never written through.
  • Restore the sites/assets/ symlink and node_modules link as before.
  • Run bench compile-po-to-mo and the app's after_build hooks (new run_hooks.py), which bench build would otherwise have run.

Agent side (builder.py):

  • upload_app_assets fetches the prebuilt tarball in a single get_archive call, unwraps it and uploads it as is, skipping the upload when nothing changed.
  • Remove repack_public_folder, which is no longer needed.
  • Copy run_hooks.py into the build context, with a test covering it.

The asset cache used to store only apps/<app>/<app>/public, so any build
output written elsewhere in the app was lost when a build was restored
from the cache. It also skipped the steps bench build runs after bundling.

Build side (get_cached_app.py):
- Touch a per-app time marker before build_assets, then list every path
  under apps/<app> changed since the marker with find -cnewer.
- Skip node_modules (directories and the public/node_modules symlink) at
  any depth. Dependencies already come from the earlier yarn install, and
  the link is recreated on restore.
- Pack the list into added_files_<app>.tar.gz inside the image, using
  --no-recursion so listed directories don't pull in unchanged contents.

Restore side (get_cached_app.py):
- extract_assets now overlays the changed files onto the app directory
  entry by entry instead of replacing public/ wholesale. Unchanged files
  are left alone, and existing symlinks are replaced, never written through.
- Restore the sites/assets/<app> symlink and node_modules link as before.
- Run bench compile-po-to-mo and the app's after_build hooks
  (new run_hooks.py), which bench build would otherwise have run.

Agent side (builder.py):
- upload_app_assets fetches the prebuilt tarball in a single get_archive
  call, unwraps it and uploads it as is, skipping the upload when nothing
  changed.
- Remove repack_public_folder, which is no longer needed.
- Copy run_hooks.py into the build context, with a test covering it.
@mergify

mergify Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

This pull request is merged.

@regdocs
regdocs merged commit 0b73125 into frappe:develop Nov 11, 2026
3 of 6 checks passed
regdocs added a commit that referenced this pull request Nov 11, 2026
Cache the changed build files instead of the public folder (backport #664)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants