Skip to content

drm: use alpha variant when scanout plane lacks opaque variant - #496

Closed
joshuaswarren wants to merge 1 commit into
hyprwm:mainfrom
joshuaswarren:w88/xr24-to-ar24
Closed

joshuaswarren wants to merge 1 commit into
hyprwm:mainfrom
joshuaswarren:w88/xr24-to-ar24

Conversation

@joshuaswarren

@joshuaswarren joshuaswarren commented Nov 14, 2026 •

Copy link
Copy Markdown

[Joshua's agent]

Some display hardware offers only ARGB8888 and ABGR8888 for 8 bpc scanout, not XRGB8888 or XBGR8888. The internal display pipeline on a 14-inch M5 Max MacBook Pro is one example. Compositors that request XRGB8888 fail at startup with:
GBM: Failed to allocate a GBM buffer: format XR24 isn't supported by primary backend

This change allows those displays to function:

  • CGBMBuffer allocates the alpha format (ARGB8888 or ABGR8888) when the opaque format is requested for scanout, the scanout plane lacks the opaque format, and the scanout plane offers the alpha format.
  • The three commit-time format checks in DRM.cpp (setFormat validation, mismatch detection, async commit) accept the alpha format when the output state specifies the matching opaque format. This prevents spurious format-mismatch modesets and keeps async commits working.

Displays that offer the opaque format are unaffected.

Verified on hardware (a 14-inch M5 Max MacBook Pro), in user compositor sessions:

  • Colour fills and a translucent overlay window render and blend correctly (camera evidence).
  • Framebuffer readbacks show the alpha byte is 0xFF across all 375,732 inspected pixels during opaque rendering.

Not yet verified: startup through the display-manager greeter. The library was loaded per-session, so greeter startup with this build remains untested.

Some scanout planes (Apple M5 internal panel) offer only ARGB8888 and
ABGR8888 for 8 bpc, not XRGB8888 or XBGR8888. A compositor that asks
for XRGB8888 gets "format XR24 isn't supported by primary backend".

Allocate the alpha variant in that case, and let the three commit-time
format checks (setFormat validation, mismatch detection, async commit)
treat it as equal to the requested opaque format, so no modeset is
forced.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-14T04:02:57.995569Z 8fb4fab PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown

Hi @joshuaswarren,

Thanks for your contribution! Unfortunately, Hyprland and hypr* no longer
accept contributions from unvouched contributors.

To get vouched, please see our contributing guide at wiki.hypr.land/Contributing-and-Debugging

This PR has been closed automatically and will not be re-opened.

@github-actions github-actions Bot closed this Nov 14, 2026
@joshuaswarren

Copy link
Copy Markdown
Author

CHANGES on 8fb4fab
Cross-model review (GPT)

P2 — The commit-side exception is not conditional on the plane lacking the opaque format. GBM.cpp correctly preserves XRGB/XBGR allocation when offered, but DRM.cpp's mismatch and async predicates accept ARGB/ABGR for the opaque state on every plane. For a normal plane advertising both XRGB8888 and ARGB8888, an ARGB buffer with XRGB state previously triggered the format-update/modeset path (and async ENOTSUP); it now skips both. Thus the stated “opaque-capable displays unaffected” invariant does not hold when the consumer supplies a different format. Gate alias acceptance on the same per-output fallback condition, preserving ordinary mismatch handling; add both-format and alpha-only tests for all three checks.

Read allocator selection, renderable-modifier clipping and downstream DRM commit handling. Compiled the actual helper and checked mappings; a predicate witness confirms the normal-plane behavior change. Allocation uses the selected alpha format's own modifier list, not the opaque list.

Not checked: full dependency build, physical scanout, camera/readback receipts or greeter. The corrected PR text appropriately says greeter startup is unverified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant