Skip to content

Stream Gloas execution payload bids over a websocket - #682

Open
JasonVranek wants to merge 2 commits into
gattaca-com:mainfrom
JasonVranek:jv/epbs-bid-stream
Open

JasonVranek wants to merge 2 commits into
gattaca-com:mainfrom
JasonVranek:jv/epbs-bid-stream

Conversation

@JasonVranek

Copy link
Copy Markdown

Replaces #670, which was stacked on #635 and closed when #635 merged and its branch was deleted. This is the same change rebased on main, with the auctioneer part adapted to #635's prebuilt, auctioneer-signed bids. Commit-Boost's ePBS bid stream (commit-boost-client#509, released in v0.12.0-rc1) is a client of this endpoint.

What this does

Before Gloas, a proposer client can open header_stream and receive the relay's improving bid until its deadline, instead of making one get_header request. Gloas replaces get_header with getExecutionPayloadBid, a POST, and has no stream. This PR adds the Gloas equivalent:

GET /eth/v1/builder/execution_payload_bid_stream/{slot}/{parent_hash}/{parent_root}/{proposer_pubkey}

The request upgrades to a websocket. Each time the top bid changes, the relay sends one binary frame, [kind = 1][fork = 7 (gloas)][ssz SignedExecutionPayloadBid], the same layout as the header stream. When the stream ends, the relay sends a close frame. Without it, a client cannot tell a stream that ended from one that failed.

A GET has no body, so the client sends its SignedBuilderRequestAuth in an X-Request-Auth header, as standard padded base64 of the ssz.

Admission

Every check runs before the upgrade, so a refused request gets an HTTP status and no socket opens. The cheap checks run first, and the BLS verify runs last:

  • The slot must be a Gloas slot by the chain's fork schedule. The client's Eth-Consensus-Version header is not trusted for this.
  • The same validate_header_request checks as header_stream: the slot being bid for, a registered proposer, timing.
  • The path pubkey must be the slot's registered proposer. A wrong key is refused before any signature work.
  • admit_header_stream decides whether this client may stream.
  • X-Request-Auth is length-capped at the base64 size of the largest valid auth (5612 characters), then decoded.
  • Last, the auth's slot and BLS signature are checked. This check is shared with the POST endpoint.

Implementation notes

  • The first commit moves header_stream's schedule and polling loop into StreamSchedule, without behaviour changes, so both streams run the same loop. The second commit adds the bid stream on top.
  • The bid stream reuses the header_stream config and admit_header_stream on purpose. An ApiProvider sees the same inputs for both streams and cannot tell PBS admission from Gloas admission, so the two streams share one admission policy.
  • ExecutionPayloadBidStream joins the condensed ProposerApi route set, so a relay serving ProposerApi serves the stream. admit_header_stream still gates every connection.
  • Each poll passes the bid the stream last sent, as its block hash and execution payment. When the top bid still matches it, the auctioneer answers BidUnchanged without signing, so a stream that polls every few ms costs one signature per new bid. The POST endpoint passes nothing, and its behaviour does not change.
  • Each frame carries the auctioneer's signed bid as it is.
  • The stream stops early when the auction has moved past the request: a later slot, a different proposer, or a payload already being delivered. It keeps polling when there is no bid yet, or when the auctioneer has not yet reached the requested slot.
  • header_stream does not send a close frame yet. That is left for a separate change.

Tested

On a Kurtosis Gloas devnet (mainnet preset, 12 s slots, a Lodestar beacon node), running this change with #681, and Commit-Boost v0.12.0-rc1 with two stream relay entries: this relay and a Lodestar builder (nflaig/lodestar#26) serving the same endpoint. The run predates the rebase over #680, which touches only crates/operator.

  • All 49 observed slots were built from a bid Commit-Boost took from a stream. Helix revealed the payload in all 49, each on the first attempt.
  • In the 17 slots where both builders streamed a bid, Commit-Boost returned both to the beacon node, which chose between them.
  • 10 of Commit-Boost's streams to this relay fell back to HTTP, all at startup: DNS, connection refused, and 400 proposer not registered before the proposer had registered.

Without #681 the stream worked the same way, and helix's streamed bids won 15 slots, but helix could not publish to the Lodestar beacon node, so no payload was revealed. #681 has the details.

Known limitation

The stream sends until the client's deadline, Date-Milliseconds + X-Timeout-Ms. getExecutionPayloadBid asks the builder to respond by that time, and the client discards anything later, so a frame sent in the last one-way network trip before the deadline arrives too late. If the one-way latency is longer than the streaming window (300 ms by default), no frame arrives in time at all. Ending the stream one return trip early would fix this, but the relay would first have to measure that trip, for example with a ping over the open stream. header_stream uses the same schedule and has the same gap, so both are left for a separate change.

Moves header_stream's schedule (handshake, window, start, end, interval)
into StreamSchedule, and its timed polling loop into StreamSchedule::run,
which calls a ProposerStream for each tick. build_frame takes any ssz bid.
The getExecutionPayloadBid request auth check (the auth's slot, then its
BLS signature) moves into verify_request_auth.

No behaviour changes: the terminating check moves from send_bid to the
top of the loop, where it ran before every send_bid call anyway.
GET /eth/v1/builder/execution_payload_bid_stream/{slot}/{parent_hash}/{parent_root}/{proposer_pubkey}
upgrades to a websocket and streams the proposer's Gloas bid as it
improves, the way header_stream does for get_header. Each update is one
binary frame, [kind = 1][fork = gloas][ssz SignedExecutionPayloadBid].
The stream ends with a close frame: a client cannot tell a dropped
socket from a failed stream.

A GET carries no body, so the SignedBuilderRequestAuth arrives in the
X-Request-Auth header as standard padded base64 of its ssz. Every check
runs before the upgrade, so a refused request is a plain HTTP status and
no socket opens. The slot must be a Gloas slot by the chain's schedule,
and the path pubkey must be the slot's registered proposer. The auth
header is length-capped and decoded only after these cheap checks, and
its BLS signature is verified last.

The stream reuses header_stream's config and admit_header_stream. An
ApiProvider decides admission from the same inputs for both streams and
cannot tell them apart, so the two share one admission policy. The
route joins the condensed ProposerApi set.

Each tick passes the bid the stream last sent, as its block hash and
execution payment. When the top bid still matches it, the auctioneer
answers BidUnchanged without signing, so a stream that polls every few
ms costs one signature per new bid. The POST endpoint passes nothing and
is unchanged. Each frame carries the auctioneer's signed bid as it is.

The stream ends early when the auction has moved past the request: a
later slot, another proposer, or a payload already being delivered.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant