Skip to content

design: Materialize SDK for SUBSCRIBE consumption and custom sinks #37463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bobbyiliev wants to merge 33 commits into
base: main
Choose a base branch
from
Open

design: Materialize SDK for SUBSCRIBE consumption and custom sinks #37463

Changes from 7 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
1a891a8
design: Add design doc for Subscribe SDKs and external sink framework
bobbyiliev Jul 6, 2026
0137bb1
design: Re-center Subscribe SDK on single-view core with cohorts as a…
bobbyiliev Jul 8, 2026
2101ea1
design: Record cohort consistency/availability boundaries and deferre…
bobbyiliev Jul 8, 2026
b0d52d1
design: Materialize SDK for SUBSCRIBE consumption and custom sinks
bobbyiliev Oct 7, 2026
e330280
design: Add future work on driver quirks to the Materialize SDK doc
bobbyiliev Oct 7, 2026
c096fbb
design: Release SDK packages like dbt-materialize and build Node from…
bobbyiliev Oct 7, 2026
4e147d5
design: Sharpen the object swap open question
bobbyiliev Oct 7, 2026
9b8fde3
design: Address review on limits, errors, turbopuffer tombstones, and…
bobbyiliev Oct 8, 2026
7a8343c
design: Make Rust the first SDK package and write the turbopuffer sin…
bobbyiliev Oct 8, 2026
7eec9f7
design: Address second review on replacements, swap race, sweep and c…
bobbyiliev Oct 8, 2026
01263d3
design: Tighten swaps, generations, commits, and end-of-stream detection
bobbyiliev Oct 8, 2026
aba1fd3
design: Limit sinks to SUBSCRIBE on a table, source, or materialized …
bobbyiliev Oct 8, 2026
77968ad
design: Specify the turbopuffer reader switch and classify errors by …
bobbyiliev Oct 9, 2026
866a544
design: Resolve or defer each open question with an owner
bobbyiliev Oct 9, 2026
16ab79a
design: Address review on re-snapshot isolation, sink ids, commit siz…
bobbyiliev Oct 9, 2026
637caa2
design: Address review on many-sink memory, native embedding, and a P…
bobbyiliev Oct 9, 2026
74211d7
design: Require a unique sink key, since the subscribe upsert envelop…
bobbyiliev Oct 9, 2026
d6e09db
design: Fix incarnation allocation, recovery, commit splitting, start…
bobbyiliev Oct 10, 2026
e5ea2e1
design: Scope recovery sweeps and narrow start errors
bobbyiliev Oct 10, 2026
bd0275c
design: Align checkpoints, end-of-stream probing, and error rules wit…
bobbyiliev Oct 12, 2026
e9fad74
design: Answer review on checkpoint stores, the sink loop, and turbop…
bobbyiliev Oct 14, 2026
e6f6247
design: Resend only while the checkpoint is unchanged, and accept a l…
bobbyiliev Oct 14, 2026
c998059
design: Track the RETAIN HISTORY dependency as a workstream item and …
bobbyiliev Oct 15, 2026
9432bd9
design: Define one sink API with sink_type and per-call guarantees
bobbyiliev Oct 15, 2026
eff63c9
design: Move the turbopuffer sink design to an appendix
bobbyiliev Oct 15, 2026
64b32b2
design: Split long paragraphs and turn step-by-step rules into lists
bobbyiliev Oct 15, 2026
f4ea167
design: Record the turbopuffer behaviors tested against the API
bobbyiliev Oct 16, 2026
9d562a1
design: Append batches for a slow target and bound the client buffer …
bobbyiliev Oct 16, 2026
c5d51f3
design: List what the prototype leaves for later and match the turbop…
bobbyiliev Oct 19, 2026
fdc3181
design: Describe the binding facade and how each package's types are …
bobbyiliev Oct 19, 2026
42cab88
design: Generate each package's value types from the protocol core's …
bobbyiliev Oct 19, 2026
5d59e6c
design: Address review on batches, result size, sweeps, key violation…
bobbyiliev Oct 19, 2026
78ccbc1
design: Fold create and commit into one sink write, from review
bobbyiliev Oct 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading