Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,15 @@ test-e2e:
test/e2e/federation/run.sh
test/e2e/federation/tresor.sh

# a local vcpkg checkout, as duckdb-acl's `make vcpkg-setup`
# a local vcpkg checkout, as duckdb-acl's `make vcpkg-setup`: the commit duckdb's merged manifest names as its
# builtin-baseline - an overlay port that includes the checkout's own port must find the version it was written for
# (postgres_scanner's vcpkg-tool-meson 1.9.0 took the head's 1.12.1 portfile and downloaded 1.9.0 against 1.12.1's
# hash; duckdb-acl since 2026-10-23, here 2026-11-05)
VCPKG_COMMIT ?= $(shell sed -n 's/.*"builtin-baseline": "\([0-9a-f]*\)".*/\1/p' duckdb/scripts/merge_vcpkg_deps.py)
.PHONY: vcpkg-setup
vcpkg-setup:
@test -n "$(VCPKG_COMMIT)" || (echo "vcpkg-setup: no builtin-baseline in duckdb/scripts/merge_vcpkg_deps.py" && exit 1)
@test -d vcpkg || git clone https://github.com/microsoft/vcpkg.git vcpkg
@git -C vcpkg cat-file -e $(VCPKG_COMMIT)^{commit} 2>/dev/null || git -C vcpkg fetch origin
git -C vcpkg checkout -q --detach $(VCPKG_COMMIT)
./vcpkg/bootstrap-vcpkg.sh -disableMetrics
13 changes: 10 additions & 3 deletions docs/cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,16 @@ CLUSTER REMOVE EXTENSION <name> [IN GROUP <group>]
CLUSTER ATTACH '<path>' AS <alias> (TYPE <type> [, SECRET <secret>] [, <option> [<value>]] …)
[LINEAGE '<identity>'] [DEPENDS ON (<alias>, …)] [IN GROUP <group>] [COMMENT '<text>']
CLUSTER DETACH <alias> [CASCADE] [FORCE] [IN GROUP <group>]
CLUSTER SET <setting> = <value> [IN GROUP <group>]
CLUSTER SET <setting> = | TO <value> [IN GROUP <group>]
CLUSTER RESET <setting> [IN GROUP <group>]
```

The statements are read on duckdb's tokens (spec 020), like acl's own management statements: a comment may stand
anywhere; a quoted value is one plain literal (`'…'` or `"…"`, never `E'…'` or a string continued on the next line);
a list is read item by item - a trailing comma is allowed, an empty item or an empty list is not; each clause,
each option, each flag and each source a source depends on is written once; a value is a word (`4GB`, `v1.0`), a quoted value or a number (`-1`). A statement that does not read is
told how its form is written.

**A cluster node's** (spec 118): every hugr_node node is a cluster node, whatever the number of nodes - a cluster
may be one (hugr-node spec 002: there is no single-node mode and no setting to choose one). One node and many differ
only in where the policy catalog and the secrets service's state live - a local file or a shared database - which is
Expand Down Expand Up @@ -85,8 +91,9 @@ in its heartbeat; the orchestrator rolls the nodes (hugr-node spec 010, hugr-pla

- **No credentials, anywhere.** A path with a credential key is refused. That covers a conninfo
`password=`, a URI's `user:password@`, a SAS `sig=` and the like, and so does an option with such a
key. A postgres, mysql, mssql or hugr ([another platform](federation.md)) source must name a `SECRET`, and the
node resolves it from its secrets service. The refusal names the key, never its value.
key (`ENCRYPTION_KEY` too). A postgres, mysql, mssql or hugr ([another platform](federation.md)) source - by its
`TYPE`, or by its path's prefix without one (`'postgres:dbname=…'`) - must name a `SECRET`, and the node resolves
it from its secrets service. The refusal names the key, never its value.
- **The node's hardening is not a profile item.** `allow_unsigned_extensions`, `lock_configuration`,
`allow_persistent_secrets`, `allow_extension_repositories`, the extension and secret directories
and the like are refused - and (spec 118) the node's trust: `acl_allow_anonymous_admin`, the quack door's
Expand Down
2 changes: 1 addition & 1 deletion duckdb-acl
Submodule duckdb-acl updated 107 files
66 changes: 66 additions & 0 deletions specs/020-cluster-rules/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Spec 020: `ACL CLUSTER` read by rules on duckdb's tokens

- **Status**: implemented
- **Date**: 2026-11-05
- **Author**: hugr lab (duckdb-acl spec 150 stage 3g-3: no statement text parsed by hand; the owner's order of
2026-11-04 - 3g-3 moves the platform part's forms onto the rules, then the old seam goes)

## Problem

The CLUSTER forms were the last management statements read by a hand scanner (`ParseCluster` on acl's
`AdminScanner`, registered through `RegisterMgmtGrammar`): option lists and `DEPENDS ON` split as text at commas, a
comment refused, a quoted value taken however it was written. duckdb-acl's 3g-3a seam lets a module register PEG
rules for its statements instead (`RegisterMgmtRules` / `MgmtCall`, `acl_mgmt_rules.hpp`).

## Design

- duckdb-acl re-pinned to f2f520f (its spec 150 3g-3a - the seam - and 3g-3c - every management statement a rule,
the old seam gone; 27 commits from 8b938d1, the same duckdb, ext-common and quack pins).
- `RegisterClusterRules` (src/acl_cluster.cpp) registers five forms under the keyword `cluster`, each a rule headed by
`'CLUSTER'` with its transform and its usage: `AclClusterExtension` (install / update / remove),
`AclClusterAttach`, `AclClusterDetach`, `AclClusterSet`, `AclClusterReset`; helper rules `AclCluster…` (the option
list and `DEPENDS ON` read item by item, `IN GROUP`, VERSION / FROM / SHA256). Values through `MgmtCall`'s readers
(a quoted value one plain literal; a value acl's `AclValue` - a word, a quoted value or a signed number). The
compiled calls are the same `acl_cluster_*` calls as before.
- Kept: the SHA256 refusal in spec 103's words (whatever follows the word - its tokens to the statement's end are
the clause's), a key alone is `true`, TYPE lower-cased, SECRET split out, other option keys upper-cased into the
JSON.
- Changed, each now refused or read as duckdb's tokens read it: a comment anywhere is read; a trailing comma in a
list is read; an empty list item, a value of more than one token (`(TYPE=duckdb)`, `(X 'a' 'b')`, `DEPENDS ON (a
b)`), a bare path, `E'…'` and joined strings are refused; VERSION / FROM / an option written twice is refused (the
last used to win), and so are a DETACH flag and a `DEPENDS ON` source (an empty one too); `REMOVE … VERSION |
FROM | COMMENT` is refused in its own words; an empty list (`()`, `DEPENDS ON ()`) is refused; an option key in
quotes or with a dot (`("Q" 1)`, `(x.y 1)`) and `+1` are refused; `""` is the tokenizer's refusal; `SET x = -1`,
a value with a dot (`0.5`, `v1.0.0`, `a.b`) and the line breaks inside an option item (`(TYPE⏎duckdb)`, once read
as one key) are read. A statement the rules do not read is told the usage of its form (by its verb), else "ACL
CLUSTER expects …".
- Found by the review and fixed with it (both older than the rules): `ENCRYPTION_KEY` was no credential key - an
option holding a database's key was stored in the profile; a source whose path names its type (`'postgres:…'`, no
`TYPE`) skipped the SECRET requirement and did not keep its extension in the profile (`SourceTypeOf`: the TYPE, else
the path's prefix as duckdb reads it - a URL's `scheme://` names a filesystem, no type); `TYPE` / `SECRET` among
the options JSON of `acl_cluster_attach` (the function and `platform.cluster_attach` route) skipped the requirement
too - refused, they are the call's own arguments; a `depends_on` CSV with an empty or a repeated name is refused
there as in the grammar.
- Messages of acl's own forms moved with the re-pin (resource groups are rules since acl 3f): the tests'
expectations follow, and the schema window test's versions are acl's v23.

## Tests

`test/sql/cluster/cluster_grammar.test`: a comment, a value-less option, trailing commas, IN GROUP, `SET … TO`, a
negative value, the refusals and each form's usage; `acl_cluster_profile.test`: `ENCRYPTION_KEY`, a `postgres:` /
`mysql:` path without a secret; `test_acl_cluster_install.cpp`: a prefix-typed source keeps its extension. The suite
(`test/sql/*`, 14 files) and `make test-cpp` pass. A differential against the old `ParseCluster` over 561 statements
(160 from the tests) found no call compiled differently but the four the old reader misread.

## Alternatives considered

- Keeping the scanner until stage 4: the old seam is what 3g-3c removes from acl; hugr-node moving first keeps both
repositories building at every step.

## Follow-ups

- The profile's "never a credential" rests on keys: option VALUES are not linted (`(METADATA_PATH 'dbname=x
password=p')`, `(DATA_PATH 's3://b/p?s3_secret_access_key=…')`), DuckLake's `META_<option>` forwards a key one
prefix away (`META_ENCRYPTION_KEY`), settings carrying one are not refused (`azure_storage_connection_string`,
`azure_proxy_password`), `motherduck_token` is no key, and `ExtensionOfType` ignores duckdb's extension aliases
(`https://…` / `s3://…` vs httpfs). Found by this spec's reviews, all older than it - spec 021.
Loading