Do we need Mac OS valkey binaries?

[roshkhatri]

Do we really need MacOS built binaries available for download from the website https://valkey.io/download/?

[davefojtik]

Probably not, but we absolutely need valkey/valkey-JSON:alpine there

[daniel-b2c2]

@roshkhatri

Yes, we do need them. Here's our use case.

We run Valkey as an embedded server in integration tests: each test starts a valkey-server on a random port and tears it down afterwards. The usual Java option for this, com.github.codemonstur:embedded-redis, still bundles Redis 6.2 binaries and nothing from Valkey, so we only use it to manage the process and supply our own binary. Our build is Bazel, and we want the server binary to be a hermetic, pinned dependency, the same way we treat compilers and other tools. That means a binary fetched by URL and checked by sha256, resolved per platform through a Bazel toolchain, with nothing assumed to be installed on the machine.

On Linux that works out of the box with the valkey-<version>-jammy-<arch>.tar.gz release artifacts. On macOS there's no equivalent, and every option we have is worse:

• Homebrew doesn't let us control the version. It only ships the current release, so there's no way to pin 7.2.x or to keep macOS developers on the same version as Linux CI. Everyone ends up on whatever brew upgrade last installed, which differs from machine to machine and changes under you without warning. Tests that pass on one laptop can fail on another for reasons that have nothing to do with the code.
• It's an extra, unmanaged install step. Every developer has to install Homebrew and Valkey and keep them in sync. That's outside the build and invisible to it, which is the opposite of hermetic.
• Homebrew bottles can't be used on their own. We tried fetching the bottle directly from ghcr.io. It's linked against Homebrew's OpenSSL through @@HOMEBREW_PREFIX@@ placeholders, so we also have to fetch the openssl bottle, rewrite the load commands with install_name_tool, and re-sign with codesign before it will run. That works, but it's fragile, and it breaks whenever Homebrew changes how the bottle is linked.
• Building from source in our build means maintaining a parallel build system for Valkey just to get a test binary on macOS.

Valkey's own CI already builds on macos-latest on every push, so macOS builds are already tested. The request is just to publish a macOS tarball alongside the Linux ones in the same release pipeline:

• valkey-<version>-macos-arm64.tar.gz
• Ideally with no dependencies outside the macOS system libraries, e.g. either built with BUILD_TLS=no or with OpenSSL linked statically, so the binary runs on any Mac without Homebrew.
• Published for every active release, (if you're feeling generous including maintenance branches, though we have no expectation here) e.g. 7.2.x, at a stable URL like the existing download.valkey.io/releases/ artifacts.

Signing and notarisation aren't needed for this use case. Binaries fetched by build tools aren't quarantined by Gatekeeper, and the ad-hoc signature the linker adds at build time is enough to run on Apple Silicon. Plenty of projects ship unsigned macOS tarballs this way. Correct?

This would make Valkey usable as a hermetic dependency in Bazel and in other build tools (Gradle, Maven test plugins, testcontainers-style tooling) on macOS, which is where most developers run their tests.

…unless you want to build a cross-platform embedded server Valkey library for Java - we'd be happy with that too as that's actually what we're trying to solve here.

[roshkhatri]

Hi @daniel-b2c2, Thanks for the explanation of your usecase. This use case makes sense. I’ll take a look at our release flow again and see how we can publish macOS binaries (unsigned) alongside the existing Linux artifacts.

I’ll post here if I need anything

[roshkhatri]

Is it possible to use valkey-container for the integration tests?

[daniel-b2c2]

Thank you for your response,

On valkey-container: that's actually what we use today in CI (partly). Our GitHub Actions CI runs a valkey container as a service for the duration of the test job, and we're moving away from it. It only works because our tests currently run on the same machine as the CI job. We plan to move to Bazel remote execution, where tests run as sandboxed actions on remote workers with no Docker daemon and often no network, so a shared container doesn't fit. Some remote execution setups can be made to support Docker-in-Docker, but it needs privileged workers and network access, and it's slow to start a container per test. Containers locally also mean every developer needs Docker installed and running, which is the same unmanaged-install problem as Homebrew.

A standalone binary is just another declared input: Bazel ships it to the worker and the test starts it on loopback, with nothing else required.

🙏