Repository navigation
fix: retry the Go module fetch in every Go sample's Dockerfile - #245
Merged
Merged
Conversation
proxy.golang.org now and then resets an HTTP/2 stream partway through a download: go: golang.org/x/text@v0.31.0: read "https://proxy.golang.org/...": stream error: stream ID 187; INTERNAL_ERROR; received from peer The next attempt succeeds. But each Go sample's Dockerfile fetched its modules exactly once, with a bare `go mod download`, a bare `go mod tidy`, or a `go build` that fetches what it needs. So one reset failed the image build, and with it the CI lane of whichever repo was building the sample: the CI of keploy/keploy and of other keploy repositories clones this repo and runs `docker build` on these Dockerfiles. Each fetch now runs in a loop: 5 attempts, 5-20s apart, and then the build fails. A module fetch is idempotent and fails because of the network, so retrying it is safe. A missing module or a bad go.mod fails every attempt and still fails the build, about 50s later. - The 19 bare `go mod download`s and go-grpc's two `go mod tidy`s run in the loop. - echo-sql, gin-redis, mux-mysql, sse-svelte and users-profile fetched inside `go build`. They now run the looped `go mod download` first. - nethttp-mysql's `go mod download || true` ignored a failed fetch and left it to an un-retried `go mod tidy`. Both run in the loop now, so a fetch that fails five times fails the build. Built all 27 Dockerfiles: 25 build, and gin-pulsar and gin-redis fail for the same reason as before this change: their image's Go is older than their go.mod requires, which a separate commit fixes. With a GOPROXY that cuts the first download of one module partway, the original go-docker-timefreeze and go-grpc server builds fail ("unexpected EOF" on golang-jwt/jwt v5.3.0 and on grpc v1.69.2), and the looped ones recover on the second attempt. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Neither image built. gin-redis's go.mod says `go 1.23.0` and `toolchain go1.23.1`, which the Go in golang:1.20 cannot parse: go: errors parsing go.mod: /app/go.mod:3: invalid go version '1.23.0': must match format 1.23 /app/go.mod:5: unknown directive: toolchain gin-pulsar's go.mod says `go 1.25.0`, which golang:1.23 refuses: go: go.mod requires go >= 1.25.0 (running go 1.23.12; GOTOOLCHAIN=local) Each now builds on the Go minor release its go.mod requires, golang:1.23 and golang:1.25, and both images build. gin-pulsar's README asked for Go 1.23+ as well; it now asks for 1.25+. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
build.yml runs `go build ./...` in every top-level sample with a
go.mod, and that build fetched the sample's modules from
proxy.golang.org with no retry. One reset stream ("stream error: stream
ID N; INTERNAL_ERROR; received from peer") would fail the whole job.
Each sample now runs `go list -deps ./...` first, in a loop: 5
attempts, 5-20s apart, and then the job fails. `go list -deps` loads
the packages `go build` loads, so it fetches the same modules and no
more. `go mod download` would also fetch every module go.mod requires,
whether the build needs it or not.
Run as build.yml runs it, with Go 1.27.0 and an empty module cache, all
44 modules build, both with and without this change. With a GOPROXY
that cuts the first download of golang-jwt/jwt partway, the old step
fails on go-docker-timefreeze ("unexpected EOF"), and the new one
recovers on the second attempt.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
slayerjain
force-pushed
the
fix/retry-go-module-fetch-in-dockerfiles
branch
5 times, most recently
from
October 17, 2026 12:03
68393bc to
3752fa5
Compare
…d retry loop
keploy's CI clones this repo at main and builds these Dockerfiles, so a
Go module fetch made only once in one of them fails a keploy lane on a
reset download: keploy enterprise pipeline 11675 failed on go-grpc's
`RUN go mod tidy` ("stream error: stream ID 167; INTERNAL_ERROR").
.github/scripts/check_go_fetch_retried.py reads every Dockerfile. It
reads continuations as one instruction, with each problem at its first
line, and drops comment and blank lines inside a continuation, as Docker
does. Quoted text, arithmetic and comments are blanked first. go is
found alone or by its path (/usr/local/go/bin/go), in backticks, and in
the script of an `sh -c`/`bash -c` (any flags: -e, -o pipefail,
--login, -c --). A here-document's lines are skipped by line number, so
a continuation inside one cannot run its delimiter into the next line.
In each stage:
- every fetch must be the condition of an `until` (or `while !`) loop
(under env assignments and a `timeout N`, if any) in the RUN's own
shell, not a subshell, an sh -c, a pipeline or the background (nor in
a brace group that is), whose exit leaves only that shell. In the loop's own body (to its `; done`),
a counter `n=$((n+1))`, never reset there, and a test of it that fails
the build after 2 to 10 tries: `[ "$n" -lt N ] || exit K` (also
`test`, `|| { ...; exit K; }`, `-ge N ] && exit K`, `if ...; then
...; exit K` within the then-block), K > 0, the test and the count
outside any subshell. A fetch is `go mod download`, `tidy`,
`vendor`, any `go get` (it adds and upgrades requirements), or `go
install`/`run` of a module at a version. One with -e is refused: it
succeeds on a fetch that failed, so its loop never retries.
- the first build must come after a loop of `go mod download`/`tidy`/
`vendor` for its module: in its WORKDIR or one above it, in the stage,
in one it is built FROM (with that stage's WORKDIR), or in one whose
whole module cache it COPYs to the same place, and no ENV or ARG has
moved GOMODCACHE or GOPATH from the image's own since. A build is go build, test, run,
vet, generate, list, tool, mod why, mod graph, or install of the
module itself: it fetches what the module cache lacks, once. A tool installed at a
version fetches nothing of the module's.
- a loop in a RUN with a cache mount at the module cache fills that
mount. Only a RUN with the same mount (by id, else target) sees it,
and such a mount hides the image's own module cache.
- a RUN this cannot read is refused: exec form, a here-document (its
delimiter read as written, quoted or not), or go in a quoted string
run by anything but echo or printf (su -c, eval), or piped into a
shell (or through xargs into one). A COPY or ADD here-document's body
is skipped too. Not read at all: what a script or Makefile the RUN
runs does.
The new dockerfile-fetch job in build.yml runs it, after a self-test on
120 fixtures. 71 of 72 mutants of the rule fail it; the other
changes nothing a real Dockerfile shows (a COPY of three paths, the
first two the same).
It flags all 27 Go Dockerfiles on main (50 lines) and none with this
branch's loops.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
slayerjain
force-pushed
the
fix/retry-go-module-fetch-in-dockerfiles
branch
from
October 17, 2026 12:10
3752fa5 to
903624c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
proxy.golang.org now and then resets an HTTP/2 stream partway through a module download:
The next attempt succeeds. But every Go sample's Dockerfile here fetched its modules exactly once, using a bare
go mod download, a barego mod tidy, or ago buildthat fetches what it needs. One reset therefore fails the image build. It also fails the CI lane of whichever keploy repo was runningdocker buildon the sample: keploy/keploy's workflows and other keploy CI clone this repo atmainand build these Dockerfiles. This repo's ownbuild.ymlhas the same exposure through its barego build ./....Changes
fix: retry the Go module fetch in every Go sample's Dockerfile(27 Dockerfiles). Each fetch now runs in a loop: 5 attempts, 5-20s apart, then the build fails.go mod downloads and go-grpc's twogo mod tidys run in the loop.go build. They now run the loopedgo mod downloadfirst.go mod download || trueignored a failed fetch and left it to an un-retriedgo mod tidy. Both are looped now.A module fetch is idempotent and fails only because of the network, so retrying it is safe. A missing module or a bad go.mod fails every attempt and still fails the build.
fix(gin-redis, gin-pulsar): build them on a Go their go.mod accepts. Neither image built before this PR. gin-redis's go.mod needs Go 1.23 but the image wasgolang:1.20; gin-pulsar's needs Go 1.25 but the image wasgolang:1.23.ci: retry each sample's module fetch before building it.build.ymlnow runsgo list -deps ./...in a loop, the same 5 attempts, before eachgo build ./....go list -depsloads the same packagesgo buildloads, so it fetches nothing more.ci: check that every Go module fetch in a Dockerfile runs in a bounded retry loop..github/scripts/check_go_fetch_retried.py, run by a newdockerfile-fetchjob inbuild.ymlafter a self-test on 26 fixtures, reads every Dockerfile. In each stage:go mod download/tidy/vendor,go install/get/runat a version) must be the condition of anuntilloop that fails the build when it runs out;go build/test/run/vet/generate/listmust come after a looped module fetch, in the stage, in one it is built FROM, or in one whose module cache it copies;It flags all 27 Go Dockerfiles on
main(50 lines) and none on this branch. keploy enterprise pipeline 11675 failed on go-grpc'sRUN go mod tidy("stream error: stream ID 167; INTERNAL_ERROR"), which this PR loops.Rebased onto
main(07c5ba9) on 2026-10-17. No Go Dockerfile added since this PR was opened is missed: the guard reads them all.Verification
docker build --checkreports no warnings on any of them.go-docker-timefreezeandgo-grpcserver Dockerfiles fail withunexpected EOF(on golang-jwt/jwt v5.3.0 and on grpc v1.69.2). The looped ones recover on the second attempt.build.yml's step: the old step fails on go-docker-timefreeze, and the new one recovers.build.yml's step was run as written, with Go 1.27.0 and an empty module cache: all 44 modules build, both with and without this change.sededits to these Dockerfiles. Two edit them: one rewrites go-docker-timefreeze'sRUN CGO_ENABLED=0 GOOS=linux go build -o /main .line, and keploy's gin_mongo macOS script rewrites gin-mongo'sEXPOSE 8080. This PR leaves both lines unchanged.