Skip to content

Expose GitHub search quotas and MCP 429 retry hints - #68728

Merged
pelikhan merged 9 commits into
mainfrom
copilot/log-code-search-rate-limit-buckets
Oct 14, 2026
Merged

pelikhan merged 9 commits into
mainfrom
copilot/log-code-search-rate-limit-buckets

Conversation

Copilot AI commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GitHub MCP search_code can exhaust its smaller search budget while core quota remains available. Missing bucket snapshots and buried 429 retry hints make these failures difficult to diagnose.

  • Quota visibility: Log core, code_search, and search limit, remaining, and reset values before and after agent execution, using GitHub tools credential precedence. End snapshots run on failure; logging remains nonfatal.
  • 429 diagnostics: Surface failed MCP responses and server retry hints across gateway/RPC logs, including legacy summaries. Exclude queries and URLs; avoid interpreting echoed queries or Retry-After values as HTTP statuses.
  • Documented alternative: Explain search budgets and recommend authenticated cross-repository checkout: with sparse-checkout for local code searches.

Example summary entry:

github | search_code | try again in 216.007769ms

Copilot AI and others added 2 commits October 14, 2026 15:36
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Log code_search/search rate-limit buckets and surface GitHub MCP search_code 429 hints Expose GitHub search quotas and MCP 429 retry hints Oct 14, 2026
Copilot AI requested a review from pelikhan October 14, 2026 15:43
Comment thread actions/setup/js/parse_mcp_gateway_log.cjs Outdated
@pelikhan
pelikhan marked this pull request as ready for review October 14, 2026 15:53
Copilot AI balanced review requested due to automatic review settings October 14, 2026 15:53
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · opus50 · 13.7 AIC · ⌖ 40 AIC · ◇ 36.5 AIC · ⊞ 1.8K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · haiku45 · 4.99 AIC · ⌖ 7.1 AIC · ◇ 2.45 AIC · ⊞ 8.3K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · gpt56 · 10.9 AIC · ⌖ 5.26 AIC · ⊞ 14.3K · ◷

Generated by Ponytail Reviewer for #68728

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

Copilot AI and others added 2 commits October 14, 2026 16:00
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI requested a review from pelikhan October 14, 2026 16:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The parser can misclassify echoed query text and discard valid HTTP-date retry hints.

2 open findings
What changed in this PR

Improves diagnostics for GitHub MCP search quota exhaustion through rate-limit snapshots, surfaced retry hints, and documented alternatives.

Changes:

  • Adds nonfatal start/end quota snapshots using GitHub tool credential precedence.
  • Summarizes MCP 429 responses and retry hints.
  • Documents sparse checkout as an alternative to remote code search.

No confirmed vulnerabilities found.

File Description
pkg/​workflow/​compiler_yaml_main_job_test.go Tests snapshot ordering and credentials.
pkg/​workflow/​compiler_yaml_ai_execution.go Generates quota snapshot steps.
docs/​src/​content/​docs/​reference/​github-tools.md Documents quotas and local search.
actions/​setup/​js/​parse_mcp_gateway_log.test.cjs Tests 429 diagnostics.
actions/​setup/​js/​parse_mcp_gateway_log.cjs Extracts and summarizes retry hints.
actions/​setup/​js/​github_rate_limit_logger.test.cjs Tests visible quota logging.
actions/​setup/​js/​github_rate_limit_logger.cjs Logs selected quota buckets.

🧠 Review effort: Balanced


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread actions/setup/js/parse_mcp_gateway_log.cjs Outdated
Comment thread actions/setup/js/parse_mcp_gateway_log.cjs Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /tdd and /grill-with-docs (fallback triage: pr-triage succeeded; see below). One actionable, non-blocking issue found.

📋 Key Themes & Highlights

Key Theme

  • Unaddressed prior feedback: pelikhan's existing review comment asking for progressive disclosure (<details>/<summary>) on the new "MCP rate limits (HTTP 429)" summary block was not applied in this revision — the block still renders as an always-expanded heading + table, inconsistent with the token-steering/DIFC-filtered sections right next to it in generateGatewayLogSummary.

Positive Highlights

  • ✅ getMcpRateLimitHint/getRpcRateLimitHint carefully scrub query text and URLs before surfacing retry hints — verified with targeted regex tests against adjacent 4xx/5xx noise, Retry-After values that look like status codes, and legacy tool_call error shapes.
  • ✅ Strong /tdd coverage: parametrized tests cover duration formats, Retry-After edge cases, HTML-escaping of tool labels, and an end-to-end main() test across gateway.jsonl/rpc-messages.jsonl/legacy log combinations.
  • ✅ compiler_yaml_ai_execution.go reuses collectMCPEnvironmentVariables for token precedence rather than duplicating resolution logic, and compiler_yaml_main_job_test.go locks down step ordering (start snapshot before execution, end snapshot with if: always() before failure handling, artifact path included).
  • ✅ Docs update clearly explains the code_search vs core quota distinction and links the sparse-checkout alternative with a concrete config example.

No security or correctness blockers identified; the review comment above is a consistency/maintainability nit that should be quick to apply.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 81.8 AIC · ⌖ 14.7 AIC · ⊞ 10.4K
Comment /matt to run again

Comment thread actions/setup/js/parse_mcp_gateway_log.cjs
@github-actions

Copy link
Copy Markdown
Contributor

Comment Memory

Peek at saved memory (pr-code-quality-reviewer)
reviewed_at: 2026-10-14T15:57:33Z
review_event: COMMENT
top_themes:
- false-positive MCP 429 detection
- unconditional GitHub quota snapshots
files_reviewed:
- actions/setup/js/github_rate_limit_logger.cjs
- actions/setup/js/github_rate_limit_logger.test.cjs
- actions/setup/js/parse_mcp_gateway_log.cjs
- actions/setup/js/parse_mcp_gateway_log.test.cjs
- docs/src/content/docs/reference/github-tools.md
- pkg/workflow/compiler_yaml_ai_execution.go
- pkg/workflow/compiler_yaml_main_job_test.go
comment_count: 2

Note

This comment is managed by comment memory.

Expand the saved memory block to view or edit the persistent context for this thread.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 82.2 AIC · ⌖ 6.09 AIC · ⊞ 21.3K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict

Useful diagnostics overall, but I found two correctness holes that make the new output easier to misread than it should be.

Highlights
  • The new 429 parser can still promote echoed query text into a fake rate-limit error when the backend message does not carry a later transport status.
  • Agent start and end quota snapshots currently run even when the GitHub tool is disabled, which burns quota and makes the logs look authoritative for workflows that never had GitHub MCP access.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 82.2 AIC · ⌖ 6.09 AIC · ⊞ 21.3K
Comment /review to run again

Comment thread actions/setup/js/parse_mcp_gateway_log.cjs Outdated
Comment thread pkg/workflow/compiler_yaml_ai_execution.go Outdated
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (actions/setup/js/parse_mcp_gateway_log.cjs:798): This still interprets status-like text inside an echoed query or URL. For example, an isError message such as failed to search code with query 'status: 429': validation failed produces a rate-limit row even though no HTTP 429 occurred; choosing the last match only avoids this when a later real status is present. Remove quoted query/URL regions before status detection (or parse only recognized transport-status forms), and add a regression case where the echoed 429 is the only status. - Expose GitHub search quotas and MCP 429 retry hints #68728 (comment)
  3. Review (actions/setup/js/parse_mcp_gateway_log.cjs:804): Retry-After also permits an HTTP-date, but this accepts only numeric seconds and reports "No retry hint provided" for a valid hint such as Retry-After: Wed, 21 Oct 2015 07:28:00 GMT. Support the standard date form as well so the summary does not discard a server-provided retry hint. - Expose GitHub search quotas and MCP 429 retry hints #68728 (comment)
  4. Review (actions/setup/js/parse_mcp_gateway_log.cjs:858): [/grill-with-docs] The author's own prior review comment on this block ("use progressive disclosure + details/summary like other step summary sections") has not been addressed — generateMcpRateLimitSummary still emits a flat ### MCP rate limits (HTTP 429) heading + table directly via core.summary.addRaw, unlike generateTokenSteeringSummary/generateDifcFilteredSummary/the gateway-log sections nearby, which all wrap their content in <details><summary>...</summary>. - Expose GitHub search quotas and MCP 429 retry hints #68728 (comment)
  5. Review (actions/setup/js/parse_mcp_gateway_log.cjs:796): This still treats an echoed HTTP 429 inside the search query as a real transport status when the backend message does not include a later status, so user input can fabricate a rate-limit warning and hide the real error class. - Expose GitHub search quotas and MCP 429 retry hints #68728 (comment)
  6. Review (pkg/workflow/compiler_yaml_ai_execution.go:21): Passing a synthetic GitHub tool set here forces start and end quota probes even when tools.github is absent or explicitly disabled, so workflows with no GitHub MCP usage still burn two core requests and emit misleading quota snapshots. - Expose GitHub search quotas and MCP 429 retry hints #68728 (comment)

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: a6f94b7
Sous-chef state: e29a040502632145e31f63cdd53d846aa707939fe82edc52caae705776dfb30a

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 16.2 AIC · ⌖ 5.98 AIC · ⊞ 1.1K · ◷
Comment /souschef to run again

Copilot AI and others added 2 commits October 14, 2026 16:44
…ch-rate-limit-buckets

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Harden echoed-query masking, resolve quota probes from the actual MCP tool set, and persist sanitized HTTP 429 retry hints alongside GitHub quota snapshots in the unified session.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Fix failing check build-wasm (FAILURE): https://github.com/github/gh-aw/actions/runs/38606595776/job/115999211704.
  3. Fix failing check impacted-go-tests (FAILURE): https://github.com/github/gh-aw/actions/runs/38606595776/job/115999211835.
  4. Fix failing check JS Tests (shard 4/4) (FAILURE): https://github.com/github/gh-aw/actions/runs/38606596205/job/115999174153.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: e70ec77
Sous-chef work: 31f7e812d45f0792093269b7b90445aa9803225a204ec130694b9ceb0227d7cc 42e5d572d9d75b7168fe7d0aeda0717cc07e65ed240f6d299e6fec661214f1ac 8acb61a077717e3a460018aacdfde96fd489f77606cd36169e2328d9079b75bb
Sous-chef state: 0f212b02da1d504ae9612957766ab2e0a7c110886d4eddfd64a64af41ba423fd

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 4.48 AIC · ⌖ 7.2 AIC · ⊞ 1.1K · ◷
Comment /souschef to run again

Keep unified session normalization independent of gateway summary logging shims, and record the intended agent-start/end quota steps in WASM fixtures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor Author

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

Main is merged locally, and the remaining parser and golden-fixture fixes are committed as ca388c6. Publishing is blocked: repeated report_progress attempts fail while rebasing over concurrent commit 3c2903a because the GPG signing service returns Bad Request. The existing review threads are already resolved; I have not bypassed signing.

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

The reported failures are fixed locally in ca388c6: wasm golden fixtures were regenerated, make build-wasm and impacted Go tests pass, and the parser test now asserts core.warning. Publishing is blocked by the GPG signing service returning Bad Request during rebase over concurrent commit 3c2903a; I have not triggered CI. The full local JS suite had a separate temp-file race in awf_reflect_summary.test.cjs, which passes in isolation.

@pelikhan
pelikhan merged commit afb53a2 into main Oct 14, 2026
46 of 47 checks passed
@pelikhan
pelikhan deleted the copilot/log-code-search-rate-limit-buckets branch October 14, 2026 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Log code_search/search rate-limit buckets and surface GitHub MCP search_code 429 hints

4 participants