Repository navigation
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The lexical indexing path chunked files but never applied splitOversizedChunks, so generated files became single BM25 documents of up to 9,411 lines. One MCP search hit then returned 763k characters and overflowed the agent's tool output budget. Apply the same LUMEN_MAX_CHUNK_TOKENS split the semantic path uses; no merging so lexical documents stay symbol-scoped. IndexVersion 5 -> 6: existing lexical indexes contain the oversized documents and must be rebuilt. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude Code defers every MCP tool behind ToolSearch by default, so the model only saw the tool name and had to load it before use. In bench-swe runs Sonnet 5.5 called the search tool in 1 of 12 tasks and used Bash grep instead. Advertise anthropic/alwaysLoad on semantic_search so it is in context with its description from the first turn (Claude Code >= 2.1.285); index_status and health_check stay deferred. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- add --retrieval (bm25|semantic, default bm25); results dir is named after the retrieval mode and preflight only checks Ollama for semantic - build lumen with -tags fts5 in preflight - register the dynamic MCP server as plugin_lumen_lumen so the tool name matches what the plugin hooks tell Claude to call; since #179 the nudge referenced a tool that did not exist in bench sessions - accept both tool names in analysis and reports; show retrieval mode in the summary header Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lumen can index and search code locally without an embedding service. This PR makes code-aware BM25 the default, retains semantic retrieval as an opt-in, and bounds search responses across both modes.
Changes
lexical_onlyexposes the direct retrieval baseline.max_tokens(MCP) /--max-tokens(CLI): default 4000 estimated tokens, a hard 16,000-byte rendered-text ceiling;-1disables it. Preserve ranked locations first, share remaining space across snippets, and report truncation/omission while keeping original line ranges. Include headers, warnings, metadata and the CLI newline in the cap; normalize invalid UTF-8 before sizing to preserve the cap through MCP JSON.Evaluation
The catalog contains 445 outcomes: 417 supported and 28 diagnostic. Unlimited BM25 output passes 397/417 supported and 13/28 diagnostic outcomes. The 20 supported misses remain explicit failures in reports; regression checks reject new misses and require removing entries that start passing. See retrieval parity.
Budget evaluation used 445 executable MCP searches per configuration:
The default budget at 10 hits changed no case outcomes; all bounded responses stayed within their byte ceiling and parsed successfully. Tighter budgets or more hits can hide evidence even when its location remains visible. This is an output-size guardrail, not evidence of better ranking or lexical–semantic equivalence. Token counts are estimates; the byte ceiling is exact.
Validation and remaining work
make test,make lint, E2E-tag lint, andmake e2e-bm25passed locally during implementation../cmdtests, focused executable CLI/MCP budget and parser tests, and both lint configurations passed again. The full suite and valid-text catalog were not rerun after that correction.git diff --checkpassed. Opus 5.5 independently assessed the budget design and implemented it, including the MCP serialization regression fix.