Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,10 +128,13 @@ their repo-root counterparts. Install-specific entrypoints live in `.codex/`,

| Variable | Default | Description |
| ------------------------ | ------------------------ | ------------------------------------------ |
| `LUMEN_BACKEND` | `ollama` | Embedding backend (`ollama` or `lmstudio`) |
| `LUMEN_BACKEND` | `ollama` | Embedding backend (`ollama`, `lmstudio`, or `openai`) |
| `LUMEN_EMBED_MODEL` | see note ¹ | Embedding model (must be in registry) |
| `OLLAMA_HOST` | `http://localhost:11434` | Ollama server URL |
| `LM_STUDIO_HOST` | `http://localhost:1234` | LM Studio server URL |
| `OPENAI_BASE_URL` | — | OpenAI-compatible server URL (`openai` backend) |
| `OPENAI_API_KEY` | — | Bearer token for the `openai` backend |
| `LUMEN_EMBED_SKIP_HEALTH_CHECK` | `false` | Skip `/v1/models` probe (`openai` backend) |
| `LUMEN_MAX_CHUNK_TOKENS` | `512` | Max tokens per chunk before splitting |
| `LUMEN_VECTOR_STORAGE` | `int8` | Vector precision (`int8` or `float32`) |

Expand Down
40 changes: 38 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,9 +273,12 @@ All configuration is via environment variables:
| Variable | Default | Description |
| ------------------------ | ------------------------ | ------------------------------------------------------------- |
| `LUMEN_EMBED_MODEL` | see note ¹ | Embedding model; use with `LUMEN_EMBED_DIMS` for unlisted models |
| `LUMEN_BACKEND` | `ollama` | Embedding backend (`ollama` or `lmstudio`) |
| `LUMEN_BACKEND` | `ollama` | Embedding backend (`ollama`, `lmstudio`, or `openai`) |
| `OLLAMA_HOST` | `http://localhost:11434` | Ollama server URL |
| `LM_STUDIO_HOST` | `http://localhost:1234` | LM Studio server URL |
| `OPENAI_BASE_URL` | — | OpenAI-compatible server URL (required for `openai` backend) |
| `OPENAI_API_KEY` | — | Bearer token for the `openai` backend; omit for gateways that trust network position |
| `LUMEN_EMBED_SKIP_HEALTH_CHECK` | `false` | Skip the `/v1/models` probe (`openai` backend); needed when a gateway doesn't expose that endpoint |
| `LUMEN_MAX_CHUNK_TOKENS` | `512` | Max tokens per chunk before splitting |
| `LUMEN_VECTOR_STORAGE` | `int8` | Vector precision (`int8` or `float32`) |
| `LUMEN_EMBED_DIMS` | — | Override embedding dimensions (required for unlisted models) |
Expand Down Expand Up @@ -304,7 +307,9 @@ of the database path hash, so different models never collide.
> **Caveat**: the DB path hash includes the model name but not the backend. If
> the same model name is configured on two backends (e.g. an Ollama and an LM
> Studio entry both named `foo`), they share the same index — use distinct
> model names per backend to avoid collisions.
> model names per backend to avoid collisions. This applies to the `openai`
> backend too — a model name shared with an Ollama or LM Studio entry collides
> in the index cache.

### Selecting a server per invocation

Expand Down Expand Up @@ -348,6 +353,37 @@ LUMEN_EMBED_DIMS=4096
LUMEN_EMBED_CTX=40960 # optional, defaults to 8192
```

### Remote / internal OpenAI-compatible servers

The `openai` backend targets any service exposing an OpenAI-compatible
`/v1/embeddings` endpoint — OpenAI itself, or an internal gateway. Configure
it via `config.yaml`:

```yaml
servers:
- backend: openai
host: https://api.example.com
model: text-embedding-3-small
dims: 1536
api_key: sk-... # optional — omit for gateways that trust network position
skip_health_check: true # optional — set when the gateway doesn't expose /v1/models
```

Or via environment variables:

```sh
LUMEN_BACKEND=openai
OPENAI_BASE_URL=https://api.example.com
OPENAI_API_KEY=sk-...
LUMEN_EMBED_MODEL=text-embedding-3-small
LUMEN_EMBED_DIMS=1536
```

`skip_health_check` exists because many custom gateways proxy a different
provider underneath (Bedrock, Gemini, etc.) and don't implement `/v1/models`
in the OpenAI shape, or don't implement it at all. Without it, Lumen's health
probe would incorrectly mark a working server as unhealthy.

## Controlling what gets indexed

Lumen filters files through six layers: built-in directory and lock file skips →
Expand Down
8 changes: 4 additions & 4 deletions cmd/index.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ import (

func init() {
indexCmd.Flags().StringP("model", "m", "", "embedding model (default: $LUMEN_EMBED_MODEL or "+embedder.DefaultModel+")")
indexCmd.Flags().StringP("backend", "b", "", "embedding backend to select (\"ollama\" or \"lmstudio\"); disambiguates when --model is configured on multiple backends")
indexCmd.Flags().StringP("backend", "b", "", "embedding backend to select (\"ollama\", \"lmstudio\", or \"openai\"); disambiguates when --model is configured on multiple backends")
indexCmd.Flags().BoolP("force", "f", false, "force full re-index")
rootCmd.AddCommand(indexCmd)
}
Expand Down Expand Up @@ -195,9 +195,9 @@ func loadConfigWithFlags(cmd *cobra.Command) (*config.ConfigService, error) {
if model == "" && backend == "" {
return config.NewConfigService(path)
}
if backend != "" && backend != config.BackendOllama && backend != config.BackendLMStudio {
return nil, fmt.Errorf("unknown backend %q (must be %q or %q)",
backend, config.BackendOllama, config.BackendLMStudio)
if backend != "" && backend != config.BackendOllama && backend != config.BackendLMStudio && backend != config.BackendOpenAI {
return nil, fmt.Errorf("unknown backend %q (must be %q, %q, or %q)",
backend, config.BackendOllama, config.BackendLMStudio, config.BackendOpenAI)
}

cfg, selErr := config.NewConfigService(path, config.WithServerSelection(model, backend))
Expand Down
2 changes: 1 addition & 1 deletion cmd/search.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ func init() {
searchCmd.Flags().BoolP("force", "f", false, "force full re-index before searching")
searchCmd.Flags().Bool("trace", false, "print per-phase timing to stderr")
searchCmd.Flags().StringP("model", "m", "", "embedding model override")
searchCmd.Flags().StringP("backend", "b", "", "embedding backend to select (\"ollama\" or \"lmstudio\")")
searchCmd.Flags().StringP("backend", "b", "", "embedding backend to select (\"ollama\", \"lmstudio\", or \"openai\")")
rootCmd.AddCommand(searchCmd)
}

Expand Down
4 changes: 4 additions & 0 deletions cmd/stdio.go
Original file line number Diff line number Diff line change
Expand Up @@ -1132,6 +1132,10 @@ func (ic *indexerCache) handleHealthCheck(ctx context.Context, _ *mcp.CallToolRe
host := srv.Host
model := srv.Model

if srv.SkipHealthCheck {
return healthResult(backend, host, model, true, "skip_health_check is set; service was not probed"), nil, nil
}
Comment thread
SConaway marked this conversation as resolved.
Resolved

probeCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()

Expand Down
47 changes: 47 additions & 0 deletions cmd/stdio_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -487,6 +487,53 @@ servers:
}
}

func TestHandleHealthCheck_RespectsSkipHealthCheck(t *testing.T) {
for _, k := range []string{"LUMEN_BACKEND", "LUMEN_EMBED_MODEL", "LUMEN_EMBED_DIMS", "LUMEN_EMBED_CTX", "OLLAMA_HOST", "LM_STUDIO_HOST", "OPENAI_API_KEY", "OPENAI_BASE_URL", "LUMEN_EMBED_SKIP_HEALTH_CHECK"} {
t.Setenv(k, "")
}

dir := t.TempDir()
cfgFile := filepath.Join(dir, "config.yaml")

// /v1/models always 503s, simulating a gateway that doesn't implement it,
// even though /v1/embeddings itself works fine.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1/models" {
w.WriteHeader(http.StatusServiceUnavailable)
return
}
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()

if err := os.WriteFile(cfgFile, []byte(fmt.Sprintf(`
servers:
- backend: openai
host: %s
model: remote-embed
dims: 3
skip_health_check: true
`, srv.URL)), 0o644); err != nil {
t.Fatalf("WriteFile: %v", err)
}

svc, err := config.NewConfigService(cfgFile)
if err != nil {
t.Fatalf("NewConfigService: %v", err)
}
fe := embedder.NewFailoverEmbedder(svc)

ic := &indexerCache{embedder: fe, cfg: svc}
result, _, err := ic.handleHealthCheck(context.Background(), &mcp.CallToolRequest{}, HealthCheckInput{})
if err != nil {
t.Fatalf("handleHealthCheck: %v", err)
}
text := mustTextResult(t, result)
if !strings.Contains(text, "Status: OK") {
t.Fatalf("expected skip_health_check to report OK without probing /v1/models, got: %s", text)
}
}

func TestHandleHealthCheck_ModelMissingIsError(t *testing.T) {
for _, k := range []string{"LUMEN_BACKEND", "LUMEN_EMBED_MODEL", "LUMEN_EMBED_DIMS", "LUMEN_EMBED_CTX", "OLLAMA_HOST", "LM_STUDIO_HOST"} {
t.Setenv(k, "")
Expand Down
4 changes: 4 additions & 0 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ const (
BackendOllama = "ollama"
// BackendLMStudio is the backend identifier for LM Studio.
BackendLMStudio = "lmstudio"
// BackendOpenAI is the backend identifier for OpenAI-compatible remote
// embedding servers (OpenAI itself, or any internal gateway exposing the
// same /v1/embeddings wire format).
BackendOpenAI = "openai"
)

// DBPathForProject returns the default int8/512-token collection path. New
Expand Down
119 changes: 88 additions & 31 deletions internal/config/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,14 @@ import (

// ServerConfig holds per-server configuration.
type ServerConfig struct {
Backend string `koanf:"backend"`
Host string `koanf:"host"`
Model string `koanf:"model"`
Dims int `koanf:"dims"`
CtxLength int `koanf:"ctx_length"`
MinScore float64 `koanf:"min_score"`
Backend string `koanf:"backend"`
Host string `koanf:"host"`
Model string `koanf:"model"`
Dims int `koanf:"dims"`
CtxLength int `koanf:"ctx_length"`
MinScore float64 `koanf:"min_score"`
APIKey string `koanf:"api_key"`
SkipHealthCheck bool `koanf:"skip_health_check"`
}

// ConfigService wraps koanf and provides typed config access.
Expand All @@ -50,12 +52,27 @@ func defaultServerForBackend(backend string) ServerConfig {
Host: "http://localhost:1234",
Model: models.DefaultLMStudioModel,
}
default:
case BackendOpenAI:
// No sane localhost default for a remote/internal gateway — host and
// model must be set explicitly via env vars or config file.
return ServerConfig{
Backend: BackendOpenAI,
}
case BackendOllama:
return ServerConfig{
Backend: BackendOllama,
Host: "http://localhost:11434",
Model: models.DefaultOllamaModel,
}
default:
// Preserve whatever was passed instead of silently normalizing an
// unrecognized value to Ollama. This function is reached from
// applyEnvOverrides with a raw, unvalidated LUMEN_BACKEND value — a
// typo like "OpenAI" (wrong case) must surface as validate()'s
// "unknown backend" error, not silently redirect the server to
// localhost Ollama while dropping an api_key that was only valid for
// the originally configured backend.
return ServerConfig{Backend: backend}
}
}

Expand Down Expand Up @@ -131,14 +148,7 @@ func NewConfigService(configPath string, opts ...Option) (*ConfigService, error)
_ = k.Unmarshal("servers", &servers)
if len(servers) > 0 {
servers[0].Model = svc.modelOverride
serverMaps := make([]map[string]any, len(servers))
for i, s := range servers {
serverMaps[i] = map[string]any{
"backend": s.Backend, "host": s.Host, "model": s.Model,
"dims": s.Dims, "ctx_length": s.CtxLength, "min_score": s.MinScore,
}
}
_ = k.Load(confmap.Provider(map[string]any{"servers": serverMaps}, "."), nil)
_ = k.Load(confmap.Provider(map[string]any{"servers": serverConfigMaps(servers)}, "."), nil)
}
}

Expand All @@ -156,14 +166,7 @@ func NewConfigService(configPath string, opts ...Option) (*ConfigService, error)
// Drop the existing list first — koanf merge would otherwise keep
// stale entries beyond the filtered length.
k.Delete("servers")
serverMaps := make([]map[string]any, len(filtered))
for i, s := range filtered {
serverMaps[i] = map[string]any{
"backend": s.Backend, "host": s.Host, "model": s.Model,
"dims": s.Dims, "ctx_length": s.CtxLength, "min_score": s.MinScore,
}
}
_ = k.Load(confmap.Provider(map[string]any{"servers": serverMaps}, "."), nil)
_ = k.Load(confmap.Provider(map[string]any{"servers": serverConfigMaps(filtered)}, "."), nil)
}

if err := svc.validate(); err != nil {
Expand Down Expand Up @@ -235,9 +238,13 @@ func applyEnvOverrides(k *koanf.Koanf) {
ctx := os.Getenv("LUMEN_EMBED_CTX")
ollamaHost := os.Getenv("OLLAMA_HOST")
lmStudioHost := os.Getenv("LM_STUDIO_HOST")
openAIHost := os.Getenv("OPENAI_BASE_URL")
openAIKey := os.Getenv("OPENAI_API_KEY")
skipHealthCheck := os.Getenv("LUMEN_EMBED_SKIP_HEALTH_CHECK")

// Only apply if at least one server env var is explicitly set
hasOverride := backendEnv != "" || model != "" || dims != "" || ctx != "" || ollamaHost != "" || lmStudioHost != ""
hasOverride := backendEnv != "" || model != "" || dims != "" || ctx != "" ||
ollamaHost != "" || lmStudioHost != "" || openAIHost != "" || openAIKey != "" || skipHealthCheck != ""
if !hasOverride {
return
}
Expand All @@ -250,10 +257,30 @@ func applyEnvOverrides(k *koanf.Koanf) {
}
srv := servers[0]

// If backend is explicitly overridden, reset server[0] to backend-specific
// defaults first to avoid mixed config (e.g. lmstudio backend with Ollama host/model).
if backendEnv != "" {
// If backend is explicitly overridden to a DIFFERENT backend than what's
// currently configured, reset server[0] to backend-specific defaults
// first to avoid mixed config (e.g. lmstudio backend with Ollama
// host/model). Skip the reset when backendEnv just confirms the backend
// that's already configured (e.g. LUMEN_BACKEND=openai set redundantly
// alongside a config.yaml that already has backend: openai) — otherwise
// this would silently wipe an explicitly configured host/model. For
// ollama/lmstudio that wipe is masked by their localhost defaults (it
// just silently redirects a remote host back to localhost); for openai,
// which has no usable default host, it turns into a hard validation
// failure at startup. SkipHealthCheck is always carried forward across an
// actual backend switch — it's a harmless no-op for any backend. APIKey is
// only carried forward when switching INTO openai: validate() rejects a
// non-empty api_key on any other backend, so carrying it into e.g. ollama
// would turn a same-invocation `LUMEN_BACKEND=ollama` override (with an
// openai server left configured in config.yaml) into a hard startup
// failure instead of the intended backend switch.
if backendEnv != "" && backendEnv != srv.Backend {
prevAPIKey, prevSkipHealthCheck := srv.APIKey, srv.SkipHealthCheck
srv = defaultServerForBackend(backendEnv)
if backendEnv == BackendOpenAI {
srv.APIKey = prevAPIKey
}
srv.SkipHealthCheck = prevSkipHealthCheck
}

if model != "" {
Expand All @@ -272,12 +299,30 @@ func applyEnvOverrides(k *koanf.Koanf) {
if lmStudioHost != "" {
srv.Host = lmStudioHost
}
case BackendOpenAI:
if openAIHost != "" {
srv.Host = openAIHost
}
default:
if ollamaHost != "" {
srv.Host = ollamaHost
}
}

// OPENAI_API_KEY only applies to the openai backend — otherwise an
// unrelated OPENAI_API_KEY left in the environment (e.g. for another
// tool) would silently attach itself to an ollama/lmstudio server
// config, where it's never sent and would trip the api_key validation
// below for no reason.
if openAIKey != "" && selectedBackend == BackendOpenAI {
srv.APIKey = openAIKey
}
if skipHealthCheck != "" {
if b, err := strconv.ParseBool(skipHealthCheck); err == nil {
srv.SkipHealthCheck = b
}
}

if dims != "" {
if n, err := strconv.Atoi(dims); err == nil {
srv.Dims = n
Expand All @@ -291,14 +336,23 @@ func applyEnvOverrides(k *koanf.Koanf) {
servers[0] = srv

// Re-marshal servers back into koanf
serverMaps := make([]map[string]any, len(servers))
_ = k.Load(confmap.Provider(map[string]any{"servers": serverConfigMaps(servers)}, "."), nil)
}

// serverConfigMaps converts ServerConfig entries into the map literal shape
// koanf expects. All three call sites that rebuild the servers list (model
// override, server-selection filter, env override) must go through this
// helper so new fields aren't silently dropped on one of the paths.
func serverConfigMaps(servers []ServerConfig) []map[string]any {
out := make([]map[string]any, len(servers))
for i, s := range servers {
serverMaps[i] = map[string]any{
out[i] = map[string]any{
"backend": s.Backend, "host": s.Host, "model": s.Model,
"dims": s.Dims, "ctx_length": s.CtxLength, "min_score": s.MinScore,
"api_key": s.APIKey, "skip_health_check": s.SkipHealthCheck,
}
}
_ = k.Load(confmap.Provider(map[string]any{"servers": serverMaps}, "."), nil)
return out
}

func (s *ConfigService) MaxChunkTokens() int {
Expand Down Expand Up @@ -453,7 +507,7 @@ func (s *ConfigService) validate() error {
if srv.Backend == "" {
return fmt.Errorf("config: servers[%d]: backend is required", i)
}
if srv.Backend != BackendOllama && srv.Backend != BackendLMStudio {
if srv.Backend != BackendOllama && srv.Backend != BackendLMStudio && srv.Backend != BackendOpenAI {
return fmt.Errorf("config: servers[%d]: unknown backend %q", i, srv.Backend)
}
if srv.Model == "" {
Expand All @@ -466,6 +520,9 @@ func (s *ConfigService) validate() error {
if err != nil || (u.Scheme != "http" && u.Scheme != "https") {
return fmt.Errorf("config: servers[%d]: host %q must be a valid http/https URL", i, srv.Host)
}
if srv.APIKey != "" && srv.Backend != BackendOpenAI {
return fmt.Errorf("config: servers[%d]: api_key is only supported for the %q backend, got %q", i, BackendOpenAI, srv.Backend)
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
if s.serverDims(i) == 0 {
return fmt.Errorf("config: servers[%d]: cannot resolve dims for model %q — set dims explicitly", i, srv.Model)
}
Expand Down
Loading
Loading