Commit Graph

171 Commits (22cbcdfa5789b33dbb8d47532c21b24da2fdf5aa)

Author SHA1 Message Date
lyingbug 4640a1ea0c
test(rerank): pin every vendor's request, and correct three facts the table surfaced (#3513) 2026-09-21 20:01:09 +08:00
lyingbug 41dde3b05f
refactor(models): delete the internal/models/provider façade (#3506) 2026-09-21 14:41:48 +08:00
lyingbug 4a9aa06274
refactor(asr): move the wire format into protocol packages, and cover every vendor whose docs offer ASR (#3503) 2026-09-21 13:25:27 +08:00
lyingbug 9030f556f2
refactor(embedding): move the wire formats into protocol packages, and correct the vendor facts (#3500) 2026-09-21 11:55:38 +08:00
lyingbug d6c7665cb5
refactor(rerank): move the wire formats into protocol packages, and correct the vendor facts (#3488)
把 8 个手写 rerank client 收敛成 3 个协议包(internal/models/api/{cohererank,dashscoperank,nimrerank}),每个带一份从厂商文档逐字粘来的 golden fixture。LKEAP 和火山保留官方 SDK,但只实现 api.Reranker。rerank.NewReranker 改成 catalog.Resolve + 按协议分发,函数体里没有厂商名。生产代码净 -147 行。

三件原来每个 client 各管各、现在由共用层统一的事:分批(按厂商声明的上限,智谱 128 / NIM 512 等此前无人执行)、分数标度(NIM 与 GPUStack 返回 logit,其余为 0~1 概率,统一换算后再与 RerankThreshold 比较)、NIM 的 truncate(默认 NONE 会让超长 passage 整请求失败,此前从未发送)。

对照官方文档订正四处厂商事实:GPUStack 分数标度、千帆三项上限、阿里 500 条上限、qwen3-rerank 是第二种协议(隐藏并在构造期拒绝)。Novita / OpenRouter / LiteLLM 三家官方有 rerank 但未开类型,本次开放,零协议代码。

运行时行为变更:NVIDIA 起发 truncate: END;GPUStack 分数按 logit 换算(可按行覆盖);truncate_prompt_tokens 收窄为厂商声明制;qwen3-rerank 存量行从静默打错变为构造期报错。详见 PR 描述。

关联 #3486。
2026-09-21 10:36:26 +08:00
lyingbug 9aa63c866f
fix(models): correct four vendor request shapes that contradict the docs (#3489)
PR #3470 逐条核对了 27 家厂商,但有四处(五个 entry)的声明与厂商文档对不上,
其中一处在默认路径上就是错的。

1. DeepSeek:参考文档写明 "required and named tool choices are not supported
   in thinking mode; the API returns a 400 error",而 vendor.go 的事实注释写成
   「plain OpenAI set」并因此没有配 tool_choice_modes,默认继承全集。现在只发
   none / auto,和 Moonshot 同类限制的写法一致。compat 无法表达「仅思考模式下
   受限」,而思考是 DeepSeek 的默认状态,所以不做条件区分。

2. 智谱 glm-5.2:官方对 GLM-5.2 的折叠规则是 none|minimal 放弃思考、low|medium
   折成 high。models.json 却把 minimal 改写成 "low",发出去被平台折成 high,
   用户选最弱档实际跑最高强度。删掉该覆盖,回落到厂商级的 minimal。

3-4. NVIDIA NIM:z-ai/glm-5.3 与 moonshotai/kimi-k3 配了 thinking_levels,却
   沿用厂商默认的 chat-template-kwargs 编码——该分支只发 enable_thinking 布尔,
   档位被整个丢弃,而前端仍按 capabilities 渲染出一个不起作用的档位选择器。
   两个 entry 改为顶层 reasoning_effort。kimi-k3 的 NIM 卡片只写了
   "configurable low, high, or max reasoning effort" 而未展示参数,按其第一方
   API 的字段推定,已记入 Unverified。

5. 阿里云 qwen3.8:官方参数表写明「不支持 reasoning_effort 与 thinking_budget
   同时设置,同时设置会报错」,vendor.go 的注释也记了这条,但 applyThinking 在
   ThinkingBudgetTokens > 0 时无条件两个都写。新增 compat 键
   thinking_budget_excludes_effort,冲突时档位优先、预算让路。

3 与 4 在当前默认路径上已经影响到档位;1 和 5 目前够不到(全仓库只有
agent/finalize.go 发 tool_choice="none",且没有任何地方给 ThinkingBudgetTokens
赋值),属于修掉的埋雷。2 是默认路径上的实际错误。

新断言做过反向验证:数据回滚到修改前,7 条 vendor 断言与新增的 golden 用例全部
报错。
2026-09-21 00:05:11 +08:00
lyingbug c40a9dd194
refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470)
把集中式的 OpenAI 方言客户端重建为三层:协议层 internal/models/api/<protocol>(OpenAI Chat Completions / OpenAI Responses / Anthropic Messages / Google GenAI),目录层 internal/models/catalog,厂商层 internal/models/vendors/<id>(27 家,go:embed)。27 家厂商逐条对照官方文档核验,厂商事实以数据声明,协议层不出现厂商名。

既有行的运行时行为会变的四处:
1. extra_config.api 变为保留的协议选择器,非法取值被拒绝;
2. Azure 未填 api_version 的行改走 /openai/v1 GA 数据面;
3. api.openai.com 的一方流量改走 Responses 协议(中转/网关不受影响);
4. 7 家厂商的输出上限字段按文档纠正。

不需要数据库迁移。详见 PR 描述与 website-docs/03-features/06-models.md。
2026-09-20 21:28:23 +08:00
lyingbug 50bbf9ad5f
fix(embedding): two wire-level bugs in the remote embedders (#3484)
* fix(embedding): stop four embedders from returning (nil, nil) on a dead upstream

doRequestWithRetry keeps resp and err in the enclosing scope and returns
(nil, err) once the retries run out. Writing `req, err := ...` inside the
loop introduces a loop-local err, so `resp, err = client.Do(req)` updates
that copy and the enclosing err stays nil. Every retry failing therefore
returned (nil, nil): BatchEmbed's `if err != nil` did not fire and the next
line dereferenced resp.Body, a nil-pointer panic that took the process down
instead of surfacing "connection refused". Stopping the embedding upstream
and asking any RAG question reproduced it.

The bug was found, documented and fixed in openai.go, then re-fixed in
zhipu.go and again, differently, in azure_openai.go; jina.go, nvidia.go,
aliyun.go and volcengine.go kept the original spelling because the retry
loop exists in eight near-identical copies and a fix to one reaches none of
the others.

TestEmbeddersReportUnreachableUpstream now drives all eight against a
refused port with the retry budget zeroed, so the next copy of this loop is
covered the moment it appears.

* fix(embedding): read the documented index field of DashScope multimodal vectors

The multimodal embedding response carries each vector's position in `index`:

  {"output": {"embeddings": [{"index": 0, "embedding": [...], "type": "text"},
                             {"index": 1, "embedding": [...], "type": "image"}]}}

The decoder read `text_index`, which is the field name of DashScope's *text*
embedding API one page over. A missing field decodes as 0, so every vector in
a batch was written to slot 0 and the rest of the slice stayed empty: with the
default BATCH_EMBED_SIZE of 5, four of every five chunks were indexed with an
empty vector. Only models routed to this client are affected — an Aliyun
embedding model whose name contains "vision" or "multimodal"; the text-only
models go through the OpenAI-compatible path.

An index outside the batch, or a response that covers fewer inputs than were
sent, is now an error. Handing back an empty vector stores it and poisons
retrieval without anything failing.

The regression test replays the response example from the reference page, so
the decoder is pinned against the vendor's own document rather than against
our structs.

https://help.aliyun.com/zh/model-studio/multimodal-embedding-api-reference
2026-09-20 21:12:33 +08:00
wizardchen ca88fe74ae fix(embedding): preserve batching and concurrency limits for WeKnoraCloud 2026-09-19 20:26:46 +08:00
lyingbug 62f7da5eea
feat(agent): persist compaction checkpoints across turns (#3407)
* feat(agent): persist compaction checkpoints across turns

Agent compaction only lived inside the turn that ran it: the summary was
discarded at turn end and LoadAgentHistory rebuilt raw history from the
messages table, so every later turn past the threshold paid for another
summarization of nearly the same history.

A compaction whose history summary ends exactly on a stored turn is now
written onto that turn's assistant message (messages.context_checkpoint).
LoadAgentHistory starts from the newest checkpoint: the summary replaces
that turn and everything before it, and later turns are replayed verbatim.
The compactor already treats a leading summary as the previous summary, so
checkpoints fold forward incrementally.

History messages carry their turn's assistant message ID (chat.Message
TurnID, engine-internal) so compaction can tell whether its cut lands on a
stored-turn boundary. Nothing is persisted when the cut ends inside a stored
turn, covers only the live turn, or the history summary degraded to a raw
archive; a split turn's prefix summary is never persisted because that turn
is replayed whole next time. Writes are column-scoped and best-effort, and a
failed lookup falls back to history without a checkpoint.

Storing the checkpoint on the covered turn keeps the boundary implicit, so
session forks copy it with the turn and deleting the turn drops it. The
HistoryTurns cap still applies, now to the turns after the checkpoint.

* test(skill): wait for the background install in the name-conflict test

InstallSkill queues the install on a goroutine, and the fixture row already
starts at installing, so reading the status right after the call raced the
mock install and failed whenever it landed on ready first. Wait for the run
to finish on the reused row instead, as the other install tests do.

* feat(agent): size agent history by tokens instead of turn count

Agent history was capped at history_turns (default 5) because every turn
used to rebuild raw history and compaction could not persist. With
compaction checkpoints persisted, the cap is no longer what keeps a session
inside the window, and it threw away context that would have fit.

LoadAgentHistory now loads the newest checkpoint summary plus the newest
turns after it that fit a token budget: the run's compaction threshold
(agent.HistoryTokenBudget, the window less the reply reserve). History
that large already triggers the first round's compaction, which persists a
new checkpoint, so compaction rather than a turn count bounds the session.
Loading more would only hand that compaction a summarizer input larger
than the model can read.

Rows are paged backwards by (created_at, id) and the read stops at the
checkpoint or once the budget is full, so a long session is never read
whole. Kept turns stay contiguous, the newest turn is kept even when it
alone exceeds the budget (compaction splits it), and a turn cut by the read
is left out rather than replayed without its original question.

Agent mode no longer reads history_turns: AgentConfig.HistoryTurns is
removed, the editor hides the input in smart-reasoning mode and explains
the automatic behavior, and the smart-reasoning builtins drop the setting.
Quick-answer (KnowledgeQA) keeps its turn cap since it has no compaction.

* Revert "feat(agent): size agent history by tokens instead of turn count"

This reverts commit d05fb6e5f1f161813a88e9127609f7a059f38f12.

Loading history up to the compaction threshold lets the loader drop the
oldest turns before compaction can trigger: whenever a single turn's
replay is larger than the system prompt plus the new question, the
trimmed history stays under the threshold, nothing is summarized, and the
session degrades into a large sliding window that never gets a
checkpoint. A turn-by-turn simulation of LoadAgentHistory with the real
compactor showed zero compactions over 60 turns in those cases.

Keep the persisted checkpoints with the HistoryTurns cap for now; token
budgeted history returns in a separate change with a budget that leaves
the overflow to compaction.
2026-09-18 20:38:56 +08:00
lyingbug d7ccd5ba51
fix: align client sessions and integration request handling (#3305)
* fix: align client sessions and integration request handling

* fix: preserve trusted Doris redirects and prepare browser tests

* fix: align proxy address checks and integration test coverage
2026-09-16 16:20:57 +08:00
lyingbug 85b76d1a30
feat(mcp)!: persist tool catalogs and add guided configuration (#3118)
* feat(mcp)!: persist tool catalogs and add guided configuration

* fix(mcp): write caller-owned catalogs and require a real tool sync

OAuth viewers can now persist their own directory, and the setup wizard no longer treats an unsynced service as complete. Missing catalogs auto-fetch on the tools step; already-used MCP functions are restored across engine turns.

* fix(mcp): skip redundant list_servers and tighten catalog UI

When the advertised server directory already fits in discover_mcp_tools, models should list_tools or describe directly. Chat and settings now show server names, collapsed docs, and initialize debug metadata for empty optional fields.

* fix(mcp): surface catalog summaries and polish tool detail UI

List responses now include a saved catalog count without live MCP calls, so settings cards and @ mentions can show sync state. Tool details open in a popup instead of expanding the row.

* fix(mcp): drop incomplete Anthropic calls and gate shared catalog refresh

Unclosed streamed tool_use was emitted as {} and executed. list_tools refresh=true now re-lists upstream instead of returning the saved snapshot, and only Admin (or MCP-capable API keys) can overwrite tenant-wide static-auth catalogs.

* fix(mcp): wrap client refresh path to satisfy lll

The metadata refresh URL made doRequest exceed the 120-character limit in the client module CI lint.
2026-09-09 13:34:56 +08:00
lyingbug 1c448bf72c
fix(chat): send one completion-token field per provider (#3082)
* fix(chat): send one completion-token field per provider

Agent dual-wrote max_tokens and max_completion_tokens, which Volcengine Ark
rejects. Collapse them to one budget and pick the wire name the way Pi does.

Fixes #3014

* fix(chat): shrink max_tokens provider whitelist to documented hosts

Only DeepSeek, Zhipu, SiliconFlow, Moonshot, NVIDIA, and self-hosted
vLLM-class runtimes stay on the legacy field. Aliyun DashScope documents
max_completion_tokens for thinking models and must not be forced onto
max_tokens.

* fix(chat): send max_tokens for LKEAP completions

Tencent LKEAP documents max_tokens only. Keep Pi's default of
max_completion_tokens, and classify every AllProviders() host so a new
vendor cannot silently inherit the default.
2026-09-07 19:16:27 +08:00
wizardchen 22489ba663 feat(chat): keep prompt prefixes stable and send provider cache markers
Provider prefix caches only hit when system/tools stay byte-stable and
cache routing is actually on the request. Stop rewriting those prefixes
mid-turn, attach prompt_cache_key / cache_control, and log hit rate
instead of showing it in the conversation tree.
2026-09-03 00:45:52 +08:00
wizardchen b747cdb3f6 feat(agent): stream live +/- progress for sandbox write and edit
Large write/edit calls used to sit on a spinner until the full JSON
arrived. Emit path and running line counts from partial arguments so
the UI can update in place, without putting file bodies on the SSE wire.
2026-09-02 23:36:58 +08:00
wizardchen 4c0d42c4cb feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files
Refuse truncated tool calls, compact-and-retry once on overflow, and let
sandbox files be appended, paged, and batch-edited instead of rewritten.
Models can set context_window (default 200K) so compaction sizes itself
from the real window; agent max_iterations of -1 means unlimited.
2026-09-02 22:09:09 +08:00
Prodman Devokadev db05a64caa
feat: add LiteLLM as a model provider (#2923)
* feat(provider): add LiteLLM as an OpenAI-compatible provider

* docs: use revive-compliant comment for ProviderLiteLLM const

* fix(provider): stop defaulting LiteLLM to localhost under SSRF

The catalog URL localhost:4000 is blocked by default SSRF and does not
match DetectProvider, so UI/CLI could neither save nor auto-detect it.
Use a placeholder host, keep thinking aligned with generic, and sync
ModelSource across types, Swagger, and docs.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-01 12:21:03 +08:00
ochan.kwon 894cbec48a feat(agent): report and attribute per-turn LLM token usage
Accumulate token usage across every LLM round of an agent turn and
expose the aggregate everywhere a turn is observable:

- AgentState.TurnUsage accumulates each round's reported usage,
  including the final-answer synthesis call; TokenUsage.Accumulate
  keeps the cache counters as prompt subsets and recomputes the
  combined cache status (unsupported survives only while every call
  was itself unsupported).
- The agent completion event carries the aggregate, and the complete
  stream event exposes it both as data.usage and as a typed
  StreamEvent.Usage that buildStreamResponse promotes into the
  previously-unused StreamResponse.Usage wire field.
- The assistant message persists it as a nullable usage column on
  both migration targets (postgres 000085 jsonb, sqlite 000012 text),
  for web and IM channels alike, so history reads still carry the
  turn's cost after the live stream is gone.
- "[LLM Usage]" log lines gain ", session_id=..., principal=type:id"
  when the context carries them; calls outside sessions stay
  byte-identical.

Usage from failed retry attempts is not accumulated: the per-call
usage log remains the ground truth, while the turn aggregate serves
as the operational signal.
2026-08-24 10:36:19 +08:00
wizardchen ffcb10f953 feat(memory): add cross-session long-term memory
Persist extracted and explicit memories across sessions with topic
resolution, semantic recall, document affinity, consolidation, workspace
settings, and inbox/interests UI.
2026-08-13 18:00:02 +08:00
PP1 06eeb95b4e fix(embedding): validate WeKnoraCloud response indexes
Reject out-of-range and duplicate response indexes before mapping them to batch inputs. Return an error when a requested input has no response item instead of silently passing an incomplete batch downstream.

Add focused coverage for malformed responses and valid out-of-order results.

Signed-off-by: PP1 <74917296+pengpengyi92@users.noreply.github.com>
2026-08-11 16:59:48 +08:00
lyingbug 1030832c4b
fix(security): harden outbound URL SSRF protection (#2626)
* fix(security): harden outbound URL SSRF protection

* test(security): add SSRF validation tests for outbound URL caching

Introduce tests for outbound SSRF validation cache to ensure proper handling of URL validation, including cache reuse for the same origin, cache invalidation on whitelist changes, and expiration of cached entries. This enhances the robustness of SSRF protection mechanisms in the application.

* refactor: clean up whitespace in engine_factory.go and ssrf_outbound_cache.go

Removed unnecessary blank lines in engine_factory.go and adjusted variable declarations for consistency in ssrf_outbound_cache.go. This improves code readability without altering functionality.
2026-08-10 13:24:14 +08:00
Rene Zander d823ef61ae fix(vlm): use max_completion_tokens for GPT-5 / o-series vision models
The VLM path sent max_tokens and temperature, both rejected by reasoning
and GPT-5 models. go-openai's reasoning validator failed the request
client-side, so every OCR and caption call errored with "this model is
not supported MaxTokens, please use MaxCompletionTokens" and no image
chunk was ever created.

The chat path already solved this in shapeOpenAIReasoning (issue #1283);
the VLM path was never wired to it. Reuse the shared
provider.IsOpenAIReasoningOrGPT5Model predicate so both paths agree on
which models are affected. Migrating max_tokens alone is not enough --
the VLM default temperature of 0.1 is independently rejected.

Also report an empty completion with finish_reason=length as an error.
max_completion_tokens covers reasoning tokens, so an exhausted budget
returns empty content rather than an API error, which was previously
recorded as no_extracted_content and looked identical to an image with
no text.

Fixes #2537

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 08:46:17 +08:00
ljl 2b31d3681b fix(embedding): prevent ants worker panic and mutex deadlock on empty/mismatched results
If the embedding provider returns an empty slice of vectors (e.g. length 0)
due to api mismatch or empty input, BatchEmbed will return length 0.
Accessing embedding[i] directly in BatchEmbedWithPool results in an out of bounds panic.
Because the panic occurs inside a mu.Lock() block, the mutex is never unlocked,
leading to a permanent deadlock of the document processing queue.

This fix checks the returned embeddings slice length and returns an error
instead of panicking, releasing the mutex properly.
2026-08-04 16:46:57 +08:00
xiaowang 31a2358bbf fix(rerank): normalize NVIDIA logit scores 2026-08-03 15:07:15 +08:00
xiaowang fe65633f38 fix(rerank): batch LKEAP requests within API limits 2026-08-03 14:59:16 +08:00
wizardchen 5e413c6478 refactor(agent): introduce modelcontext registry and fix handle-codec bugs
Unify request-local model-handle handling behind a single
modelcontext.Registry (source citations, durable resource handles, wiki
issue handles) with per-tool argument policies and unresolved-handle
rejection before tool execution.

Also fixes issues found in review:
- decode wiki_read_issue's iN issue_id argument (was never resolved)
- preserve stored wiki page aliases when wiki_write_page omits aliases
- validate http/https scheme before registering url fields as web sources
- gate tool-role MultiContent text behind the tool result policy
- dedup KB IDs in NewWikiScopesFromKBIDs to avoid false ambiguity
- remove dead codec facades (decodeJSON, EncodeMessagesWithPolicy,
  UnresolvedToolHandles)
2026-07-27 11:05:43 +08:00
wizardchen 6fb858105a feat(llm): add prompt-cache observability and wiki cache-friendly prompts
Normalize provider-specific cache counters into TokenUsage, propagate call
purpose and prefix fingerprints through context for logging and Langfuse,
and restructure wiki page-modify prompts with warmup/singleflight to improve
prefix cache hit rates during reduce batches.
2026-07-23 19:17:30 +08:00
wizardchen 146e341748 refactor(http): share SSRF-safe transport across model clients
Extract NewSSRFSafeTransport and NewSSRFSafeHTTPClientWithTransport so
embedding and rerank can reuse one connection pool without allocating a
throwaway transport per client. SSRF dial and redirect policies are
unchanged; each client still keeps its own timeout.
2026-07-23 12:00:05 +08:00
zhangzherui ccf02e1a2a fix(embedding): reuse SSRF-safe HTTP transport 2026-07-23 11:51:54 +08:00
Martín Alcalá Rubí 83da0903b6 fix(rerank): stop sending vLLM truncate_prompt_tokens by default
The generic OpenAI-compatible reranker hardcoded `truncate_prompt_tokens:
511` on every /rerank request. Providers that honor this vLLM-specific
field (e.g. SiliconFlow serving Qwen3-Reranker) keep only the last 511
tokens of the templated rerank prompt, cutting the instruction and query
off any document longer than ~511 tokens. Every relevance score then
collapses to near zero (~0.0067 instead of ~0.998), falls below the score
threshold, and knowledge-base search returns empty results. A direct curl
without the field returns the correct scores.

WeKnora already removed this exact hardcoded parameter from the embedding
path in #960; the rerank path was never fixed.

- Add `omitempty` to `truncate_prompt_tokens` and `additional_data` so
  empty optional fields are not sent.
- Make truncation opt-in via `extra_config.truncate_prompt_tokens`
  (validated > 0 at construction), preserving an escape hatch for
  self-hosted vLLM deployments with small-context rerankers.
- Add regression tests reproducing the collapsed-score symptom and the
  opt-in path.

Fixes #2143
2026-07-21 12:12:56 +08:00
wizardchen b1b42abe23 fix(rerank): batch Volcengine rerank requests over API document limit
Split oversized candidate sets into concurrent batches instead of failing when retrieval exceeds the VikingDB rerank per-request cap, and align ko/ru locale hints with zh-CN.
2026-07-20 20:07:21 +08:00
KKtwo 3011f132e9 feat: support Volcengine rerank 2026-07-20 19:58:48 +08:00
wizardchen 6d14cc3536 feat(resource): add stable resource registry with LLM-context alias compaction
Introduce resource://<handle> as the stable application identity for stored
objects, decoupling API responses, persisted rich text, and LLM prompts from
physical storage paths. Adds a resource catalog service (register/resolve/bind,
revocable expiring access grants), a request-local alias registry that compacts
resource references while in the LLM context and restores them (including across
streaming chunks), plus router, IM, chat-pipeline, and multimodal integration.

Includes schema migrations (resources, resource_bindings, resource_access_grants)
for sqlite, and hardens URL/output security handling.
2026-07-15 17:02:43 +08:00
Thibault Jaigu 35b92f73aa feat(provider): add Requesty as an OpenAI-compatible model provider
Requesty is an OpenAI-compatible LLM gateway that uses the same provider/model
naming convention as OpenRouter, so this mirrors the existing OpenRouter provider.

- internal/models/provider/requesty.go: RequestyProvider mirroring openrouter.go
  (base URL https://router.requesty.ai/v1, REQUESTY_API_KEY, RequiresAuth)
- provider.go: ProviderRequesty const + AllProviders() entry + DetectProvider case
- internal/types/model.go + client/model.go: ModelSourceRequesty
- provider_test.go: TestRequestyProviderValidation + DetectProvider case
- frontend model editor + i18n (en/zh/ko/ru) + docs/BUILTIN_MODELS.md provider list
- docs/swagger.{json,yaml} + docs.go: requesty enum value (hand-edited to match; these
  are normally swag-generated)

Verified: go build/vet/test pass; live call to https://router.requesty.ai/v1 succeeds.
Signed-off-by: Thibault Jaigu <thibault.jaigu@gmail.com>
2026-07-13 17:03:23 +08:00
wizardchen b9cccc7bee feat(system): expose model concurrency stats on runtime queues dashboard
Surface per-model active/waiting/limit from the background task limiter so admins can distinguish task scheduling from model-service throttling, with full i18n coverage.
2026-07-11 07:17:28 +08:00
wizardchen 4404ea74e3 feat(models): implement model.max_concurrency for background tasks
Add a new `max_concurrency` setting to the model configuration, allowing for fine-tuned control over the maximum number of concurrent background tasks (ingestion/enrichment) per model. This update includes localization support across multiple languages and integrates the setting into the system's concurrency governor, enabling immediate application of changes without requiring a service restart.
2026-07-09 20:40:51 +08:00
wizardchen 20256287ec feat(models): add max concurrency setting for background tasks
Introduce a new `max_concurrency` parameter to the model configuration, allowing control over the maximum number of concurrent background tasks (ingestion/enrichment) per model. This feature is applicable to chat, embedding, and VLM models, with a value of 0 or undefined falling back to the global default. Update UI components and localization files to support this new setting, ensuring a seamless user experience.
2026-07-09 20:40:51 +08:00
wizardchen 200c344065 feat(models): cap background embedding concurrency per model
Extend the per-model background concurrency governor to embedding, the
highest-volume background model call (document ingestion vectorises every
chunk, so a batch upload can burst the whole worker pool against one provider).

The wrapper sits innermost (below debug/langfuse) because BatchEmbedWithPool
fans a batch out into per-sub-batch BatchEmbed calls via the pooler, and the
pooler invokes BatchEmbed on whichever Embedder is threaded down as `model`;
sitting innermost routes those per-sub-batch provider round-trips through Gate
so the semaphore bounds real concurrent calls rather than one coarse per-doc
unit. Only background (asynq) calls are throttled; interactive query embedding
is never gated.
2026-07-09 20:40:51 +08:00
wizardchen 98ad41cda4 fix(models): harden background concurrency governor
Follow-up hardening on the per-model background concurrency governor:

- limiter: heartbeat now refreshes the ZSET key's PEXPIRE alongside the
  lease score, so a semaphore that stays saturated with no slot turnover
  can't let the whole key expire (which would drop every lease and admit
  over the limit); reuse one poll timer instead of allocating time.After
  each iteration under contention.
- governor: extract the process-wide governor + Gate helper into the
  limiter package so chat and vlm share one limiter/limit without
  importing each other; add an in-process localLimiter for Lite mode.
- vlm: gate background multimodal (VLM) calls through the same governor;
  previously only chat was throttled.
- chat: ChatStream releases its slot when the consumer abandons the
  stream and cancels ctx (select on ctx.Done, drain inner), fixing a
  slot/goroutine leak.
- container/sync_task: install the local governor in Lite mode and tag
  SyncTaskExecutor runs as background; honour model.max_concurrency=0 as
  an explicit disable instead of falling back to the default.
- tests: cover the governor gating, local limiter, and stream release.
2026-07-09 20:40:51 +08:00
wizardchen 63198c5d9f feat(models): cap background LLM concurrency per model
Background ingestion/enrichment stages (summary, question, graph,
multimodal) all target the same chat model, and asynq queue weights are
scheduling priority rather than throttling. A large upload could therefore
burst up to the entire worker pool of concurrent chat calls against a
single provider, unbounded across replicas, with no model-side rate limit.

Govern concurrency at the chat-client layer (the one place that sees every
task type), keyed by model ID:

- internal/models/limiter: Redis ZSET + Lua distributed semaphore with a
  lease heartbeat, expired-lease reclaim for crashed holders, and fail-open
  on any backend error or context cancellation.
- chat: concurrencyChat decorator (outermost in NewChat) acquires a
  per-model slot only for background tasks; interactive chat is never gated.
- router/task: asynq middleware tags every worker execution as background.
- container: wire the Redis limiter with WEKNORA_MODEL_MAX_CONCURRENCY
  (model.max_concurrency), default 8; enabled only when Redis is present.

Tested with miniredis: concurrency cap, expired-lease reclaim, per-key
isolation, and all fail-open paths (nil client, backend down, cancelled ctx).
2026-07-09 20:40:51 +08:00
wizardchen 1661827d6e fix(security): harden model client redirects 2026-07-02 19:15:46 +08:00
wizardchen 663d13eb58 fix(security): harden outbound requests and file previews 2026-07-02 19:15:46 +08:00
lawrence fdfc7297de fix(vlm): allow configurable temperature for VLM models
Some VLM models (e.g. certain vision models) only accept temperature=1.
Previously temperature was hardcoded to 0.1. Now reads from model's
ExtraConfig["temperature"] with fallback to default 0.1.

Usage: set ExtraConfig.temperature=1 in the VLM model configuration.
2026-07-01 12:20:09 +08:00
wizardchen 6b60d292f8 fix(embedding): add SSRF validation and safe HTTP client for embedders
Align embedding outbound requests with chat module defenses by validating
base URLs at construction time and using SSRFSafeDialContext with redirect checks.
2026-06-26 22:41:04 +08:00
wizardchen 2c44a41f2a feat(models): add model test debugger for saved model configs
Let admins send real requests against configured chat, embedding, rerank,
VLM, and ASR models from settings, with thinking controls shown only when
the model supports them.
2026-06-25 17:05:33 +08:00
wizardchen de525a4f02 feat(observability): enhance Langfuse retrieve and rerank tracing
Add shared retrieval observation helpers, expand rerank generation output
with previews and score stats, and enrich retrieve/rerank pipeline spans.
2026-06-16 01:09:23 +08:00
wizardchen 53e0333a53 Add embedding dimension override support 2026-06-15 19:18:19 +08:00
MaoMeng cf0a64b88e
fix:dimensions not sent in embedding API requests (#1654)
fix:dimensions not sent in embedding API requests
2026-06-15 17:32:23 +08:00
Yeongpil Yoon c6fc315042 fix(chat): preserve provider tool-call metadata 2026-06-09 20:55:20 +08:00
wizardchen 28de82a92e refactor(chat): improve model name sanitization and path validation
- Updated the sanitization logic in `newStreamPacketDumper` to exclude the '.' character, enhancing security against path traversal vulnerabilities.
- Added checks to ensure the cleaned file path remains within the specified directory, preventing unauthorized access to the filesystem.
- Improved overall robustness of file handling by enforcing stricter path validation rules.
2026-06-05 19:35:02 +08:00