Commit Graph

11 Commits (de2dfd3034a7b506fa269910401bbdcbd4efc71d)

Author SHA1 Message Date
LL_Ricky 9842e51d40
fix(frontend): 修复复制模型时模型名称被改写的问题 (#3830)
* fix(frontend): 修复复制模型时模型名称被改写的问题

* fix(model): preserve model name and credentials when copying a config

Copying rewrote the upstream model name and posted redacted parameters, so the duplicate could not authenticate. Clone the stored row, cap the display name, and skip a new download when the local model is already active.

* fix(lint): clear new golangci-lint findings on model copy

Check c.Error results, wrap the long test line, and format the stub. The pull request lint range also includes the new HTTP error test, whose unused request parameter failed revive.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-29 19:32:08 +08:00
Victor Maznyuk cf079b8dcd fix(models): keep Retry-After on HTTPError and honour it in agent LLM retries
Endpoint.Do turns a non-2xx reply into &HTTPError{StatusCode, Body} and
drops the response headers. A caller that retries a 429 or 503 therefore
cannot see the provider's Retry-After and falls back to its own guess: the
agent retry loop waits 1s, 2s, 3s, which against a provider asking for, say,
20s spends every attempt inside the rate-limit window and then fails the
turn.

- HTTPError carries the reply's Header, and RetryAfter() parses the header
  in both forms (delay-seconds and HTTP-date); absent, malformed or past
  values give zero.
- The agent retry loop waits max(linear backoff, Retry-After), with the
  vendor's value capped at 30s so a large Retry-After cannot hold a turn
  indefinitely.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 01:22:15 +08:00
DerrickLinus a9d71038b6
fix(models): stop advertising graded thinking levels the wire format drops (#3552)
* fix(models): stop advertising graded thinking levels the wire format drops

A generic-provider row (relays, self-hosted runtimes — the default for
uncatalogued models) resolves an empty ThinkingLevelMap, whose missing-key
semantics report every graded rung as supported. Capabilities() then hands
minimal/low/medium/high to the frontend, which renders a selector whose
choice applyThinking silently discards: the chat-template-kwargs branch
sends only the enable_thinking boolean and no reasoning_effort is computed.
Same defect class as the two NVIDIA NIM entries fixed in #3489, but there
is no catalog entry to repair here.

Collapse Capabilities() to the levels the wire can actually express for a
boolean-only switch (off/auto) unless the settings opt into
supports_reasoning_effort. The vendor map itself is untouched, so vendors
that do carry effort keep their ladders.

* fix(models): carry reasoning_effort through the chat-template switch when opted in

The Capabilities() guard kept a vendor's ladder when it opts into
supports_reasoning_effort, but applyThinking's chat-template-kwargs branch
wrote only the enable_thinking boolean and dropped the already-computed
effort — so the opt-in combination still sent byte-identical bodies for
every graded rung (#3551's defect, alive in exactly the shape the review
flagged). Mirror the enable_thinking dialect: write reasoning_effort on
top of the switch when one was computed. A golden case pins the opt-in
combination; the no-opt-in guard case keeps asserting the level is
dropped.

* style(models): wrap long Resolve call in thinking-levels test

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
2026-09-25 22:23:08 +08:00
wizardchen 8356742dd5 fix(llm): treat cut-off streams as broken and tighten LLM retries
A stream body that ran out without a finish reason (a proxy cutting the
connection) was closed as a clean stop on the Completions, Responses and
Gemini paths, so a half-written answer was stored as complete. It is now
marked incomplete, as the Anthropic path already did, and both the chat
pipeline and the agent treat an incomplete answer as a stream error.

Retry classification checks typed HTTP/transport errors first and matches
status codes as whole numbers, so "max_tokens 5000" no longer reads as a
500. The retry backoff returns as soon as the turn is cancelled.
2026-09-25 18:22:14 +08:00
lyingbug 04adc13272
fix(retrieval): fix recall correctness, score scales, cost and agent tools (#3687)
* fix(retrieval): keep index writes and shared-KB lookups correct

Index writes
- Image OCR/caption chunks were indexed without IsEnabled, so every engine
  but Postgres stored them disabled and retrieval never returned them.
  The SQLite engine re-enables the affected rows at startup.
- FAQ edits, similar-question additions and import merges re-index under
  the old SourceID and relied on the engine overwriting it. Postgres and
  SQLite kept the stale row (ON CONFLICT DO NOTHING); ES v8 and Qdrant
  mint a new document ID and kept both. FAQ re-indexing now deletes the
  old rows first, and Postgres/SQLite BatchSave upserts by
  (source_id, source_type).
- Postgres: drop the GORM default tag on is_enabled, which replaced false
  with true on insert and indexed disabled chunks as enabled.
- SQLite: BatchSave no longer attaches vectors and FTS rows to the wrong
  rows when part of a batch already exists, and vecTables is guarded
  against concurrent map writes.
- CopyIndices: Postgres filters to the copied knowledge, pages by id and
  keeps is_enabled; SQLite copies every row of a chunk (generated
  questions included) with stable source IDs and enabled state.

Shared knowledge bases
- Chat merge: FAQ answer population and neighbor expansion look chunks up
  without the caller-tenant filter, like parent resolution (#3342).
- Neighbor expansion never cuts the retrieval hit; the length budget goes
  to the text adjacent to it.
- FAQ negative questions are applied whenever an FAQ KB is in scope and
  to chunks of shared FAQ KBs.
- Search results get image_info for shared-KB chunks.
- search_knowledge reads FAQ metadata from the search result instead of
  one tenant-scoped query per hit, so shared FAQs render their answers.

* fix(retrieval): put every retrieval score on one scale

Scores from different retrieval paths were ranked together while living on
different scales, so ordering depended on which path produced a hit.

Fusion
- RRF ranked a chunk by its position in the concatenation of every
  vector (or keyword) list, appended in goroutine completion order. The
  second list's best hit ranked behind every hit of the first, and the
  order changed between identical requests. Each list is now ranked on
  its own by score and a chunk takes its best rank.
- RRF is divided by its maximum, so hybrid scores land in [0, 1] like
  vector-only (cosine) and keyword-only (rescaled BM25) results. Raw RRF
  topped out near 0.016 and lost to any single-retriever hit it was ranked
  against across searches (document targets, model groups, FAQ calls);
  rerank's base-score term and MMR's relevance term had nothing to use.
- Keyword-only BM25 is rescaled per list; engines' BM25 is not comparable.
- FAQ iterative retrieval fuses each round like the first pass instead of
  sorting raw BM25 and cosine together, starts at twice the depth already
  searched, triggers when any single vector list is full (the old summed
  count never matched with more than one list) and replaces results per
  round.

Rerank and chat pipeline
- The FAQ boost is no longer capped at 1; capping made strong FAQs tie at
  1.0 and fall to the tie-breaker. The composite score before boosts is
  kept as composite_score.
- The FAQ direct-answer threshold is compared against that pre-boost
  score, and the FAQ that cleared it is the one marked match="exact"
  rather than the first FAQ.
- Graph hits no longer carry a fabricated retrieval score of 1.0, which
  ranked them above every scored hit when rerank did not run; the
  composite score uses the model score in its place.

Engines
- OpenSearch and Weaviate return (1 + cos) / 2; their drivers now report
  cosine similarity and map thresholds onto the engine scale, so a
  threshold means the same on every engine.
- Milvus L2 returned squared distance, ranked farthest-first downstream,
  and took the threshold as a radius; scores and radius are converted.
- The multi-store normalizer no longer maps Milvus through (s + 1) / 2,
  which put it on a different scale from every other engine and from its
  own single-store results.

Docs updated for fusion, normalization, FAQ boost and iterative recall.

* perf(retrieval): bound rerank, agent and pipeline cost; stop failing on slow or missing models

Rerank
- Rerankers report the longest document a request can carry
  (max_document_chars / max_request_chars minus the query) and the rerank
  stage fits its own passages to it, trimming appended captions, OCR and
  generated questions first. One chunk with a large screenshot's OCR used
  to exceed the vendor limit and fail the whole batch, so every candidate
  lost its score and the threshold stopped applying.
- Rerank HTTP requests default to a 60 s timeout; chat and agent contexts
  carry no deadline, so a stalled self-hosted rerank server held the turn
  open.
- OCR and caption chunks no longer repeat their own text from image_info in
  the passage.
- The chat pipeline and the agent tool rerank at most 200 candidates, best
  retrieval score first.
- A rerank model that cannot be loaded degrades to retrieval order instead
  of failing every chat turn, as the search APIs already do; without rerank
  results the merge stage keeps only RerankTopK hits instead of enriching
  every one.

Retrieval APIs
- knowledge-search match_count and rerank.top_k are capped at 200
  (unbounded values multiplied into tens of thousands of billed rerank
  passages); normalizedMatchCount clamps to the pool size so the
  over-retrieval arithmetic cannot overflow.
- hybrid-search with rerank recalls at least rerank.top_k deep.

Agent
- search_knowledge fits its rendered output to four fifths of the tool
  output budget, dropping the lowest-ranked rows and telling the model how
  many it omitted; the model's view is rebuilt from Data, which registry
  truncation never reached.
- Agent searches skip neighbor/parent/related enrichment rows, which were
  never retrieved yet tripled the rerank input.
- ParallelToolCalls is enabled. The model was always told it may call tools
  in parallel while the engine ran them one by one; only read-only tools
  overlap, anything else stays a barrier.

Chat pipeline
- Query expansion searches keywords only (its variants are local rewrites
  whose embeddings repeat the main search) and is skipped when keyword
  matching is off.
- Entity extraction, a second LLM call, is skipped for intents that need no
  KB retrieval and uses the rewritten query.
- Partial-overlap dedup tokenizes each text once instead of per pair, and
  only compares token overlap between texts within 3x of each other's size.

Storage
- Migration 000112 adds btree indexes on embeddings(knowledge_id) and
  (chunk_id); document deletes, chunk re-index and enable/tag updates
  scanned the whole table.
- Shared-KB knowledge rows are fetched in one query instead of one per ID.
- Drop GORM Debug() from the ParadeDB keyword query and the knowledge batch
  lookup, which logged SQL and user queries on every search.

Docs updated for the API limits, rerank timeout and passage fitting, agent
parallel tools and output budget, and pipeline changes.

* fix(retrieval): make agent tools and pipeline stages report what they did

Agent tools
- query_knowledge_graph now queries the graph store: entity names from the
  query are matched in each KB's graph, relations are returned (and shown to
  the model), and the entities' source chunks lead the results, with text
  search as a supplement. It used to run text search only, so a graph-only
  KB always answered "no relevant graph information".
- search_knowledge reports failures instead of "no matching chunks": when
  every search fails the tool fails with the reason; partial failures are
  listed. A failed query embedding falls back to keyword search for hybrid
  KBs, as the chat pipeline does.
- Keyword mode uses rerank to order hits, not to reject them (identifiers
  score low), and the empty-result hint no longer sends keyword mode back to
  keyword mode.
- Dedup keeps siblings under one parent, no longer merges every image chunk
  of a document through chunk_index 0, and keeps the better-scored copy.
- Search rows omit a <match> excerpt that only repeats the content; snippet
  terms are segmented for Chinese queries and skip stopwords, and FAQ
  matched questions are chosen by match strength.
- read_document query mode returns next_offset when truncated and resumes
  from offset.
- wiki_search over-fetches under document/tag scopes before filtering and
  says when every match was out of scope.
- Prompts point to read_document(id=cN, context=k) for surrounding text.
- The agent editor hides the retrieval-strategy section in agent mode,
  where search_knowledge uses the global retrieval settings.

Chat pipeline
- Intent labels are normalized; an unknown label retrieves instead of
  silently skipping the knowledge base, and web_search without web search
  enabled searches the knowledge bases.
- Image turns get a 2048-token rewrite budget, and a reply cut off by the
  cap is salvaged field by field instead of being discarded.
- History references are matched by how much of the query they cover;
  Jaccard against chunk-sized text never reached its threshold.
- Entity search resolves chunks of org-shared KBs within the turn's KB
  scope, skips chunks whose document is gone (it used to panic) and caps
  graph chunks at 30.
- Empty history is not fetched twice; fetched pages are cut on a rune
  boundary.

Retrieval
- A multi-KB search whose primary KB has no vectors embeds the query with a
  vector KB's model instead of failing on an empty model ID.
- Rerank passages include the chunk's heading breadcrumb, which is part of
  what was embedded.
- SQLite FTS searches the content column only and turns a lone Han
  character into a prefix query; TencentVectorDB ef follows TopK.

Docs updated for the agent tools and pipeline stages.

* fix(retrieval): address review findings on the retrieval fixes

- Restore NeedsRetrieval for web_search without web search enabled: the
  intent prompt "web_search" is how the user learns web search is off, and
  running KB retrieval instead bypassed it.
- query_knowledge_graph keeps the case of entity terms; Neo4j matches node
  names with case-sensitive CONTAINS, so lowercased terms never matched
  English entities.
- An empty search_knowledge or query_knowledge_graph result states partial
  failures in its text, which is all the model sees for an empty result.
- SQLite BatchSave replaces a source in one transaction, so a failed insert
  keeps the old rows instead of leaving the source unindexed.
- Reuse stringSliceValue and an exported searchutil.ContainsChinese instead
  of new copies; drop an unused named return and the FAQ obsolete-source
  delete that the full per-chunk delete now covers.

* fix(retrieval): address second review of the retrieval fixes

- query_knowledge_graph: under a document/tag scope, keep only relations
  whose entities come from an in-scope evidence chunk (the graph namespace
  is the whole KB); drop chunks whose document no longer exists; report a
  graph or text search failure even when the other source returned hits.
- search_knowledge: record the hybrid -> keyword fallback taken when the
  query embedding fails in mode_fallbacks, so an empty result is not read
  as "no semantic match".
- query understanding salvage: take a cut-off value only for the image
  description; a truncated rewrite_query or intent is dropped instead of
  replacing the user's query.
- Graph hits (no retrieval score) are kept past the no-rerank RerankTopK
  cut and the rerank candidate cap instead of always falling below them.
- FAQ boost is capped at 1 again so scores stay on the [0, 1] scale; FAQs
  tied at the cap are ordered by their pre-boost score.
- Negative-question filtering loads only candidates of FAQ knowledge bases.
- read_document query continuation starts one chunk before offset so the
  first match keeps its preceding context at a page boundary.
- graphSearchTerms and searchQueryTokens share one queryTerms splitter.
2026-09-25 10:48:18 +08:00
lyingbug e179f44445
refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589)
* refactor(models): separate providers, catalog and runtime configuration

* refactor(models): isolate provider definitions and runtime catalogs

* docs: exclude refactoring analysis notes from the pull request
2026-09-23 14:20:08 +08:00
lyingbug 4a9aa06274
refactor(asr): move the wire format into protocol packages, and cover every vendor whose docs offer ASR (#3503) 2026-09-21 13:25:27 +08:00
lyingbug 9030f556f2
refactor(embedding): move the wire formats into protocol packages, and correct the vendor facts (#3500) 2026-09-21 11:55:38 +08:00
lyingbug d6c7665cb5
refactor(rerank): move the wire formats into protocol packages, and correct the vendor facts (#3488)
把 8 个手写 rerank client 收敛成 3 个协议包(internal/models/api/{cohererank,dashscoperank,nimrerank}),每个带一份从厂商文档逐字粘来的 golden fixture。LKEAP 和火山保留官方 SDK,但只实现 api.Reranker。rerank.NewReranker 改成 catalog.Resolve + 按协议分发,函数体里没有厂商名。生产代码净 -147 行。

三件原来每个 client 各管各、现在由共用层统一的事:分批(按厂商声明的上限,智谱 128 / NIM 512 等此前无人执行)、分数标度(NIM 与 GPUStack 返回 logit,其余为 0~1 概率,统一换算后再与 RerankThreshold 比较)、NIM 的 truncate(默认 NONE 会让超长 passage 整请求失败,此前从未发送)。

对照官方文档订正四处厂商事实:GPUStack 分数标度、千帆三项上限、阿里 500 条上限、qwen3-rerank 是第二种协议(隐藏并在构造期拒绝)。Novita / OpenRouter / LiteLLM 三家官方有 rerank 但未开类型,本次开放,零协议代码。

运行时行为变更:NVIDIA 起发 truncate: END;GPUStack 分数按 logit 换算(可按行覆盖);truncate_prompt_tokens 收窄为厂商声明制;qwen3-rerank 存量行从静默打错变为构造期报错。详见 PR 描述。

关联 #3486。
2026-09-21 10:36:26 +08:00
lyingbug 9aa63c866f
fix(models): correct four vendor request shapes that contradict the docs (#3489)
PR #3470 逐条核对了 27 家厂商,但有四处(五个 entry)的声明与厂商文档对不上,
其中一处在默认路径上就是错的。

1. DeepSeek:参考文档写明 "required and named tool choices are not supported
   in thinking mode; the API returns a 400 error",而 vendor.go 的事实注释写成
   「plain OpenAI set」并因此没有配 tool_choice_modes,默认继承全集。现在只发
   none / auto,和 Moonshot 同类限制的写法一致。compat 无法表达「仅思考模式下
   受限」,而思考是 DeepSeek 的默认状态,所以不做条件区分。

2. 智谱 glm-5.2:官方对 GLM-5.2 的折叠规则是 none|minimal 放弃思考、low|medium
   折成 high。models.json 却把 minimal 改写成 "low",发出去被平台折成 high,
   用户选最弱档实际跑最高强度。删掉该覆盖,回落到厂商级的 minimal。

3-4. NVIDIA NIM:z-ai/glm-5.3 与 moonshotai/kimi-k3 配了 thinking_levels,却
   沿用厂商默认的 chat-template-kwargs 编码——该分支只发 enable_thinking 布尔,
   档位被整个丢弃,而前端仍按 capabilities 渲染出一个不起作用的档位选择器。
   两个 entry 改为顶层 reasoning_effort。kimi-k3 的 NIM 卡片只写了
   "configurable low, high, or max reasoning effort" 而未展示参数,按其第一方
   API 的字段推定,已记入 Unverified。

5. 阿里云 qwen3.8:官方参数表写明「不支持 reasoning_effort 与 thinking_budget
   同时设置,同时设置会报错」,vendor.go 的注释也记了这条,但 applyThinking 在
   ThinkingBudgetTokens > 0 时无条件两个都写。新增 compat 键
   thinking_budget_excludes_effort,冲突时档位优先、预算让路。

3 与 4 在当前默认路径上已经影响到档位;1 和 5 目前够不到(全仓库只有
agent/finalize.go 发 tool_choice="none",且没有任何地方给 ThinkingBudgetTokens
赋值),属于修掉的埋雷。2 是默认路径上的实际错误。

新断言做过反向验证:数据回滚到修改前,7 条 vendor 断言与新增的 golden 用例全部
报错。
2026-09-21 00:05:11 +08:00
lyingbug c40a9dd194
refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470)
把集中式的 OpenAI 方言客户端重建为三层:协议层 internal/models/api/<protocol>(OpenAI Chat Completions / OpenAI Responses / Anthropic Messages / Google GenAI),目录层 internal/models/catalog,厂商层 internal/models/vendors/<id>(27 家,go:embed)。27 家厂商逐条对照官方文档核验,厂商事实以数据声明,协议层不出现厂商名。

既有行的运行时行为会变的四处:
1. extra_config.api 变为保留的协议选择器,非法取值被拒绝;
2. Azure 未填 api_version 的行改走 /openai/v1 GA 数据面;
3. api.openai.com 的一方流量改走 Responses 协议(中转/网关不受影响);
4. 7 家厂商的输出上限字段按文档纠正。

不需要数据库迁移。详见 PR 描述与 website-docs/03-features/06-models.md。
2026-09-20 21:28:23 +08:00