Commit Graph

721 Commits (c40b33ecc34f5a2c33bc47d33e33c577e2ae1c2d)

Author SHA1 Message Date
lyingbug 687b178706
feat(agent): add the built-in WeKnora assistant with an operation catalog (#4302)
* feat(agent): add the built-in WeKnora assistant with an operation catalog

Adds builtin-weknora-assistant, which inspects and manages WeKnora with the
signed-in user's permissions:

- find_operations / run_operation over an allow-listed API operation catalog,
  dispatched in-process through the real router as the user
- builtin approval for write/destructive operations, refused when
  non-interactive
- agent attribution in knowledge base activity, including async tasks
- page_context request field, narrowed to the turn's search targets and
  rendered into runtime_context with model handles

* refactor(agent): compile the operation catalog from x-agent Swagger annotations

The catalog was a hand-written copy of what the swag annotations already
describe: routes, parameters, enums and response shapes. It now compiles
from the Swagger document embedded in the binary. Handlers opt in with

  // @x-agent      {"id":"knowledge.delete","risk":"destructive",...}
  // @x-agent-hint "Cannot be undone."
  // @Param        id path string true "知识ID" extensions(x-agent=knowledge_id)

and response types mark the fields the model may see with an
extensions:"x-agent" struct tag. Exposure stays opt-in at every level:
unmarked operations, parameters and fields are not reachable, so a new
field on Knowledge or a new query parameter never reaches the model by
accident. NewCatalog still enforces the safety rules (forbidden areas,
risk versus method, approval targets), and a test fails when the
annotations and docs/swagger.json disagree.

The twelve operations compile to the same routes, parameters and
results as before; document results now share one field set, and the
parse_status filter accepts deleting, which the API supports.

* fix(agent): enforce resume and wiki context scopes
2026-10-15 12:18:30 +08:00
lyingbug 234aa948db
docs(swagger): document the real request and response types (#4304)
* docs(swagger): add shared response envelope types

* docs(swagger): type knowledge base responses

* docs(swagger): add concrete response body types

* docs(swagger): type chunk list responses

* docs(swagger): type knowledge mutation responses

* docs(swagger): type wiki search and autofix responses

* docs(swagger): type FAQ responses

* docs(swagger): type tag responses

* docs(swagger): type message responses and SSE schemas

* docs(swagger): type agent responses

* docs(swagger): type remaining handler responses

* docs(swagger): regenerate OpenAPI spec

* docs(swagger): type knowledge, activity and wiki maintenance endpoints

Covers the endpoints left out of the earlier passes: knowledge base
detail, document list/detail/spans/reparse/cancel/delete, knowledge
base activity and the Wiki stats, issues, lint and rebuild-links
endpoints. The spans handler now writes the documented structs instead
of gin.H, so the schema cannot drift from the body; the JSON is
unchanged. List filters gain their real enums and bounds, including
the finalizing, cancelled and deleting parse states.

* docs(swagger): document JSON-valued fields as JSON, not byte arrays

types.JSON and json.RawMessage are byte slices, so swag rendered every
metadata, details and config field as an array of integers. A swag
overrides file documents them as objects; the four fields that hold
arrays (relation chunk ids, allowed origins, image refs) say so with a
swaggertype tag.

* chore(lint): exempt swag annotation lines from the line-length limit

swag reads each @Param, @Success and @Failure as a single line, so an
enum list or a composed response type cannot be wrapped to fit.

* chore(lint): exempt struct tags carrying swag attributes from the line limit

Struct tags cannot wrap either; a field documented with swaggertype or
extensions keeps its tags on the declaration line.
2026-10-15 01:41:42 +08:00
lyingbug 383d26535a
feat(agent): persist human-in-the-loop prompts with a shared broker (#4250)
* feat(agent): persist human-in-the-loop prompts with a shared broker

Tool approvals and MCP OAuth prompts are now rows in agent_prompts,
awaited through hitl.Broker (lease, DB polling, Redis wake-up, per-kind
answer validation). Resolve is bound to session and kind. This is P1 of
the suspend/resume design; runs still block as before.

* feat(agent): suspend runs whose prompt outlives a short in-memory wait

Behind agent.prompt_suspend_enabled (off by default). A run raised from the
HTTP stream saves its state when a tool approval is raised; after the hot
wait the prompt moves to suspended and the assistant message is saved as
awaiting_input instead of blocking. A reaper takes over prompts whose
waiting replica stopped renewing its lease. Resume comes in a later change.

* feat(agent): resume suspended runs once their prompt is answered

The replica that records the answer resumes the run in process under the
prompt's lease. Superseded, expired and abandoned runs are closed with a
result for every tool call, and approved tools are never replayed.

* feat(agent): let users answer suspended tool approvals and follow the resumed run

* feat(agent): ask_user tool and generic prompt answer endpoint

- ask_user built-in tool: options, short forms or free text; validated
  answers, decline, suspend/resume like tool approval; non-interactive
  callers (IM, API keys) are told to ask in the reply instead
- POST /agent/prompts/{prompt_id} (and embed variant) answers ask_user
  and tool_approval prompts, returns resumed; SDK ResolveAgentPrompt
- agent_prompt_required/resolved events; AskUserCard sharing the
  countdown with ToolApprovalCard; answered questions rebuilt as cards
  from history
- awaiting-input messages keep their steps expanded so the pending card
  stays visible; prompt cards keyed by their own id
- i18n for all locales; design doc updated

* fix(agent): keep prompt answers that race a suspending or vanished owner

- Resolve suspends a pending prompt whose owner's lease lapsed, as the
  reaper would, so the answer resumes the run instead of waking nobody.
- Resuming waits for the suspending run to persist its message and clear
  its live-run marker; a marker left by a vanished owner is cleared.
- A resumed run that suspends again on a new prompt no longer has its
  message closed from the old state.
- tool_approval answers are decoded strictly and require "approved";
  fix the SDK doc that showed {"decision": "approve"}.
- API-key runs block instead of suspending, since they cannot answer.
- A run state that cannot be saved falls back to an in-memory wait, and
  NUL escapes are stripped from snapshots for jsonb.

* style(chat): align the ask_user card with the other interaction cards

Question and answer use the timeline's step text styles instead of
primary-colored and monospace text; options and form controls use the
compact step size, the number input spans the row, and the free-text
character counter is hidden.

* feat(chat): dock pending agent prompts above the composer

Tool approvals, ask_user questions and MCP OAuth prompts were answered
inline in the agent timeline, where a long answer or scrolling back
pushes them out of view. Open prompts of a streaming or suspended run
now gather in a dock above the composer (main chat and embed), one at a
time with a pager, lettered options and Enter to submit. The timeline
keeps a one-line record with a link back to the dock and the outcome
once answered. Views without a dock still answer inline.

* fix(agent): fence prompt ownership and preserve resumed turn context

* fix(agent): keep runs through consume errors and file prompts under the caller

Consume reported a store error as a lost claim, so an answer consumed
during a database hiccup detached its run: the message was never
persisted, the live-run marker stayed, and the reaper skipped the row
because the waiter had already dropped the snapshot. Consume now retries,
then decides from the row, and gives a run up only when the prompt is
known to have moved on.

Tool approvals and OAuth prompts of a shared agent were filed under the
agent's workspace, so the user's answer failed the tenant check and the
prompt timed out. Like ask_user, they now use the caller's workspace;
policy and token lookups stay in the agent's.

Enter on a picked option submits it instead of toggling it off, and
resume errors no longer report a nil cause.

* fix(agent): validate prompt choices and require durable approval claims

Reject malformed serialized option lists so the model can retry with valid arguments. Distinguish prompt consumption failures from lost ownership without authorizing side effects, and preserve native Enter selection for unselected choices.

* fix(agent): resolve prompt workflow lint errors
2026-10-14 20:37:28 +08:00
lyingbug 3e22822c1d
chore(release): finalize v0.8.3 website and documentation (#4285)
* chore(release): prepare v0.8.3 website and documentation

* docs(website): broaden release highlights and simplify browser layout

* docs(website): keep feature descriptions and screenshots in view

* docs(website): show four expanded release highlights

* docs(website): show browser states beside screenshots

* docs(website): unify section backgrounds and spacing

* fix(website): sync product screenshots and refine hero content

* fix(website): add gallery detail view and refine feature headings

* docs: illustrate mention resource selection in v0.8.3 release notes

* docs(release): finalize v0.8.3 release date and publication wording
2026-10-14 15:20:30 +08:00
Ochan Kwon 31c6055687
feat(agent): name the turn's chat model on the completion event (#4252)
The agent completion event carries the turn's aggregated token usage
but not the chat model that produced it. A relay or gateway that meters
agent chat cannot price that usage: unit prices, and the discounts for
cache reads, cache writes and uncached input, differ per model, and
changing an agent's model blends the old and the new numbers in one
total.

The completion event's data now carries model_id, taken from the
engine's chat model. The turn's usage is aggregated only from calls
made through that model (each round and the final answer), so it names
the model the usage belongs to. A consumer resolves the model's name
and settings through the model API with its own credentials. The
message row's model_id keeps the requested binding and can differ when
a request override is invalid and the turn falls back to the agent's
model; the event names the model that actually ran. The key is omitted
when no model is known. The quick-answer path is not changed.

The model name is left out on purpose. The completion event reaches
everyone who chats with the agent, including shared-agent users and
embed visitors, and an operator may not want to reveal which model
serves an agent; the provider-facing string can also be a deployment
override that the model API hides from non-admin callers. The message
history already returns the same ID to these users.
2026-10-13 23:07:32 +08:00
Ochan Kwon e0712df3e0
feat(mcp): let callers without a browser obtain MCP OAuth tokens (#4229)
WeKnora runs behind other applications: they sign their users in and
call WeKnora with a tenant API key and X-External-User-ID, and WeKnora
keeps sessions and MCP OAuth tokens per external user. When an agent
uses an OAuth-protected MCP service, the only way to get that user's
token is the browser authorization code flow, and in this deployment
the user never sees WeKnora, so there is no browser to complete it.
Independently, an OAuth MCP service can only get its client through
dynamic client registration (RFC 7591), so authorization servers
without a registration endpoint cannot be used at all ("server does not
support dynamic client registration"), and servers that allow a grant
only to clients an administrator registered refuse a dynamic one.

JWT bearer grant. The application in front already knows who the user
is. POST /api/v1/mcp-services/:id/oauth/assertion lets it present a
signed assertion of that identity, such as the Identity Assertion JWT
Authorization Grant (ID-JAG) used by MCP's Enterprise-Managed
Authorization extension. WeKnora sends it to the service's
authorization server with the JWT bearer grant (RFC 7523) and stores
the token exactly like the callback does, so status, revoke, runtime
connections and, when the response carries a refresh token, refresh
use it unchanged. The assertion is forwarded unread; validating it is
up to the authorization server. The request reuses the principal from
authorize-url, the SSRF-safe HTTP client and the callback's connection
recycling, and the client the service already has. It does not run
dynamic client registration, which describes a redirect-based client;
without a client it answers 400. The token request outlives the caller,
so a token issued for an assertion that can only be used once is not
lost. A 4xx response other than 408 and 429, or a 2xx response carrying
an OAuth error, answers 400 with the OAuth error code and is logged; a
5xx, 408 or 429 response or a network failure answers 500. Request
logging masks the assertion field.

Pre-registered client. auth_config.client_id names a client registered
with the authorization server in advance, and client_secret is its
secret for a confidential client. When client_id is set it takes the
place of dynamic registration everywhere a client is read: the
authorization URL, the code exchange, the assertion grant and runtime
connections with their token refreshes. The values live on the service
rather than in mcp_oauth_clients, because a refresh rejected with
invalid_client deletes that row so the next authorization registers
again; an administrator's client would silently disappear. client_id is
non-secret: it is set through the main PUT, where an empty value clears
it, and returned in auth_config. client_secret follows api_key and
token: encrypted at rest, never returned, accepted in the create body,
and changed afterwards through the credentials subresource. Changing
client_id rebuilds the service's connections, and a connection built
with a pre-registered client is not treated as holding a replaced
registration when an authorization recycles connections. The MCP
service dialog adds optional Client ID and Client Secret fields to the
OAuth section, in all six locales.

Nothing changes for existing services: with client_id empty every
existing path reads the stored registration and registers dynamically
as before, and the authorization code flow, its state, the callback and
the dynamic registration request are untouched. The authorization
server HTTP client is moved into a function that both paths call.
2026-10-13 16:37:24 +08:00
lyingbug 194bffbb0a
feat(agent): read chat attachments on demand (#4203)
* feat(agent): read chat attachments on demand

Agent turns pasted query-selected attachment excerpts into the prompt before the model could act. List stored files as a manifest so the agent can page and search them through read_attachment.

* fix(agent): preserve attachment access and image reads

* fix(agent): bound attachment cleanup and image processing

* Remove standalone read_attachment tool

* Read attachments and images through read_file

Merge read_attachment into read_file (attachment:// paths, numbered
document images, staged-copy aliasing, query search), return sandbox
images as images, describe images for non-vision models inside the
tool call with bounded concurrency and model-chosen detail/focus,
render image reads in the UI, and record image metadata in Langfuse.

* fix(agent): make attachment pagination complete and bound image reads

* refactor(agent): separate attachment reads and cache image observations

* fix(attachment): read image batches concurrently and harden description cache

- ReadImages authorizes once and describes images in parallel
- start VLM deadlines after a slot is held; longer timeout for full reads
- detach shared description calls from the first caller's context
- skip rewriting unchanged sandbox projections
- strip the description cache from temporary document API responses
- render upload text before calling the vision model

* fix(attachment): number images in document order and polish attachment UI

- return resolved images in document order instead of reverse
- show attachment file names instead of attachment:// IDs in tool titles
- use the shared file icon and a lighter card for chat attachments

* fix(ci): renumber attachment migrations and satisfy lint

Main already took sqlite 000035 and versioned 000116, so the attachment
lookup indexes move to the next free numbers. Also wrap the ReadImages
signature and drop an unused test parameter.

* fix(ci): expect sqlite migrations to reach version 36
2026-10-13 14:37:17 +08:00
Ochan Kwon 809bda7b2f
fix(agent): make agent tool selection match the tools the runtime registers (#4211)
The tools a smart-reasoning agent may use were defined in three places
that never checked each other: the agent editor's checkboxes, the stored
allowed_tools, and what registerTools registers. The editor drew from a
hard-coded list that had drifted (search_conversations, a default tool,
had no checkbox), hid stored entries outside that list, and could not
untick a tool whose knowledge base capability was missing. The save API
stored any name. And an empty list could not mean "no tools": three
places treated it as "use the default tools", so clearing every checkbox
gave an agent without a knowledge base search_conversations, while the
editor's preview said it had none.

One source for the tool list: GET /api/v1/agents/tools serves the tools
of AvailableToolDefinitions with their group, danger flag and knowledge
base requirement, the default tools, and the tools other agent settings
register whatever the list says. The editor renders from it, falling
back to the catalog's label for a tool it has no translation for.

Stored values stay visible: a missing capability only blocks ticking a
tool, and stored names the catalog does not offer are shown ticked in
their own section and can be removed.

An empty list now means no built-in tools; only a list that was never
set selects the defaults. tools.EffectiveAllowedTools makes that
decision for tool registration, the session config and the rerank
requirement. Migration 000116 (SQLite 000035) writes the default tools
into the empty list of every existing smart-reasoning agent, so they
keep the tools they ran with. The knowledge base filter follows the
stored list, so such an agent with an "all" scope now gets the same
scope as one that ticked the defaults: bases search_knowledge cannot use
leave it. The editor writes the defaults into an unset list it loads,
so saving does not silently turn the defaults into no tools.

Saving validates the list: retired names are rewritten, switch-managed
and retired skill tools are dropped, names the request adds that no
tool answers to are rejected with a 400 that lists them, and unknown
names already stored are kept so an existing agent can still be saved.
The response reports what was kept or dropped.
2026-10-13 11:29:24 +08:00
tayadwx 98362dd296
feat(gallery): filter images by source document (#4159)
* feat(gallery): filter images by source document

Add a gallery toolbar button that opens a paginated, keyword-searchable
document picker; confirming restricts the listing to images owned by
the selected documents. The backend gallery endpoint accepts a
knowledge_ids query param (capped at 50) narrowing chunk_images by the
owning document before de-duplication, so the count query stays index
served.

* feat(gallery): lift knowledge_ids cap via POST body, sortable doc picker

Add POST /knowledge-bases/:id/images accepting the gallery filter as a
JSON body so knowledge_ids escapes the URL-length cap (50 -> 1000, an
abuse guard rather than a transport limit); the GET variant keeps its
cap and both share the contract-constrain logic. The source picker
gains sortable column headers (file name / upload time, newest first
by default), a header select-all that unions and removes whole hit
lists across keyword searches, overflow guards capping the no-keyword
view at the 500 most recent documents, and a wider dialog.

* style(gallery): replace td-token fallbacks and font-size literals with design tokens

styleGuard.test.mjs enforces ratchets: --td-* vars must not carry
fallbacks (theme.css guarantees them) and font sizes must use
--app-text-* tokens. GallerySourcePicker.vue added 4 of each,
pushing the ratchets over baseline.

* fix(gallery): keep shared images in source filter and align picker UI

- Scope the de-duplication probe to the selected documents, so an image
  whose newer copy lives in an unselected document is no longer hidden.
- Reset the source filter when the gallery switches knowledge bases.
- Accept an empty POST /images body; drop pruned attribute maps to nil.
- Keep the source filter out of the filter panel's count and clear action;
  the empty state's clear still resets it.
- Rework the picker dialog to the BatchTagDialog shell: heading with
  subtitle, footer with selection count, clickable rows, debounced search,
  select-all toggling against every hit, stale select-all responses dropped.
- Remove the unreachable GET knowledge_ids branch in the API client.

---------

Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-12 15:43:28 +08:00
Ochan Kwon be581c41e0
fix(mcp): close only the authorizing user's connection after OAuth (#4138)
An OAuth MCP service keeps one cached connection per user. After a
successful OAuth callback, after a token is revoked, and when an agent
resumes from an in-conversation authorization, the service's connections
were recycled with CloseClient, which disconnects every user's
connection and cancels connection attempts still in flight. Closing a
Streamable HTTP connection cancels the requests it has in flight, so one
user authorizing again cancelled other users' running tool calls on the
same server process. The agent retries a failed call once on a fresh
connection, but a long call fails, and a non-idempotent tool can run
twice.

A connection is not affected when another user's token changes: each
one reads its own user's token from the store on every request. Add
MCPManager.CloseClientForPrincipal, which closes only that user's
connection, and use it when a token is revoked.

After an authorization, the callback and the in-conversation retry call
CloseClientsAfterAuthorization instead. It closes the authorizing user's
connection, since the new authorization may be for another account of
the upstream service, and every connection of the service in the tenant
that was built with an OAuth client ID other than the registration
stored now. The registration is shared by all users of the service and
can be replaced by any replica. A connection still holding the old
client ID can fail its next refresh with invalid_client, and handling
that failure deletes the stored registration, by then the replacement.
If the registration cannot be read, the whole service is closed as
before, unless the caller's context has ended, in which case only the
authorizing user's connection is closed. The callback bounds this
cleanup with its own timeout.

This runs in the process that handles the callback and, for an
in-conversation authorization, in the process where the agent waits.
Connections cached by other replicas are not examined, as before, and
neither are other users' connection attempts still in flight.

CompleteAuthorization now returns the tenant, service and principal of
the attempt, and connections report the client ID they were built with.
Disconnecting happens after the manager lock is released, because
closing a transport can wait on the server.
2026-10-12 11:19:42 +08:00
JLi 643f64180e
feat(system): add platform user management and streamline settings (#4153)
* feat(system): platform user management — list, enable/disable, reset password, role toggle

Adds a SystemAdmin-only user management panel under platform settings
(System global -> Access). The backend already had repository primitives
and promote/revoke endpoints; this wires the missing service/handler/route
for listing and account-status changes and adds the UI.

Backend
- New GET /system/admin/users: paginated, cross-tenant user list
  (offset/limit, limit capped at 200), serialised via ToUserInfo so password
  hashes are never exposed; has_more derived with a limit+1 fetch (no COUNT).
- New POST /system/admin/users/status: enable/disable an account.
  Disabling revokes all of the user's tokens immediately. Service guards
  reject self-disable and disabling the last active system administrator.
- ValidateToken now re-checks is_active on every request, so a disabled
  account cannot keep using a previously issued access token.

Frontend
- UserListDialog: table with status/role tags, pagination, in-page
  username/email filter, enable/disable and reset-password actions, and
  one-click system-admin promote/revoke (reuses existing endpoints) with
  confirmation prompts. Reset-password dialog is a sibling (not nested)
  dialog so close/cancel/overlay all work.
- All new strings are i18n keys under system.globalSettings.userList in
  all six locales (en-US, zh-CN, zh-TW, ja-JP, ko-KR, ru-RU).

Tests: handler unit tests for list pagination/redaction/limit-cap and for
status-change success/guard/404 paths; existing handler/service/router
suites pass.

Co-authored-by: MOSS Agent <noreply@example.com>

* fix(system): make account status changes atomic and validate requests

* refactor(system): streamline settings and add dedicated user management tab

* fix(system): resolve user management CI lint findings

* fix(system): preserve account status during ordinary user updates

---------

Co-authored-by: desksk <26273255+desksk@users.noreply.github.com>
Co-authored-by: MOSS Agent <noreply@example.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-11 17:07:00 +08:00
nullptr-error 150d8cc605
feat(sandbox): Docker 后端支持交互式终端 (#4023)
Docker 沙箱后端支持会话侧栏的交互式终端:TTY exec 劫持连接收发原始字节,窗口大小走 ExecResize;终端打开期间定期刷新空闲回收活动标记。

Docker exec 无法重新附着,关闭时按每次打开注入的 WEKNORA_TERMINAL_ID 向 shell 及其子进程发送 SIGHUP(nohup 任务保留),仅对仍存活的 shell 补 SIGKILL。ready 帧新增 reattachable,Docker 为 false 时前端不自动重连。Docker 沙箱未运行时返回 SANDBOX_STOPPED,提示可能已被回收。

Co-authored-by: WHUTcjh-2024 <202126324+WHUTcjh-2024@users.noreply.github.com>
2026-10-10 21:38:03 +08:00
hahahahahayesyeseys 89e3273daf fix(mcp): preserve disabled state on service creation 2026-10-10 20:50:05 +08:00
诉说晚秋 932b750b2f
fix(agent): preserve steering order across persistence failures (#4078)
* fix(agent): 保持追加指令在持久化失败后的消费顺序

* fix(session): 限制追加指令持久化重试次数,避免单条消息阻塞队列

同一条追加指令的用户消息连续写库失败 3 次后,将其标记为 consumed+dropped 退出队列,
后续指令按原顺序继续注入,也不会把它作为 follow-up 的 query 导致整批积压滞留。
入队时去除 NUL 字符(PostgreSQL text 列不接受)。

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-10 20:32:20 +08:00
Sairaj-24 8125ed250f
fix(chat): keep custom metadata on knowledge-chat references (#4087)
* fix(chat): keep custom metadata on knowledge-chat references

Redis round-trips and graph or analysis result builders dropped knowledge custom metadata, so chat citations lost fields such as course and lesson URLs.

* test(chat): wrap metadata fixtures to satisfy line length lint

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-10 14:20:38 +08:00
lyingbug c3f077ed18
fix(models): align configuration and multimodal embedding workflows (#4080)
* fix(models): align configuration defaults and multimodal workflows

* feat(image): enhance image retrieval diagnostics and filtering

- Implemented `retrieveObservedImagePool` to observe engine calls and record image recall decisions without altering results.
- Added detailed diagnostics in `summarizeImagePool` and `summarizeImageCandidateCut` to track image filtering decisions and scores.
- Introduced tests for image pool diagnostics to ensure accurate filtering and decision recording.
- Updated existing functions to utilize new retrieval and summarization methods for improved observability and accuracy in image handling.

* feat(embeddings): enhance image handling with new formats and message support

- Introduced `EmbeddingImageFormat` to manage image input formats, allowing for both object and messages formats.
- Updated `EmbeddingsSettings` to include new fields for `ImageFormat`, `ImagePrompt`, and `TextAsMessages` to support multimodal embedding workflows.
- Enhanced `Client` methods to build request bodies for images in both formats, ensuring compatibility with vLLM's requirements.
- Added tests to validate the new image message handling and ensure correct request formatting for different scenarios.
- Updated documentation to reflect changes in image input handling and configuration options.

* feat(similarity): add embedding similarity mode for enhanced document scoring

- Introduced a new `similarity` option in `ModelDebugOptions` to score documents and an optional image against a query.
- Updated `ModelDebugDrawer.vue` to support a new input mode for similarity, allowing users to select similarity as an input type.
- Enhanced the UI to display similarity results, including ranked candidates based on cosine similarity.
- Added tests to validate the functionality of the new similarity mode in embedding models.
- Updated localization files to include new strings related to similarity mode in multiple languages.

* feat(embeddings): add Qwen3-VL-Embedding support for multimodal input

- Introduced Qwen3-VL-Embedding model with support for both text and image inputs.
- Updated model configuration files to include compatibility settings for the new embedding type.
- Enhanced tests to validate the functionality of Qwen3-VL-Embedding, ensuring correct handling of inputs and outputs.
- Updated documentation to reflect the new model's capabilities and usage instructions.

* test(models): wrap embedding assertions to satisfy lint

* test(opensearch): isolate HTTP transports between parallel tests
2026-10-10 11:06:00 +08:00
lyingbug 65f88f4121
feat: judge image hits by their image in rerank and show them to vision models (#4013)
* feat(models): expose embedding input modalities

Whether an embedding model maps images into the same space as text is
what will decide if a knowledge base can store image vectors. The
catalog already had the field, but most multimodal embedding models
were missing it and embedding rows never reported it.

- Catalog: declare input modalities for the Doubao and Tongyi vision
  embeddings, the Aliyun multimodal / qwen-vl embedding families and
  jina-embeddings-v4, as listed in each vendor's documentation. The
  migration baseline changes for exactly these nine entries.
- Model responses resolve capabilities for embedding rows too, so
  capabilities.input reflects the catalog merged with spec.input.
- Model editor: an "Image input" switch for embedding models. It follows
  the catalog and writes spec.input only when the row differs from it;
  picking another catalog model or switching vendor drops the
  declaration. Model cards show the image badge for such models.

* feat(embedding): embed images with multimodal embedding models

Adds the image side of an embedder, for models that map images into the
same space as their text vectors so a text query can find an image.

- api: EmbedImage (bytes + MIME type, sent inline as base64 because the
  object store is usually unreachable from the vendor) and an
  ImageEmbedder implemented by all four protocols, each in the image
  shape its reference documents: Ark image_url, DashScope contents[].image,
  Gemini inlineData parts, and on the OpenAI shape an object keyed by the
  vendor's image_field (Jina: "image"). A plain OpenAI-compatible
  endpoint names no image field and takes no images.
- Settings: image_field, max_image_batch_size (default one per request),
  max_image_bytes and image_mime_types, filled from vendor docs for the
  DashScope multimodal models, gemini-embedding-2 and Jina.
- embedding: an optional ImageEmbedder interface reached through
  AsImageEmbedder, so existing Embedder implementations and fakes are
  untouched. An embedder takes images only when the catalog (or the
  row's spec.input) lists image AND the endpoint can carry one. Images
  are checked against the documented limits before any request, then
  split at the documented images per request. The concurrency, debug
  and Langfuse decorators forward it; logs and traces carry the MIME
  type and size, never the bytes.

The migration baseline changes only for the entries whose resolved
embedding settings now carry image limits.

* feat(knowledge): index images by their own vector and recall them

When the knowledge base's embedding model embeds images, each image now
gets a vector of its own next to its caption and OCR chunks, so a query
can find an image by what it shows even where the caption left it out.

Ingestion
- After the caption/OCR chunks are indexed, the multimodal task embeds
  the image (embedding.PrepareImage converts and shrinks it to the
  vendor's documented limits) and stores it as a new image_vector chunk:
  Content is the caption (OCR text when there is none) for reranking and
  the answer context; its index row carries the image's vector with
  SourceType ImageSourceType. Scanned PDF pages are skipped, their OCR
  text serves better. A failure here is logged and never fails the task.
- The image vector gets its own chunk rather than a second index row on
  the caption chunk: OpenSearch and Weaviate key documents by ChunkID,
  so a second row would overwrite the caption's text vector, and copy,
  move and enable/disable all follow chunk IDs.
- CompositeRetrieveEngine.BatchIndexVectors writes precomputed vectors
  through each engine's normal BatchIndex, keeping row order (the
  composite's de-duplication does not).
- Paths that rebuild vectors from text leave image_vector alone:
  syncChunkIndex only syncs its enabled flag, updateChunkVector skips
  it, and a caption edit updates its Content only.
- The multimodal task's caption/OCR index rows now set IsEnabled; only
  backends with a column default were storing them as enabled.

Retrieval
- A text query scores lower against an image than against text, so when
  the model embeds images the document vector search is widened by half
  (capped at 500) and run at min(VectorThreshold, 0.1); each store
  group's hits are then held to their own threshold before score
  normalization: text at VectorThreshold, images at min(VectorThreshold,
  0.1). Keyword hits on image rows are dropped, their Content being the
  already-indexed caption.
- image_vector hits are searchable and merge like caption hits: parent
  context is attached, and the caption is not repeated from ImageInfo.

Adds golang.org/x/image for scaling and WebP/BMP decoding.

* fix(models): report no thinking levels for embedding, rerank and ASR models

Embedding rows now carry capabilities, and the end-to-end run showed them
listing off/auto/minimal/low/medium/high: these references resolve no chat
protocol, and an empty level map reads as "every level supported". A model
without a chat protocol does not think, so its capabilities say so.

* fix(compaction): never read a cancelled turn's stream as a summary

streamSummary selects on the turn's context and the provider stream at
once. When the turn is already cancelled both are ready (providers close
the stream once their context ends), and select picks at random, so a
stopped turn could come back as a completed, partial summary with a nil
error. TestSummarizationIsNotRetriedOnceTheTurnIsCancelled caught it
intermittently in CI and reproduces under -race with several -cpu values.

Check the context before each receive and again when the stream closes.

* feat(rerank): score images with multimodal rerank models

Rerank models that read images can now be sent an image document and
score it against a text query on the same scale as text, which is what a
recall made by an image's own vector needs: a text reranker only sees the
caption, and rejects an image for what the caption left out.

- api: ImageReranker, implemented by the Cohere and NIM shapes. Image
  documents go in requests of their own, so text requests keep their
  exact shape. NIM sends image-only passages without text, as base64 data
  URLs (its VLM reranking reference requires both). The Cohere shape sends
  {"<image_field>": data URI} documents and asks for no echo of them.
- The image settings embeddings already had (image_field,
  max_image_batch_size, max_image_bytes, image_mime_types) move into a
  shared ImageInput embedded in both settings; resolved embedding settings
  serialize exactly as before.
- Catalog: jina-reranker-m0 and nvidia/llama-nemotron-rerank-vl-1b-v2 take
  text and image. Jina names "image" documents; its base64 form is not
  shown in any reranker example, so the data URI its embeddings reference
  documents is used and marked unverified. NVIDIA lists its six formats
  and 512 passages per request.
- rerank: an optional ImageReranker reached through AsImageReranker, gated
  on the model's catalog input and the endpoint; images are checked against
  the documented limits before any request, scores are normalized like
  text, and the debug and Langfuse decorators forward it without logging
  image bytes.
- Image preparation (format conversion, shrink to fit) moves from the
  embedding package to internal/models/imageprep, since rerank needs it
  too.

* feat(retrieval): keep a vector hit's own similarity through fusion

Hybrid fusion replaces a hit's Score with a rank-based RRF score. That is
right for ordering, but it erases the evidence reranking needs to judge an
image found by its own vector: how close the vector actually was. Record
the engine's similarity on each vector hit (IndexWithScore.VectorScore)
before normalization and fusion rewrite Score, and carry it onto the
search result (SearchResult.VectorScore, not serialized).

* feat(reranking): judge image hits by their image, not only their caption

An image found by its own vector was reranked on its caption and OCR text,
so a text reranker rejected it for exactly what the caption left out.

- With a reranker that reads images and a LoadImage callback, image_vector
  hits are scored by their image (prepared to the model's limits) on the
  same scale as text, and pass or fail the same threshold. A hit whose
  image cannot be read, prepared or scored falls back to its text.
- With a text reranker, its verdict stands for images that are mostly
  text (OCR of 200 characters or more). A pictorial hit it rejects is kept
  when its vector similarity (VectorScore) reaches Options.ImageKeepScore:
  at most two, strongest vector first, appended after the ranked results
  and marked kept_by=image_vector. This also returns an image when no text
  passes at all.
- Diagnostics report images_scored and images_kept.

* feat(rerank): read image hits' images in every rerank path

The chat pipeline, the agent search_knowledge tool and the hybrid-search
API now pass the rerank stage a way to read an image hit's image and an
image keep score, so a multimodal reranker scores images by what they show
and a text reranker can no longer drop a pictorial image its caption
undersells.

- KnowledgeBaseService.ReadChunkImage reads the first image of a result's
  image_info from the storage of its knowledge base (the knowledge base's
  tenant, so shared knowledge bases read from their owner's storage),
  capped at 32 MB. Plain http(s) images go through the SSRF-safe
  downloader.
- The backend resolution the image multimodal task used moves into a
  shared knowledgeImageReader, so ingestion and reranking find an image in
  the same place.
- reranking.ImageKeepScoreFor: an image is kept over a text reranker only
  when its vector was at least as close as a text hit had to be, and never
  below 0.25 (empirical).

* feat(chat): show a vision model the images its answer rests on

An image recalled by its own vector reached the answer as its caption only,
so a vision model could not answer from what the caption left out.

- Knowledge QA: INTO_CHAT_MESSAGE reads, for a model that supports vision,
  the images of the contexts that rest on an image vector match (at most
  three distinct ones, prepared to what every vision API takes: PNG, JPEG,
  WebP or GIF within 5 MB), attaches them after the user's own images, and
  names the contexts they belong to.
- Agent: search_knowledge attaches the same images when the agent's model
  supports vision (the agent describes tool images to models that cannot
  see them, and these already carry captions). Stored history drops them;
  they are read again from storage.
- An image hit and its caption hit read the same once merged, and
  de-duplication keeps either. The kept copy now inherits the image
  (image_vector_match), in the pipeline's exact and partial-overlap
  de-duplication and in the agent tool's.
- Results the rerank stage kept outside its top-k (kept_by) no longer fall
  to the later top-k cuts: the pipeline's FilterTopK and the agent tool's
  MMR and limit let them ride along after the ranked results.

* docs(website): describe how image hits are reranked and shown to vision models

* Fix image evidence preservation and model source mapping
2026-10-09 13:01:48 +08:00
hahahahahayesyeseys 419f17fc65
fix(wiki): resolve titles for unloaded backlinks (#3931)
* fix(wiki): resolve titles for unloaded backlinks

* fix(wiki): return backlink titles with page details

* docs(wiki): verify backlink titles in page detail schema
2026-10-09 10:50:52 +08:00
lyingbug 0c5d28abb7
feat: index and recall images with multimodal embedding models (#3683)
* feat(models): expose embedding input modalities

Whether an embedding model maps images into the same space as text is
what will decide if a knowledge base can store image vectors. The
catalog already had the field, but most multimodal embedding models
were missing it and embedding rows never reported it.

- Catalog: declare input modalities for the Doubao and Tongyi vision
  embeddings, the Aliyun multimodal / qwen-vl embedding families and
  jina-embeddings-v4, as listed in each vendor's documentation. The
  migration baseline changes for exactly these nine entries.
- Model responses resolve capabilities for embedding rows too, so
  capabilities.input reflects the catalog merged with spec.input.
- Model editor: an "Image input" switch for embedding models. It follows
  the catalog and writes spec.input only when the row differs from it;
  picking another catalog model or switching vendor drops the
  declaration. Model cards show the image badge for such models.

* feat(embedding): embed images with multimodal embedding models

Adds the image side of an embedder, for models that map images into the
same space as their text vectors so a text query can find an image.

- api: EmbedImage (bytes + MIME type, sent inline as base64 because the
  object store is usually unreachable from the vendor) and an
  ImageEmbedder implemented by all four protocols, each in the image
  shape its reference documents: Ark image_url, DashScope contents[].image,
  Gemini inlineData parts, and on the OpenAI shape an object keyed by the
  vendor's image_field (Jina: "image"). A plain OpenAI-compatible
  endpoint names no image field and takes no images.
- Settings: image_field, max_image_batch_size (default one per request),
  max_image_bytes and image_mime_types, filled from vendor docs for the
  DashScope multimodal models, gemini-embedding-2 and Jina.
- embedding: an optional ImageEmbedder interface reached through
  AsImageEmbedder, so existing Embedder implementations and fakes are
  untouched. An embedder takes images only when the catalog (or the
  row's spec.input) lists image AND the endpoint can carry one. Images
  are checked against the documented limits before any request, then
  split at the documented images per request. The concurrency, debug
  and Langfuse decorators forward it; logs and traces carry the MIME
  type and size, never the bytes.

The migration baseline changes only for the entries whose resolved
embedding settings now carry image limits.

* feat(knowledge): index images by their own vector and recall them

When the knowledge base's embedding model embeds images, each image now
gets a vector of its own next to its caption and OCR chunks, so a query
can find an image by what it shows even where the caption left it out.

Ingestion
- After the caption/OCR chunks are indexed, the multimodal task embeds
  the image (embedding.PrepareImage converts and shrinks it to the
  vendor's documented limits) and stores it as a new image_vector chunk:
  Content is the caption (OCR text when there is none) for reranking and
  the answer context; its index row carries the image's vector with
  SourceType ImageSourceType. Scanned PDF pages are skipped, their OCR
  text serves better. A failure here is logged and never fails the task.
- The image vector gets its own chunk rather than a second index row on
  the caption chunk: OpenSearch and Weaviate key documents by ChunkID,
  so a second row would overwrite the caption's text vector, and copy,
  move and enable/disable all follow chunk IDs.
- CompositeRetrieveEngine.BatchIndexVectors writes precomputed vectors
  through each engine's normal BatchIndex, keeping row order (the
  composite's de-duplication does not).
- Paths that rebuild vectors from text leave image_vector alone:
  syncChunkIndex only syncs its enabled flag, updateChunkVector skips
  it, and a caption edit updates its Content only.
- The multimodal task's caption/OCR index rows now set IsEnabled; only
  backends with a column default were storing them as enabled.

Retrieval
- A text query scores lower against an image than against text, so when
  the model embeds images the document vector search is widened by half
  (capped at 500) and run at min(VectorThreshold, 0.1); each store
  group's hits are then held to their own threshold before score
  normalization: text at VectorThreshold, images at min(VectorThreshold,
  0.1). Keyword hits on image rows are dropped, their Content being the
  already-indexed caption.
- image_vector hits are searchable and merge like caption hits: parent
  context is attached, and the caption is not repeated from ImageInfo.

Adds golang.org/x/image for scaling and WebP/BMP decoding.

* fix(models): report no thinking levels for embedding, rerank and ASR models

Embedding rows now carry capabilities, and the end-to-end run showed them
listing off/auto/minimal/low/medium/high: these references resolve no chat
protocol, and an empty level map reads as "every level supported". A model
without a chat protocol does not think, so its capabilities say so.

* fix(compaction): never read a cancelled turn's stream as a summary

streamSummary selects on the turn's context and the provider stream at
once. When the turn is already cancelled both are ready (providers close
the stream once their context ends), and select picks at random, so a
stopped turn could come back as a completed, partial summary with a nil
error. TestSummarizationIsNotRetriedOnceTheTurnIsCancelled caught it
intermittently in CI and reproduces under -race with several -cpu values.

Check the context before each receive and again when the stream closes.

* feat(knowledge): put image vectors behind a knowledge-base switch, off by default

Image vector indexing and recall used to turn on whenever the knowledge
base's embedding model declared image input. A model that takes images is
often chosen for text alone, so an upgrade would have added an embedding
call per image, widened vector recall 1.5x and lowered the image threshold
to 0.1 for those knowledge bases without anyone asking.

ImageProcessingConfig gains image_vector_enabled (default false; stored
JSON without the key reads as off, so existing knowledge bases stay off
with no migration). Both sides now need the switch and the model:

- indexImageVector returns before touching the model unless the KB opted
  in (KnowledgeBase.IsImageVectorEnabled, which also needs vector indexing).
- applyImageRecall resolves the embedding model only when some KB in the
  search opted in, and records the opted-in KBs per store group
  (storeGroup.ImageKBIDs). Only those groups widen the pool and lower the
  threshold; image hits of any other KB, including ones indexed before the
  switch was turned off, are dropped.

The KB editor shows the switch in the image processing section, with a
warning when the selected embedding model does not declare image input;
docs describe the switch, its cost and what turning it off does.

* fix(knowledge): leave image vectors behind when cloning into a KB that is off

CloneChunk copied every chunk of the source document and then CopyIndices
for all of them, so copying a document from a knowledge base with image
vectors on into one with the switch off (same model and store) still
created image_vector chunks and their index rows, bypassing the switch
that indexImageVector honours.

Image vector chunks are now dropped from the copy when the target has not
opted in. Every engine's CopyIndices copies only the rows of chunks in the
mapping, so their index rows stay behind too.

* fix(search): refill the vector pool past stale image vectors

Image hits of a knowledge base whose switch is off were dropped only after
the engine had ranked and cut its results at TopK. Image vectors indexed
before the switch was turned off could fill that pool and push out valid
text hits: with MatchCount 5 the pool is 50, and 50 stale image rows
ranked above the text left the search empty.

No engine can filter rows by source type (Postgres and SQLite do not even
take ExcludeChunkIDs), so the exclusion cannot move into the query. When
the document vector pool comes back full and holds stale image rows, that
search alone runs again on a pool twice as large, until it holds TopK rows
that are not stale, the index runs out, or the pool reaches
maxRetrievalPoolSize (at most four extra calls from a pool of 50). The pool
is then cut back to TopK such rows, matching an engine-side exclusion
unless more than maxRetrievalPoolSize-TopK stale rows rank above the text.

* feat(i18n): translate the image vector switch and image input keys into zh-TW

The zh-TW locale arrived from main after this branch added five keys: the
knowledge base's image vector switch (label, description, unsupported-model
warning) and the embedding model's image input flag (label, description).

* fix(knowledge): leave image vectors behind when moving into a KB that is off

A reuse_vectors move relocates a document's chunks and index rows in place:
MoveKnowledgeIndices rebinds the rows by document ID and
MoveChunksByKnowledgeID rebinds the chunks. Moving a document from a
knowledge base with image vectors on into one with the switch off therefore
carried its image_vector chunks and their vectors along, the same bypass the
clone path had.

When the target has not opted in, the image_vector chunks are now deleted
during the move, their index rows first and then the chunks, so a retry
finds the chunks again rather than leaving orphaned rows. A target that
opted in still receives them unchanged.

* fix(retriever): keep image vectors out of keyword-only engines

BatchIndexVectors handed image rows to every engine of the composite,
including one that serves keyword retrieval alone (Elasticsearch v7 next to
a vector store). There the row holds nothing but the image's caption, which
the caption's own chunk already indexes, and retrieval discards every
keyword hit on it.

Image rows now go only to engines that serve vector retrieval. An engine
that serves both keeps one row for both kinds, so its keyword search still
meets them; the search side handles that.

* fix(search): refill the keyword pool past image rows

Every engine that serves both kinds of retrieval stores one row per index
entry and runs its keyword search over that row's content, so an image
vector row, whose content is its caption, is a keyword hit wherever the
caption's own chunk is. filterImageHits drops all of them, but only after
the engine has cut its ranking at TopK, so they crowded out text and caption
hits; a keyword ranking that put the image rows first came back empty.

The keyword pool is now refilled the way the vector pool is refilled past
stale image vectors: when it came back full and held image rows, the
keyword search runs again on a pool twice as large, up to
maxRetrievalPoolSize, and is cut back to TopK hits that are kept.

Leaving the rows out of the keyword index instead would mean writing them
without searchable content (Milvus and Tencent VectorDB derive a BM25
sparse vector from it, and how they take an empty one is unverified) or a
source-type filter in every engine's keyword query (ten engines, and
filterable fields that existing collections lack).

* revert: let image vectors move with the document into a KB that is off

This reverts commit e259977fe915f7d0bc30c4b02bc7af04da1c8dd4.

Deleting the image vector index rows by chunk ID before the move is not
reliable: the Weaviate and Elasticsearch v8 delete wrappers ignore partial
failures, and OpenSearch rejects more than 1000 IDs with ErrBatchTooLarge.
A partial delete followed by the chunk delete leaves index rows whose chunks
are gone, which nothing ever cleans up.

Without it, the image_vector chunks and their index rows move together
with the document. While the target's switch is off they are filtered at
query time (the pool refill keeps them from crowding out text), they are
removed with the document on delete, reparse or knowledge base delete, both
chunks and index rows by knowledge ID, and they become active again if the
target turns the switch on. The clone path keeps dropping them, since there
the rows are copied rather than moved.

* fix(search): keep the image row refill from losing hits it already had

refillPool replaced a pool with the answer to each larger search. A refill
that answered only in part (RetrieveResult.Error, swallowed into a warning
by retainPartialResults, as when some collections of the store did not
answer) could hold fewer text rows than the complete smaller pool, or none:
40 stale image rows above 10 text came back from a partial refill as the
40 image rows alone, and the search lost all 10 text hits. A refill that
failed outright failed the whole store group, discarding a first round that
had succeeded.

A refill is now best effort. A failed or incomplete refill search stops
the refill and leaves the last complete pool in place, cut back to TopK
kept rows as before, so a refill can no longer leave a search worse off
than it was without it. Merging the partial answer into the pool was not
done: its rows would need deduplicating and its ranking is cut at a
different TopK over a different set of collections, while keeping the
previous pool is exact.

* docs(knowledge): say what happens to image vectors in a KB that is off and what the refill costs

The image vector section now says that with the switch off, image vectors
that were indexed before it was turned off, or that came along when the
document was moved in from a KB that has it on, are kept but filtered at
query time, are removed when the document is deleted or reparsed or the
KB is deleted, and are recalled again once the switch is turned on; a copy
into such a KB leaves them behind.

It also states the cost of keeping dropped image rows from crowding text
out of the pools: the vector and keyword pools are searched again with
TopK doubled up to the 500 candidate cap, which adds queries for store
groups that hold such rows, and text can still be missed when more than
about 500 candidates are image rows. A failed or partial refill keeps the
previous complete pool.

* fix(search): log unexpected refill result sets, and describe the refill stop condition precisely
2026-10-08 18:19:53 +08:00
Junhao Ye d2bc0df6a2
feat(i18n): add Traditional Chinese (zh-TW) locale support (#3916)
* i18n(zh-TW): add Traditional Chinese (Taiwan) locale bundle (6,547 keys)

機械產生自 zh-CN.ts:以 zhconv (zh-tw) 為字形基底,詞彙層以 tw-terms.tsv(364 條生效詞條)覆寫,保留 26 條 never 與 47 條保護詞。骨架等價、鍵序列、Node 真實載入、內嵌區塊 四道檢查皆通過(詳見審計目錄)。

* i18n(zh-TW): register locale bundle in i18n infrastructure (5 → 6)

- resolveDefaultLocale.ts: SUPPORTED_LOCALES = ['zh-CN','zh-TW','en-US','ru-RU','ko-KR','ja-JP']
- i18n/index.ts: 載入 zhTW 並掛入 messages['zh-TW']
- localeKeyAudit.ts: 加入 LOCALE_BUNDLES['zh-TW'] 與測試守門(LOCALE_ORDER、emptyBundles)
- localeKeyAudit.test.ts: 把 zh-TW 補進期望清單
- locales/{en-US,ja-JP,ko-KR,ru-RU,zh-CN}.ts: language 區塊補 zhTW 顯示名(讓語言切換器正確顯示繁中選項)

* i18n(zh-TW): fix normalizeEmbedLocale and add zh-TW embed block (514 keys)

原實作 startsWith('zh') 直接命中 zh-TW/zh-HK/zh-Hant 並回傳 zh-CN
→ 繁中使用者將永遠看到簡體字串。修正後:
  - 繁中變體 (zh-tw/zh-hk/zh-mo/zh-hant) 排在通用 zh 之前,優先回傳 zh-TW
  - 內嵌 zh-TW 區塊加入 514 個字串,骨架與 zh-CN 區塊逐字元相同(只換區塊名)

* i18n(zh-TW): wire zh-TW into UI language pickers and TDesign locale

- App.vue: 載入 tdesign-vue-next esm/zh_TW,tdLocaleMap['zh-TW'] = zhTWConfig
- Login.vue: 語言切換器新增 { value: 'zh-TW', label: '繁體中文', shortLabel: '繁中', flag: '繁' }
  (刻意不用旗標 emoji,改以「繁」字標示,語意清楚且無爭議)
- GeneralSettings.vue: 設定頁語言下拉新增 <t-option value='zh-TW' ...>
- AgentEmbedChannelPanel.vue: 代理嵌入頻道語言選單新增繁中
- IMChannelPanel.vue: IM 頻道語言選單新增繁中

* i18n(zh-TW): widen API union types and update tests for zh-TW

- api/agent/index.ts: IMChannel.locale 聯合型別補 'zh-TW'
- api/embed/index.ts: EmbedLocaleTag 聯合型別補 'zh-TW'
- i18n/embedLocale.test.ts: supported-list 期望補上 zh-TW;新增 7 條 normalizeEmbedLocale 斷言(覆蓋 zh-tw / zh-hk / zh-mo / zh-hant / zh-Hant-TW 等變體正確路由到 zh-TW)
- locales/workspaceTerminology.test.ts: workspace 詞彙測試也對 zh-TW 做同樣的禁用詞斷言(與其他 locale 同步)

* i18n(zh-TW): add zh-TW to backend supportedLocales whitelist

- internal/types/locale.go: supportedLocales["zh-TW"] = {}
  (NormalizeSupportedLocale 會靜默丟掉未列出的 tag,導致前端語言選擇器實際寫入空字串——
   此修正保證前端選 zh-TW 時 embed 預設語言與 IM 回覆語言會被後端接受)
- internal/types/locale_test.go: 期望清單補上 zh-TW
- internal/handler/mcp_usage_instructions.go: MCP 工具使用說明對照表新增 zh-TW → 'Traditional Chinese'

* fix(i18n): translate revoke as 撤銷 in zh-TW and allow zh-TW as the runtime default locale

The zh-CN → zh-TW conversion turned every 撤销 into 復原 (restore), so
revoke buttons for OAuth tokens, device pairing, invitations and system
admins read as the opposite action. Only the undo sense (此操作不可撤销)
keeps 復原.

docker-entrypoint.sh whitelists DEFAULT_LOCALE and did not accept zh-TW,
so DEFAULT_LOCALE=zh-TW silently fell back to Simplified Chinese. Also
list zh-TW in the IM locale error and the locale hints.

---------

Co-authored-by: junhao-ye <junhao-ye@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-08 16:18:17 +08:00
zaqqmm 19542f2a1d
feat(datasource): add Seafile data source connector (#3616)
* fix(datasource): omit last_sync_cursor from the data source list response

The settings page polls GET /datasource every few seconds during a sync,
and connector cursors can grow to megabytes. No list consumer reads the
cursor, so NewDataSourceResponses now blanks it; detail, create and
update responses still return it through NewDataSourceResponse.

* feat(datasource): add Seafile connector with fingerprint-based incremental sync

Sync folders and files from one Seafile library per data source. The
connector walks the selected scope one directory level at a time,
downloads files through the link + fileserver flow without credentials,
and reconciles deletions against a per-file oid/mtime/size fingerprint
cursor that is checkpointed every 50 mutations.

- resource ids are "<repo_id>:<path>"; ancestors collapse, one library
- per-file failures become coded error items and keep the path pending
- directory failures and Retry-After > 60s abort without deletions
- forced full runs keep the old cursor as a deletion baseline
- API token never reaches cursors, metadata, logs or error text
- share the supported import extension set through internal/utils

* feat(datasource): add Seafile frontend connector and docs

Wire the Seafile data source into the editor dialog, channel labels
and the five locales, and document how to connect a Seafile 10.0.1
library.

- connector definition with base_url + API token fields and icon
- picker guard: one library per data source, at least one selection
- "library" resource type and icon in the lazy tree
- channel label in the knowledge base filter, document views and chat
- i18n for labels, guide steps and the eight seafile_* sync error codes
- website-docs integration guide, data source and extension point doc
  updates, and Seafile in the README data source lists

---------

Co-authored-by: zaqqmm <q-zhang10@qq.com>
2026-10-08 16:18:09 +08:00
LuckVd b5c8d90cc2
fix(chat_pipeline): treat non-JSON model refusals as skipped graph extraction (#3606)
* fix(chat_pipeline): treat non-JSON model refusals as skipped graph extraction

When a chunk carries nothing extractable (e.g. a table-of-contents page),
the LLM correctly answers with a prose refusal such as "抱歉…". parseOutput
then fed that prose to json.Unmarshal, failing with a confusing
"invalid character 'æ'" (the first byte of a CJK character), marking the
chunk task GRAPH_EXTRACT_FAILED and letting asynq replay the identical
doomed LLM call up to the retry limit.

Classify responses that contain no JSON structure at all (no '{' or '[')
as errModelDeclined and complete the extraction with an empty graph so the
task succeeds as an explicit skip. Malformed or truncated JSON (structure
present) still fails and retries as before.

Fixes #3600

* fix(chat_pipeline): keep empty model responses retriable in graph extraction

An empty response is usually transient (gateway returning 200 with no
content, a reasoning model spending its whole budget on reasoning,
content filtering), and a retry often succeeds. Only non-empty prose
with no JSON structure is treated as a declined extraction.

* fix(graph): skip prose refusals only in the async chunk task, keep truncation retriable

ParseGraph no longer turns a prose answer into an empty graph. It returns
an error wrapping ErrModelDeclined, and ChunkExtractService.Handle is the
only caller that treats it as a skip (span output skipped=model_declined).
The /initialization "try extract" endpoint and query-side entity extraction
report the refusal as a failure again, so the graph settings form no longer
gets its few-shot examples replaced with an empty result.

Extractor.Extract drops the ErrModelDeclined marker when finish_reason is
"length": prose with no JSON because the output budget ran out is
truncation, and asynq should retry it.

* fix(graph): retry every truncation spelling, not only finish_reason=length

Extract kept a prose answer retriable only when FinishReason was exactly
"length". max_tokens / max_output_tokens (and case/space variants that
IsLengthFinishReason already accepts), a stream that broke before the stop
event ("incomplete") and an adapter that reports no reason were still read
as a refusal, so the async chunk task skipped the chunk for good.

Only a reason that proves the model stopped on its own now counts as a
refusal. Ollama's non-streaming Chat also forwards done_reason, so a
num_predict cut-off is reported as "length" instead of an empty string.

* fix(graph): only treat a response with no JSON anywhere as a refusal

The refusal check looked at the first extracted fence, so a prose fence
followed by a JSON fence was skipped as a refusal instead of failing as a
retriable parse error.

---------

Co-authored-by: LuckVd <LuckVd@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-08 14:57:11 +08:00
5quan 3fb6b9d631
fix(kb): let the update API set vlm_config (#3704)
* fix(kb): let update API set vlm_config

UpdateKnowledgeBaseRequest only carried name/description/config, so the
PUT /knowledge-bases/:id endpoint could never change a knowledge base's
multimodal (VLM) config, even though CreateKnowledgeBase accepts a
top-level vlm_config. Thread an optional vlm_config through the handler
and service so the update path mirrors create (nil = no change).

Fixes #1723

* test(kb): pass the vlm_config argument in tests added on main

* fix(kb): accept only model-managed vlm_config fields on update

The update API copied the request's VLMConfig wholesale, so a caller
could set the legacy ModelName/BaseURL/APIKey/InterfaceType fields and
have the VLM client call BaseURL directly, outside model management and
its SSRF checks. Only Enabled/ModelID/DescriptionLanguage/
CustomInstructions are taken now; legacy fields keep their stored value.
A referenced model must exist and be a VLLM model (400 otherwise), and
disabling clears ModelID, matching the initialization config endpoint.

* fix(kb): validate and gate vlm_config on the update API

- Enforce the 4000-character custom_instructions cap
  (types.ValidateKnowledgeBasePromptInstructions), as create and
  /initialization/config already do.
- vlm_config is a KB setting: require kbSettingsAccess (owner workspace
  or admin share) like /initialization/config, so share editors get 403.
- Stored legacy model_name/base_url keep IsEnabled() true on their own,
  so disabling returned 200 but left VLM on, and managed -> disabled fell
  back to the old endpoint. Clear them when the request enables a managed
  model_id, or disables VLM without echoing back the stored
  model_name+base_url (GET-then-PUT keeps legacy on). Request legacy
  fields are only compared, never written. Enabling without model_id on a
  non-legacy KB is now a 400 instead of a silent no-op.
- Document the vlm_config semantics in the KB API reference.

* fix(kb): keep the legacy VLM endpoint only for pure legacy knowledge bases

CreateKnowledgeBase stores vlm_config as sent, so a managed model_id can
sit next to legacy model_name/base_url; resolveVLM then uses the managed
model. The update's keep-legacy paths (disable echoing the stored legacy
fields, enabled=true without model_id) cleared model_id but kept the
legacy fields, so IsEnabled() stayed true and image calls moved to the
stored base_url.

Those paths now apply only when no model_id is stored. On a mixed config
a disable clears everything, and enabled=true without model_id is a 400,
the same as on any managed knowledge base.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-08 14:37:23 +08:00
Jang Dong Ik b5fd79edf2
fix(model): treat 400 as reachable in chat connection test again (#3929)
* fix(model): treat 400 as reachable in chat connection test again

checkChatModelConnection probes with max_tokens=1 and is meant to treat a
400 as "endpoint reachable, auth OK". It detected that by matching the
go-openai wording "status code: 400". Since #3470 the error comes from
api.HTTPError ("API request failed with status 400: ..."), so the branch
never matched and the probe reported a failure.

This shows up with reasoning models behind LiteLLM: the 1-token budget
cannot finish a message, the gateway answers 400, and Test Connection
fails although the saved model chats fine.

Match the status code via errors.As on *api.HTTPError instead of the
message text.

* test(handler): drop unused request parameter in fake chat gateway

* fix(model): report a 400 that carries an auth error as auth failure

Gemini's native API answers an invalid key with 400 INVALID_ARGUMENT /
API_KEY_INVALID. Since the connection check now treats any typed 400 as
reachable, that showed "连接正常,模型可用" for a bad key. Keep the 400 pass
only when the body does not describe a credential problem, and let
classifyConnectionError recognise those bodies so the message says
认证失败 instead of a bare 连接失败.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-08 14:02:35 +08:00
VictorMaznyuk 3206e15e83
fix(docs): make `make docs` reproduce the committed Swagger again (#3751)
Running `make docs` on main no longer reproduces the committed docs/*:
every definition loses its package-qualified name (types.* instead of
github_com_Tencent_WeKnora_internal_types.*), enums disappear, ErrorCode
becomes a plain integer, and docs/swagger_contract_test.go fails
(TestModelDeleteUsageContract).

Cause: internal/localsandbox is not a dependency of cmd/server (its
importers are behind the `desktop` build tag), but swag's directory walk
reaches it. Its `const X = core.X` aliases make swag load
internal/localsandbox/core again through go/loader, and that second load
re-registers every already-deduplicated type under its short name without
enum values. Excluding the directory removes the second load.

That load was also hiding a second problem: skill_handler.go references
errors.AppError in @Failure without importing internal/errors, so with the
exclude swag stops with "cannot find type definition: errors.AppError".
A blank import lets swag resolve it.

Co-authored-by: Victor Maznyuk <victor@maznyuk.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 01:26:08 +08:00
linus-liu-web 626185a120
fix(wiki): 重建链接逐页写失败不再静默,半成品不再回 200 (#3884)
RebuildLinks 的保存循环把每页 UpdateMeta 的失败降级成一条 Warn,函数最后
无条件 return nil,于是 handler 里既有的 err != nil 分支永远进不去:任意
一页写失败(行锁等待、死锁、连接中断、超时)时调用方仍拿到
200 "Links rebuilt successfully",而这些页面的 in_links/out_links 保持旧
值,反向链接与图谱继续显示过期关系,且没有任何途径知道"只重建了一部分"。

保持尽力而为的语义不变(一页失败不影响其余页面继续写入),但把失败页聚合
起来在结尾返回,沿用同仓库 knowledge_delete.go / wiki_ingest.go 的
errors.Join 写法,以及同文件 MovePage 对同一个 UpdateMeta 调用的报错写法。
handler 无需改动:其既有 500 分支会如实返回 "N of M pages failed" 与失败页
slug。

回归用例用 SQLite 触发器让指定页的 UPDATE 真的报错(不是 mock),走真实仓库
与真实 handler:
- 部分失败:RebuildLinks 返回聚合错误,可写页仍被刷新,失败页保持旧值;
- 全部失败:错误里含 3 of 3 与全部 slug;
- HTTP 层:部分失败回 500 且错误体含失败页与底层原因,全部成功才回 200。

Co-authored-by: linus-liu-web <11640009+linus-liu-web@users.noreply.github.com>
2026-10-08 01:25:53 +08:00
Mohamed Arbi 9fa9e7c29d
fix: use consistent qdrant env settings (#3960)
* fix qdrant env settings

* fix test formatting
2026-10-07 23:59:56 +08:00
hailongzhao bccb4b151b fix(embed): 内嵌网页只传图片不输入文字时不再返回 400
内嵌网页的输入框允许只带图片或附件就点击发送,但 CreateKnowledgeQARequest.Query
带有 binding:"required",parseQARequest 也拒绝空 query,于是只传图片直接返回
400 "Query content cannot be empty"。

入口处理:去掉 binding:"required";文字为空但带有内联图片数据或内联附件时,
用 types.UploadOnlyQuestion 生成一句替用户提问的问题(中文界面为「请根据我
上传的内容回答。」,其他语言为英文),交给模型、检索、标题、会话历史索引、
追问建议和记忆使用。只有 URL 的图片不算上传,因为客户端传入的图片 URL 会被
清掉;预上传的 attachment_ids 也不算,这类文件在流开始后才解析,可能失败或
超时,届时模型没有任何内容可答。其余空 query 仍返回 400。

存储与显示:qaRequestContext 新增 userInput,保存用户消息时只存用户实际
输入,只传图片时为空,刷新后与发送当下显示一致;query 仍是给模型的问题。
steer 追问复制上一轮的请求上下文,显式设置 userInput,避免在只传图片的一轮
之后把追问存成空消息。

会话历史:文字为空但带图片或附件的用户消息,在两处历史重建里补上同一句
问题。知识问答流水线(loadAndProcessHistory)原先会整轮丢弃;Agent 历史
(LoadAgentHistory)原先会发出空的用户消息,被 SanitizeMessages 剔除后
前后两条回答被合并。

去掉 binding 标签会让 gofmt 重新对齐整个 CreateKnowledgeQARequest 的行尾
注释,这些既有的超长行因此会被 PR 的增量 lint 视为新增。按仓库惯例把字段
注释移到字段上一行(注释文字不变,swagger 描述不受影响),并把 Go 字段
KnowledgeIds 改名为 KnowledgeIDs(JSON 名仍是 knowledge_ids,接口不变)。

同步更新 swagger 文档,query 不再是必填字段。
2026-09-30 15:26:51 +08:00
lyingbug 5225edc0f7
feat(auth): 新增仅限邀请注册档位 (#3900)
* feat(auth): add invitation registration mode

* fix(auth): resolve registration mode lint errors
2026-09-30 13:32:55 +08:00
LL_Ricky 9842e51d40
fix(frontend): 修复复制模型时模型名称被改写的问题 (#3830)
* fix(frontend): 修复复制模型时模型名称被改写的问题

* fix(model): preserve model name and credentials when copying a config

Copying rewrote the upstream model name and posted redacted parameters, so the duplicate could not authenticate. Clone the stored row, cap the display name, and skip a new download when the local model is already active.

* fix(lint): clear new golangci-lint findings on model copy

Check c.Error results, wrap the long test line, and format the stub. The pull request lint range also includes the new HTTP error test, whose unused request parameter failed revive.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-29 19:32:08 +08:00
Victor Maznyuk bad7afa6ae fix(sandbox): pass desktop slot timing to the renew goroutine
renewDesktopSlot read desktopSlotRenew and desktopSlotLease from its own
goroutine. release() stops that goroutine without waiting for it, so the
read is unordered with the test cleanup that restores the values, and
`go test -race ./internal/handler/session/` fails in
TestDesktopSlotCancelsHoldWhenKeyExpires on every run.

acquireDesktopSlot now reads both durations before starting the goroutine
and passes them in; the go statement orders the reads, and the renew loop
no longer touches the package variables. Behaviour is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:01:39 +08:00
wizardchen 6c83d7ab47 revert: take the plugin system back out of main
Reverts the eleven plugin system PRs merged on 2026-09-26 so they can be
reviewed before landing:

* a829d7cc9 agent tools served by plugins (#3733)
* 4f5d475c3 Jira example plugin (#3730)
* 01222aac8 dynamic form options, OAuth connections and resource limits (#3729)
* cf502ba46 event delivery and webhooks (#3728)
* 996696057 sandboxed plugin pages (#3727)
* da4ef68d9 standalone plugin hosts (#3725)
* 728007ada remote runtime, Python SDK (#3724)
* ff49f9491 Host API and document parsers (#3721)
* 4328ab26a code plugins (#3720)
* ed63e35cc declarative plugins (#3716)
* c91a8d11f plugin system foundation (#3711)

The tree is back to 47773b5bd, migrations included: no environment ran
the plugin migrations (versioned 000115-000119, sqlite 000034-000038).
2026-09-26 14:58:18 +08:00
lyingbug 01222aac8e
feat(plugin): dynamic form options, OAuth connections and resource limits (P2 batch 7) (#3729)
* feat(plugin): dynamic form options, OAuth connections and resource limits
* fix(plugin): show what an options list waits for as a hint, not an error
2026-09-26 14:47:37 +08:00
lyingbug cf502ba46a
feat(plugin): event delivery and webhooks for plugins (P2 batch 6) (#3728)
* feat(plugin): event delivery and webhooks for plugins
2026-09-26 14:47:13 +08:00
lyingbug 996696057f
feat(plugin): sandboxed plugin pages and the @weknora/plugin-ui bridge (P2 batch 5) (#3727)
* feat(plugin): sandboxed plugin pages and the @weknora/plugin-ui bridge
* ci(pluginsdk): run the Python job's steps in bash on Windows too
2026-09-26 14:46:49 +08:00
lyingbug 728007ada9
feat(plugin): remote runtime, Python SDK and python host plugins (P2 batch 3) (#3724)
* feat(plugin): remote runtime
* feat(plugin): Python SDK and python host plugins
* fix(pluginsdk): skip reverse DNS when a Python plugin binds
2026-09-26 14:46:03 +08:00
lyingbug ff49f9491d
feat(plugin): Host API with a key-value store, and document parsers from plugins (#3721)
* feat(plugin): Host API with plugin tokens and a key-value store
* feat(plugin): document parsers from code plugins
* docs(pluginsdk): parsers and the Host API
2026-09-26 14:45:39 +08:00
lyingbug 4328ab26a9
feat(plugin): code plugins — extension protocol, Go SDK, plugin host, web search and data source adapters (#3720)
* feat(pluginsdk): extension protocol v1, Go SDK and conformance suite
* feat(plugin): run code plugins in an embedded plugin host
* feat(plugin): remote adapters for web search and data sources
* fix(plugin): show plugin errors by their message
* feat(frontend): plugin connectors and web search providers in the UI
* docs(pluginsdk): how to write, package, test and install a plugin
* fix(plugin): gofmt two files and read openapi.yaml line-ending agnostic
* fix(plugin): mark a host plugin ready before Activate returns
2026-09-26 14:45:15 +08:00
lyingbug ed63e35cc7
feat(plugin): install declarative plugins — MCP servers, skills and model vendors (#3716)
* feat(plugin): install declarative plugin packages across nodes
* feat(plugin): serve installed plugins' MCP servers in each workspace
* feat(plugin): let installed plugins contribute skills and model vendors
* feat(plugin): register plugin skills into the workspace skill catalog
* feat(frontend): manage installed plugins and their configuration
* test(weaviate): read the class state before the probe barrier
* fix(plugin): keep MCP tool directories reachable behind the plugin wrapper
* test(plugin): wrap a long line
2026-09-26 14:44:34 +08:00
lyingbug c91a8d11f8
feat(plugin): plugin system foundation — builtin plugins, config schemas and workspace switches (#3711)
* feat(plugin): add plugin manifest, registry and builtin catalog
* feat(plugin): add config schemas with unified secret handling
* feat(frontend): render config schemas with SchemaForm
* feat(datasource): serve connector credential forms from the backend
* feat(im): describe IM platforms and credentials from the backend
* feat(frontend): render model vendor extra fields from the config schema
* feat(frontend): build the settings modal from an extension registry
* refactor(agent): register builtin tools and their chat views
* feat(plugin): let workspaces enable and disable plugins
* feat(plugin): add the runtime driver interface and gather builtins
2026-09-26 14:43:28 +08:00
xiaojunxiang 0bf105499b
fix(knowledge): reject invalid JSON uploads before async parse (#3360)
* fix(knowledge): reject invalid JSON uploads before async parse

Validate .json payloads at HTTP upload/replace time with the same gate used
by the simple JSON reader, so JSONC or truncated files fail with HTTP 400
instead of after document-process retries. CreateKnowledgeFromFile itself
skips the gate so datasource sync and IM still create a knowledge row that
fails in async parse.

Signed-off-by: xiaojunxiang <xiaojunxiang@kingsoft.com>

* fix(knowledge): localize JSON upload validation messages

The upload queue shows the backend message verbatim and the rest of the
upload handler speaks Chinese. Map ValidateJSONContent's errors (now
sentinels with unchanged text, still shared with the async JSON reader)
to Chinese messages at the upload/replace boundary.

---------

Signed-off-by: xiaojunxiang <xiaojunxiang@kingsoft.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
2026-09-25 22:23:27 +08:00
Perfecto 00e6366fd2
fix(datasource): validate cron schedules and persist manual-only mode (#3653)
* fix(datasource): preserve schedule state on invalid updates

* docs(datasource): document PUT /datasource/:id as a full replace

The update now always persists sync_schedule, so a partial body clears
the schedule. Show a full-object example and state that omitted fields
are reset rather than preserved.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-25 22:23:18 +08:00
MercuialC c7601b2d2a
feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535)
* feat(wiki): add cross-KB wiki search POST /wiki-search

Give external callers a knowledge-search counterpart for wiki pages: one
SQL IN query ranked by title/slug/summary/content, slim hits with
match_snippet, and Viewer retrieve auth including shared KBs.

* fix(wiki): satisfy golangci-lint on cross-KB wiki search

Check c.Error returns, wrap lines over 120 columns, and use wikiDialect
instead of the embedded Dialector selector.

* fix(wiki): address review on cross-KB search helpers

Share KB viewer auth and snippet extraction, keep ID limits in the
service layer, and escape SQLite LIKE wildcards.

* fix(lint): wrap knowledge span fixtures over 120 columns

Go Lint still diffs from the original PR base, so later main tests
in knowledge_spans_test.go were reported as new line-length issues.

* fix(wiki): authorize KB access before checking wiki enablement in cross-KB search

Checking IsWikiEnabled before AuthorizeKBAccess let a caller tell apart a
foreign KB with wiki disabled (400) from one it cannot see (404), leaking
the KB's existence and settings. Run the access check first.

---------

Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
2026-09-25 22:22:58 +08:00
lyingbug 1bf717a54d
feat(chat): open cited documents at the cited passage (#3700)
* feat(chat): open cited documents at the cited passage

Clicking a citation used to show only the parsed chunk. It now opens the
original file in the references panel, jumps to the cited place and
highlights it, narrowed to the sentence the citation supports.

Parsing records where each chunk sits in the original file
(chunks.source_locators): PDF page and paragraph boxes from the docreader,
MinerU and PaddleOCR-VL layouts; Word paragraphs, slides, sheet rows and
EPUB chapters aligned in Go against any parser engine's Markdown; text
offsets for Markdown/TXT; segment times for audio. Pipeline text rewrites
are followed with a line-diff remapper. Scanned PDF pages get their OCR
text as quotes at search time so a sentence can pick its page.

The frontend renders PDFs with pdf.js and every other previewable type
with the existing preview, then highlights with the CSS Custom Highlight
API in a highlighter yellow that stays visible against the brand green.

* fix(docreader): serialize source blocks outside main so tests import it

test_docx_tables stubs docreader.parser in sys.modules, so importing
docreader.main from a later test failed in the full unittest discover run.
The wire conversion now lives in docreader.source_wire, which needs only
the proto.

* fix(chat): tighten source locators from review

- Remap rewritten-line-for-line stretches by line so scanned PDF pages
  no longer drift onto the wrong page.
- Keep the source preview mounted for citations in the file already open.
- Wait for in-flight PDF page renders before looking for the text layer.
- Count image rune offsets in one pass; stream xlsx rows up to maxUnits.
- Start text locators after a UTF-8 BOM, as viewers decode without it.
- Only re-query image children for scanned-page results.
- Seek audio citations without autoplay and drop stale seek listeners.
- Fix the SQLite migration's cross-reference to versioned/000113.

* fix(migrations): renumber chunk source locators after main's chunk_images

main took versioned 000113 and sqlite 000032 for chunk_images, so the
source_locators column moves to versioned 000114 and sqlite 000033.
2026-09-25 13:26:16 +08:00
tayadwx 68fe9599cd
feat(gallery): add an image gallery tab with schema-driven filters (#3694)
* feat(image): observe image attributes and let the policy decide OCR

Describe each image with the VLM as before, but ask it to also observe two
attributes about the image in the same round — how much body text it carries
(contain.text) and whether it presents data visually (contain.data_visual) —
then let a small pure function decide whether that image is worth a second,
separate OCR round.

Per-class OCR was the first shape of this idea; a class (chart / photo / logo)
is only an indirect proxy for 'is there text worth reading here', and the class
table has to grow whenever a new kind of image shows up. The decision actually
needs the observed attributes, so it reads them directly, and the attribute set
is a backend registry row rather than a new class.

The observation and the caption share one VLM call; OCR stays a separate call
(prompt isolation, see #3291). An attribute the model does not answer for is
left absent rather than filled with a conservative default, so the trace never
claims an observation that did not happen, and an incomplete observation falls
back to the on_unobserved policy (OCR still runs). The registry also carries the
panel's display text, so a new attribute needs no frontend change.

Off by default: image_attrs_enabled=false keeps today's behaviour (one caption
and one OCR per image). KB config: image_processing_config.image_attrs_enabled /
image_actions; per-upload override via process_config. Registry:
GET /api/v1/image-attrs/schema.

* feat(gallery): add image gallery tab with schema-driven filters

Adds a 4th knowledge-base tab parallel to Documents/Wiki/Graph that
browses every image asset in a KB.

Backend:
- GET /api/v1/knowledge-bases/:id/images lists image assets projected
  from chunks.image_info, with keyword (caption/OCR), multi-value
  attribute (attr_<name>=<value>), is_enabled and sort filters applied
  in memory, plus pagination.
- Resolves the source knowledge item title (SourceName) via a batched
  GetKnowledgeBatch so the viewer can show a human-readable source.

Frontend:
- ImageGallery.vue: thumbnail grid + search/sort + schema-driven filter
  panel (from GET /image-attrs/schema) + single-image viewer with
  prev/next, attributes/caption/OCR/source.
- Wired as the gallery tab in KnowledgeBase.vue; i18n keys in all 5 locales.

* feat(gallery): drive filters/search/sort from a configurable attribute contract

The gallery UI is no longer hardcoded: it renders from a self-describing
contract served by GET /knowledge-bases/:id/gallery-config (live attribute
sources, every attribute's definition with usage flags merged across the
config tiers, and the caller's search activation state).

- Attribute sources form an open registry (types.RegisterGalleryAttrSource);
  a builtin source models caption/ocr_text/timestamps/enabled so the gallery
  has zero special cases.
- Usage resolution: user > KB > system > source, field by field; the system
  tier lives in system_settings (gallery.policy, JSON, no rebuild), the KB
  tier slot is wired in the engine but not stored yet.
- Search activation is the user's own record: mode (all|custom) plus
  per-field on/off in users.preferences.gallery; source declarations never
  set defaults (unrecorded = off in custom mode).
- image_info is read through a gallery-local struct, removing the compile
  dependency on the attribute-pipeline types; the gallery builds and runs
  with or without that feature deployed.
- ListImages validates search_in / sort_by / attr_filters against the
  contract; ineligible fields are dropped, not served.

* fix(frontend): inline the shared graph-drawer stylesheet (@import instead of <style src>)

plugin-vue 6.0.6 keys unscoped src-style descriptors by the imported file
path, so AgentStreamDisplay.vue and WikiBrowser.vue — both importing
components/css/wiki-graph-drawer.less — overwrite each other during the
build and the second one crashes with "reading 'scoped'". Adding the
gallery changed the module order enough to make the collision hit every
time. Inlining the same stylesheet keeps the emitted CSS identical and
removes the shared descriptor key.

* feat(gallery): render attributes from short labels with descriptions in tooltips

A source could only say one thing about a value, so the filter panel had to put
a full sentence inside a checkbox. Attributes and their values now carry two
pieces of text: a short label for the space-constrained controls (checkbox,
search field, sort option) and a description for wherever there is room
(tooltip, detail panel).

The contract's value list becomes an array of {value, label, description}, and
the display helpers resolve each field separately: the gallery stamps the short
name on a control and moves the sentence into a tooltip. Translation overlays
are looked up per field, so a source that ships no gallery-specific text still
picks up the attribute pipeline's own wording.

* fix(gallery): let locale text win over the contract's wording

The filter panel read the backend's own label before asking the locale,
so a translated value never got a chance: the contract ships its own
default-language wording, and it shadowed the overlay for every locale.
Built keys were also off by a segment — imageAttr.<name> has to be
imageAttr.<name>.label, and the gallery's own keys live under
knowledgeEditor.wikiBrowser.gallery.attr — so builtin and system
attributes fell through to English on a Chinese UI.

* feat(gallery): top bar layout, pinned panels and per-value verdicts

The toolbar moves onto its own row above the grid, so the controls no
longer eat into the space the images are there for: search box, the two
scope switches, sort field and direction, and the result count all sit in
one flex row. The settings behind them are no longer inline either — each
opens as a popover under its control and can be pinned to a right-hand
rail, where the two panels take turns as tabs instead of both claiming
width at once.

Filtering gains a third state per attribute value. Where a value could
only be picked or not picked, it can now be "off" (hide those images),
"on" (show them whatever else says) or neutral (leave them alone), which
is what lets one attribute's rules talk about an image another attribute
tried to hide without the pair of them cancelling each other out: a
forced display outranks a forced hide, and anything no rule speaks about
stays visible.

The switch that used to only exist inside the panel now has a toolbar
home as well. "筛选:全显示" genuinely drops the attribute constraint
instead of imposing it against a panel the user cannot see.

Filtering used to empty the list the moment it was switched on. Any
image whose attribute was never observed — every image ingested before
the attribute pipeline existed — read as a mismatch for every filter,
which is not narrowing a list but losing it. Absence now stays silence:
an attribute no image carries places no constraint on it, until the
pipeline catches up and fills the value in.

Co-authored-by: WorkBuddy

* feat(gallery): expose the image attribute pipeline as a gallery source

The gallery's contract resolved to a single source, so the filter panel
could only offer "Enabled" while the attributes the observe-and-describe
pipeline fills in — contain.text, contain.data_visual — never reached the
UI. The registry is a gallery attribute source like any other: every
attribute it declares is offered to the gallery, filterable by default,
with the config tiers able to override the usage flags at runtime.

The registration lives in the attribute pipeline's own commit; this branch
carried an earlier, squapshot of that work which predated the split of an
attribute value into a short label and a description, so the registry's
value shape and this file have been brought in line with it. Both PRs now
hold byte-identical copies of the source hook, which is what lets either
one land first and the other merge cleanly afterwards.

Co-authored-by: WorkBuddy

* fix(gallery): viewer race, pin icon, panel anchoring and scope wording

Opening the viewer rendered the img element with the previous or empty
source while the proxy fetch was still producing a blob, and the error
that empty source fired locked the viewer in its failure state even
after the real image arrived. The resolved source now lands before the
flag is consulted, and a token keeps a slow fetch of one image from
overwriting a faster one that navigated past it.

The pin button drew nothing: TDesign has no "pushpin" glyph, so the
button rendered as an empty shell. It now uses pin / pin-filled, keeps
its place beside the tab bar, and a pinned rail assigns its columns
explicitly — the panel precedes the main section in the DOM, so leaving
the grid to source order handed the wide column to the panel and
squeezed the images into the rail.

The floating panel no longer appears at a hardcoded offset: it anchors
just below the arrow that summoned it, the arrow flips up while its
panel is out, a second press retracts, and any click elsewhere in the
toolbar closes it. When a rail is pinned the arrows turn to the
requested tab instead of spawning a second surface.

The scope words (全部 / 全显示 / 自定义) become plain status text rather
than a second dropdown — they name the current mode, and only the arrow
beside them opens the panel that changes it.

Co-authored-by: WorkBuddy

* fix(web): align the OCR-on-unobserved switch with the other setting rows

The switch sat bare inside its setting-row while every other switch on
the page is wrapped in setting-control, which is what reserves the right
column and centers the control vertically. With the row laid out from
the top, the bare switch hugged the row's upper edge and crowded the
description text — visibly out of place next to its neighbors. Wrapping
it in setting-control gives it the same right-aligned, centered slot as
the rest.

Co-authored-by: WorkBuddy

* fix(web): move the OCR-on-unobserved row out of the vertical schema row

The row lived inside setting-row-vertical, whose descendant rule stretches
every setting-info and setting-control beneath it to 100% width. Two
full-width flex children side by side cannot fit, so the control overflowed
the container and the switch landed past the right edge — visibly further
out than the switch on the row above. The row and its note now sit beside
the vertical block instead of inside it, under their own condition, so the
switch shares the ordinary row layout and its right edge lines up with the
"image attribute observation" switch above.

Co-authored-by: WorkBuddy

* fix(gallery): applying a verdict now activates the filter scope

Setting an off/on verdict while the panel's master switch was off
silently ignored it: buildParams only attaches attr_rules in custom
scope, and setVerdict never flipped the scope. A verdict the user just
expressed now switches the scope to custom; clearing the last meaningful
verdict returns to all, since custom with no rules constrains nothing.

* fix(gallery): search checkboxes stay visible and idle panels read as dimmed

The search panel hid its field checkboxes while the master switch was
on, so the only way to reach them was to flip the switch first. They now
sit next to the switch like the filter panel's verdicts do: dimmed in
all-field mode (where they carry no weight) yet clickable, and touching
one drops the mode to custom. The filter panel's verdict rows dim the
same way while the filter switch is off.

* fix(gallery): bulk verdict row, neutral default mark, panel polish

The clear-filters button never matched what the panel is for, so it is
now a bulk verdict row: one set of -/off/on buttons that applies the
chosen position to every value of every value-typed attribute, with the
default mark turned into a language-neutral dash. The panel head drops
its second divider (the tab bar already draws one) and the tabs follow
the toolbar's order: search left, filter right.

* fix(gallery): drop the all-fields caption and center the order arrow

The caption above the search checkboxes showed and hid with the master
switch, shoving the checkboxes down on every flip; it said nothing the
dimmed state doesn't, so it is gone. The order button's arrow is an
inline svg, which sits on the text baseline and reads as riding high;
it now centers against the label.

* fix(gallery): center the order arrow for real

vertical-align never applied: the icon is a flex item inside the
button's inline-flex text wrapper, and the wrapper's default stretch
pins the explicitly-sized 16px icon to the top of the 22px line box.
align-items: center on the wrapper is what actually centers it.

* fix(gallery): shorten panel tab labels in en and ru

The two-word labels overflowed the 300px panel and forced the tab bar
into scroll arrows; single-word labels fit in every locale that was
affected.

* docs: drop the image pipeline rewrite from this branch

The rewritten 'Go 侧接手后' paragraph belongs to the image attribute observation
work (#3635), which this branch carries as a prerequisite and which already
documents the caption_ocr / observation_driven split. Keeping a competing
rewrite of the same paragraph on the gallery branch makes it collide with
upstream, which extended that paragraph with the custom_instructions and inline
HTML table notes. Restoring the shared wording keeps the gallery branch free of
this file so upstream's edits merge cleanly.

* style: gofmt the gallery attribute types

Struct field tags in the resolved gallery config were one column narrow, so
gofmt rebroke the alignment block. Whitespace only.

* style(gallery): use design tokens in the gallery styles

The style guard ratchet fails on any new violation. ImageGallery.vue shipped
with 47 TDesign token fallbacks, 21 literal font sizes, 4 literal radii and a
0.15s transition; all of them now read the --app-* / --td-* tokens, which is
what the guard asks for and what dark mode depends on.

* lint(gallery): fix the golangci-lint findings in the gallery code

lll: the three over-long swagger @Param lines and one registry row were
shortened. ineffassign: ctx was assigned before the access check and then
reassigned, so the first value was dead; it is now declared once, after the
check. errcheck: the three unchecked c.Error calls in ListImages. revive: the
AttrSpec / AttrValue doc comment block was attached to the wrong type and split
into one block each. staticcheck S1016: the gallery registration converts the
registry value instead of restating its fields.

* lint(gallery): wrap the image query select and check the last c.Error

* style: reformat the wrapped image select in chunk.go

* fix(gallery): address review findings on ordering, KB switch and blob lifetime

- sortImageAssets: swap operands for desc instead of negating, keeping a
  strict weak ordering so ties page stably
- ImageGallery: reload on knowledgeBaseId change, drop stale list
  responses, fetch thumbnails in parallel with one cached blob per URL,
  revoke blobs on unmount, default sort to builtin:created_at
- KnowledgeBase: non-wiki KBs with a stale wiki/graph tab fall back to
  documents again
- handler: move ListImages godoc out of ListKnowledgeFolders' block and
  document attr_filters / attr_rules
- DecideOCR: a payload with no OCR conditions uses the built-in policy
  instead of silently skipping OCR

* docs(parsing): describe the caption_ocr and observation_driven image pipelines

Carried over from #3635, which this PR supersedes.

* fix(gallery): load local:// and other storage handles through the KB file proxy

The gallery's own scheme check only caught resource://, provider:// and
storage://, so local:// (the default storage), minio://, cos:// and s3://
images went straight into <img src> and never rendered. Resolve through
isProviderFileURL, the scheme list chat and wiki already use, and lock the
routing with a test.

* fix(kb-editor): keep API-customised OCR conditions on save and upload

The KB editor always wrote the registry default into image_actions.ocr.on,
so any unrelated edit replaced a knowledge base's custom OCR conditions
with the default table; the upload dialog did the same for the per-document
override. Keep a stored custom list and fall back to the registry default
only when there is none, and show the conditions actually in effect instead
of the default table in both panels.

* perf(gallery): de-duplicate, filter, sort and page images in SQL

Every gallery request used to read every image-bearing chunk of the KB,
OCR text included, decode it in Go and page in memory. A single query now
expands the image_info arrays, keeps one row per image URL, applies the
keyword / attribute / verdict / enabled filters and the sort, and returns
only the page with the filtered total.

The expansion carries narrow scalars only; filters and JSON sort keys are
evaluated on the winning entry of each image, and the page's JSON is read
back for its rows alone. Postgres runs the query with JIT off, since the
fixed 100-rows-per-call estimate of jsonb_array_elements otherwise prices
it ~100x high and compilation dominates.

On 20k images with 2 KB of OCR text each (40k chunks), a page drops from
~750 ms to ~100-200 ms on postgres and ~200 ms on sqlite; keyword search
over OCR text stays a full scan (~400-600 ms) until the text is indexed.
The filter semantics are pinned against both databases.

* perf(gallery): serve the gallery from a trigger-maintained chunk_images table

Reading images out of chunks.image_info meant parsing every image chunk of
the knowledge base on each request, so a page still cost time in
proportion to the KB. chunk_images (versioned/000113, sqlite/000032) keeps
one row per image_info entry with caption, OCR text and attributes as
columns; triggers on chunks keep it in step with every write path, and the
migration backfills existing chunks.

Copies of one image stay separate rows and the query lists the copy on the
most recently updated chunk through an indexed anti-join, so writes decide
nothing and concurrent writers need no locking. On postgres, caption and
OCR text get pg_trgm indexes when the extension is available.

On 20k images with 2 KB of OCR text each (40k chunks):
  postgres  first page 10 ms, page 500 41 ms, keyword 2 ms, filters 23 ms
  sqlite    first page  3 ms, page 500 39 ms, keyword 147 ms, filters 51 ms
down from ~100-600 ms. Writing those 40k image chunks costs 7.0 s instead
of 1.3 s on postgres (1.9 s instead of 1.0 s on sqlite), mostly the trigram
indexes; status and enable-flag updates patch rows in place.

---------

Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-25 12:33:59 +08:00
lyingbug 04adc13272
fix(retrieval): fix recall correctness, score scales, cost and agent tools (#3687)
* fix(retrieval): keep index writes and shared-KB lookups correct

Index writes
- Image OCR/caption chunks were indexed without IsEnabled, so every engine
  but Postgres stored them disabled and retrieval never returned them.
  The SQLite engine re-enables the affected rows at startup.
- FAQ edits, similar-question additions and import merges re-index under
  the old SourceID and relied on the engine overwriting it. Postgres and
  SQLite kept the stale row (ON CONFLICT DO NOTHING); ES v8 and Qdrant
  mint a new document ID and kept both. FAQ re-indexing now deletes the
  old rows first, and Postgres/SQLite BatchSave upserts by
  (source_id, source_type).
- Postgres: drop the GORM default tag on is_enabled, which replaced false
  with true on insert and indexed disabled chunks as enabled.
- SQLite: BatchSave no longer attaches vectors and FTS rows to the wrong
  rows when part of a batch already exists, and vecTables is guarded
  against concurrent map writes.
- CopyIndices: Postgres filters to the copied knowledge, pages by id and
  keeps is_enabled; SQLite copies every row of a chunk (generated
  questions included) with stable source IDs and enabled state.

Shared knowledge bases
- Chat merge: FAQ answer population and neighbor expansion look chunks up
  without the caller-tenant filter, like parent resolution (#3342).
- Neighbor expansion never cuts the retrieval hit; the length budget goes
  to the text adjacent to it.
- FAQ negative questions are applied whenever an FAQ KB is in scope and
  to chunks of shared FAQ KBs.
- Search results get image_info for shared-KB chunks.
- search_knowledge reads FAQ metadata from the search result instead of
  one tenant-scoped query per hit, so shared FAQs render their answers.

* fix(retrieval): put every retrieval score on one scale

Scores from different retrieval paths were ranked together while living on
different scales, so ordering depended on which path produced a hit.

Fusion
- RRF ranked a chunk by its position in the concatenation of every
  vector (or keyword) list, appended in goroutine completion order. The
  second list's best hit ranked behind every hit of the first, and the
  order changed between identical requests. Each list is now ranked on
  its own by score and a chunk takes its best rank.
- RRF is divided by its maximum, so hybrid scores land in [0, 1] like
  vector-only (cosine) and keyword-only (rescaled BM25) results. Raw RRF
  topped out near 0.016 and lost to any single-retriever hit it was ranked
  against across searches (document targets, model groups, FAQ calls);
  rerank's base-score term and MMR's relevance term had nothing to use.
- Keyword-only BM25 is rescaled per list; engines' BM25 is not comparable.
- FAQ iterative retrieval fuses each round like the first pass instead of
  sorting raw BM25 and cosine together, starts at twice the depth already
  searched, triggers when any single vector list is full (the old summed
  count never matched with more than one list) and replaces results per
  round.

Rerank and chat pipeline
- The FAQ boost is no longer capped at 1; capping made strong FAQs tie at
  1.0 and fall to the tie-breaker. The composite score before boosts is
  kept as composite_score.
- The FAQ direct-answer threshold is compared against that pre-boost
  score, and the FAQ that cleared it is the one marked match="exact"
  rather than the first FAQ.
- Graph hits no longer carry a fabricated retrieval score of 1.0, which
  ranked them above every scored hit when rerank did not run; the
  composite score uses the model score in its place.

Engines
- OpenSearch and Weaviate return (1 + cos) / 2; their drivers now report
  cosine similarity and map thresholds onto the engine scale, so a
  threshold means the same on every engine.
- Milvus L2 returned squared distance, ranked farthest-first downstream,
  and took the threshold as a radius; scores and radius are converted.
- The multi-store normalizer no longer maps Milvus through (s + 1) / 2,
  which put it on a different scale from every other engine and from its
  own single-store results.

Docs updated for fusion, normalization, FAQ boost and iterative recall.

* perf(retrieval): bound rerank, agent and pipeline cost; stop failing on slow or missing models

Rerank
- Rerankers report the longest document a request can carry
  (max_document_chars / max_request_chars minus the query) and the rerank
  stage fits its own passages to it, trimming appended captions, OCR and
  generated questions first. One chunk with a large screenshot's OCR used
  to exceed the vendor limit and fail the whole batch, so every candidate
  lost its score and the threshold stopped applying.
- Rerank HTTP requests default to a 60 s timeout; chat and agent contexts
  carry no deadline, so a stalled self-hosted rerank server held the turn
  open.
- OCR and caption chunks no longer repeat their own text from image_info in
  the passage.
- The chat pipeline and the agent tool rerank at most 200 candidates, best
  retrieval score first.
- A rerank model that cannot be loaded degrades to retrieval order instead
  of failing every chat turn, as the search APIs already do; without rerank
  results the merge stage keeps only RerankTopK hits instead of enriching
  every one.

Retrieval APIs
- knowledge-search match_count and rerank.top_k are capped at 200
  (unbounded values multiplied into tens of thousands of billed rerank
  passages); normalizedMatchCount clamps to the pool size so the
  over-retrieval arithmetic cannot overflow.
- hybrid-search with rerank recalls at least rerank.top_k deep.

Agent
- search_knowledge fits its rendered output to four fifths of the tool
  output budget, dropping the lowest-ranked rows and telling the model how
  many it omitted; the model's view is rebuilt from Data, which registry
  truncation never reached.
- Agent searches skip neighbor/parent/related enrichment rows, which were
  never retrieved yet tripled the rerank input.
- ParallelToolCalls is enabled. The model was always told it may call tools
  in parallel while the engine ran them one by one; only read-only tools
  overlap, anything else stays a barrier.

Chat pipeline
- Query expansion searches keywords only (its variants are local rewrites
  whose embeddings repeat the main search) and is skipped when keyword
  matching is off.
- Entity extraction, a second LLM call, is skipped for intents that need no
  KB retrieval and uses the rewritten query.
- Partial-overlap dedup tokenizes each text once instead of per pair, and
  only compares token overlap between texts within 3x of each other's size.

Storage
- Migration 000112 adds btree indexes on embeddings(knowledge_id) and
  (chunk_id); document deletes, chunk re-index and enable/tag updates
  scanned the whole table.
- Shared-KB knowledge rows are fetched in one query instead of one per ID.
- Drop GORM Debug() from the ParadeDB keyword query and the knowledge batch
  lookup, which logged SQL and user queries on every search.

Docs updated for the API limits, rerank timeout and passage fitting, agent
parallel tools and output budget, and pipeline changes.

* fix(retrieval): make agent tools and pipeline stages report what they did

Agent tools
- query_knowledge_graph now queries the graph store: entity names from the
  query are matched in each KB's graph, relations are returned (and shown to
  the model), and the entities' source chunks lead the results, with text
  search as a supplement. It used to run text search only, so a graph-only
  KB always answered "no relevant graph information".
- search_knowledge reports failures instead of "no matching chunks": when
  every search fails the tool fails with the reason; partial failures are
  listed. A failed query embedding falls back to keyword search for hybrid
  KBs, as the chat pipeline does.
- Keyword mode uses rerank to order hits, not to reject them (identifiers
  score low), and the empty-result hint no longer sends keyword mode back to
  keyword mode.
- Dedup keeps siblings under one parent, no longer merges every image chunk
  of a document through chunk_index 0, and keeps the better-scored copy.
- Search rows omit a <match> excerpt that only repeats the content; snippet
  terms are segmented for Chinese queries and skip stopwords, and FAQ
  matched questions are chosen by match strength.
- read_document query mode returns next_offset when truncated and resumes
  from offset.
- wiki_search over-fetches under document/tag scopes before filtering and
  says when every match was out of scope.
- Prompts point to read_document(id=cN, context=k) for surrounding text.
- The agent editor hides the retrieval-strategy section in agent mode,
  where search_knowledge uses the global retrieval settings.

Chat pipeline
- Intent labels are normalized; an unknown label retrieves instead of
  silently skipping the knowledge base, and web_search without web search
  enabled searches the knowledge bases.
- Image turns get a 2048-token rewrite budget, and a reply cut off by the
  cap is salvaged field by field instead of being discarded.
- History references are matched by how much of the query they cover;
  Jaccard against chunk-sized text never reached its threshold.
- Entity search resolves chunks of org-shared KBs within the turn's KB
  scope, skips chunks whose document is gone (it used to panic) and caps
  graph chunks at 30.
- Empty history is not fetched twice; fetched pages are cut on a rune
  boundary.

Retrieval
- A multi-KB search whose primary KB has no vectors embeds the query with a
  vector KB's model instead of failing on an empty model ID.
- Rerank passages include the chunk's heading breadcrumb, which is part of
  what was embedded.
- SQLite FTS searches the content column only and turns a lone Han
  character into a prefix query; TencentVectorDB ef follows TopK.

Docs updated for the agent tools and pipeline stages.

* fix(retrieval): address review findings on the retrieval fixes

- Restore NeedsRetrieval for web_search without web search enabled: the
  intent prompt "web_search" is how the user learns web search is off, and
  running KB retrieval instead bypassed it.
- query_knowledge_graph keeps the case of entity terms; Neo4j matches node
  names with case-sensitive CONTAINS, so lowercased terms never matched
  English entities.
- An empty search_knowledge or query_knowledge_graph result states partial
  failures in its text, which is all the model sees for an empty result.
- SQLite BatchSave replaces a source in one transaction, so a failed insert
  keeps the old rows instead of leaving the source unindexed.
- Reuse stringSliceValue and an exported searchutil.ContainsChinese instead
  of new copies; drop an unused named return and the FAQ obsolete-source
  delete that the full per-chunk delete now covers.

* fix(retrieval): address second review of the retrieval fixes

- query_knowledge_graph: under a document/tag scope, keep only relations
  whose entities come from an in-scope evidence chunk (the graph namespace
  is the whole KB); drop chunks whose document no longer exists; report a
  graph or text search failure even when the other source returned hits.
- search_knowledge: record the hybrid -> keyword fallback taken when the
  query embedding fails in mode_fallbacks, so an empty result is not read
  as "no semantic match".
- query understanding salvage: take a cut-off value only for the image
  description; a truncated rewrite_query or intent is dropped instead of
  replacing the user's query.
- Graph hits (no retrieval score) are kept past the no-rerank RerankTopK
  cut and the rerank candidate cap instead of always falling below them.
- FAQ boost is capped at 1 again so scores stay on the [0, 1] scale; FAQs
  tied at the cap are ordered by their pre-boost score.
- Negative-question filtering loads only candidates of FAQ knowledge bases.
- read_document query continuation starts one chunk before offset so the
  first match keeps its preceding context at a page boundary.
- graphSearchTerms and searchQueryTokens share one queryTerms splitter.
2026-09-25 10:48:18 +08:00
nullptr-error e2433b1293
feat(lite): install and run skills on the host (#3661)
* fix(desktop): keep Lite signed in with a stable JWT secret

Auto-setup fell through to the login form, and a newly generated signing key replaced the HMAC already used for presigned links. Reuse an existing HMAC, fill only a placeholder JWT secret, and clear a sticky auto-setup failure so the desktop window can enter the local account.

* fix(desktop): make wails dev start locally and store uploads at home

The dev app has no copied config, and the repo env asks for Redis and the Docker path /data/files. Stay on the repo config, use the in-memory stream manager when Redis is unset, write uploads to ~/.weknora/data/files, and allow dragging the top of the window.

* feat(lite): install and run skills on the host instead of a remote sandbox

Lite has no sandbox image, so skills are stored under ~/.weknora/skills as version directories with a live symlink, and chats execute them in the host Seatbelt. The settings UI no longer asks which sandbox to install into. The base profile opens /tmp, chat commands may use the network, and a failed ready-write rolls the symlink back to the previous version.

* fix(ci): align Lite skill checks with the JWT split and lint limits

The desktop signing test still expected the session secret to equal the AES key, and the skill settings source test still looked for the old i18n keys. Wrap the lines golangci-lint rejects so the PR checks can pass.

* fix(ci): separate multi-line test helpers for gofumpt

gofumpt requires a blank line after a multi-line function. The host skill fakes were still packed together, so lint failed on the first one.

* fix(lite): address review findings on host skill install

- Ask mode denies network again, and /tmp writes open only with a
  writable root, so nothing runs unapproved.
- Lite still reports and accepts the workspace script switch; the
  Skills page offers to re-enable it when it is off.
- The host process refuses symlinks the installer planted before it
  clears the install cache or reads the env declaration.
- An existing signing or AES key stays the signer and is never copied
  into signing.key; sessions then get their own jwt.key.
- Skill env capture writes under the host target on Lite.
- Sweep keeps the previous version and skips skills another process
  holds the lock for.
- Terminal and desktop panels never resolve remote sandboxes on Lite.
- A detached install stream resubscribes instead of freezing.
- Upload migration follows the bundle name and the old relative path.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-24 21:30:34 +08:00
nullptr-error 65d8a34ead
fix(desktop): pair the Lite browser against the local API listener (#3677)
* fix(desktop): pair the Lite browser against the local API listener

The Wails page origin cannot become the extension WebSocket, and a page-controlled host must not be able to redirect that pairing link.

* fix(desktop): check the proxy test response body close

golangci-lint errcheck rejects an unchecked Body.Close in the new desktop proxy tests.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
2026-09-24 20:43:25 +08:00
lyingbug 9fc08c2079
feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675)
* feat(retrieval): unify rerank across search APIs and expose diagnostics

The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.

- internal/reranking: one rerank stage (passage = title + cleaned body +
  captions/OCR + generated questions, threshold with degradation and
  top-1 fallback, composite score, MMR) used by the chat pipeline, the
  agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
  switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
  rerank model; an unknown requested model is a 400. Load or call
  failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
  an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.

Refs #2976

* docs(retrieval): lead with which search API to use and when

knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
2026-09-24 20:41:49 +08:00
wizardchen 3cf0be2e97 fix(embed): warn at startup when no signing key is configured
Deployments upgraded from the old .env.example keep the published example
SYSTEM_AES_KEY, which SystemHMACKey rejects since v0.8.2. Embed sessions
then fail at request time with no hint at boot.

- Log a [startup-env] warning when no usable signing key is found, and
  surface SYSTEM_SIGNING_KEY in the startup env banner.
- Name SYSTEM_SIGNING_KEY in the embed session 503 error.
- Provision the desktop signing key before printing the banner so the
  desktop build does not warn about a key it is about to create.
- Document the symptoms and fix in troubleshooting.
2026-09-24 20:29:17 +08:00