* feat(models): expose embedding input modalities
Whether an embedding model maps images into the same space as text is
what will decide if a knowledge base can store image vectors. The
catalog already had the field, but most multimodal embedding models
were missing it and embedding rows never reported it.
- Catalog: declare input modalities for the Doubao and Tongyi vision
embeddings, the Aliyun multimodal / qwen-vl embedding families and
jina-embeddings-v4, as listed in each vendor's documentation. The
migration baseline changes for exactly these nine entries.
- Model responses resolve capabilities for embedding rows too, so
capabilities.input reflects the catalog merged with spec.input.
- Model editor: an "Image input" switch for embedding models. It follows
the catalog and writes spec.input only when the row differs from it;
picking another catalog model or switching vendor drops the
declaration. Model cards show the image badge for such models.
* feat(embedding): embed images with multimodal embedding models
Adds the image side of an embedder, for models that map images into the
same space as their text vectors so a text query can find an image.
- api: EmbedImage (bytes + MIME type, sent inline as base64 because the
object store is usually unreachable from the vendor) and an
ImageEmbedder implemented by all four protocols, each in the image
shape its reference documents: Ark image_url, DashScope contents[].image,
Gemini inlineData parts, and on the OpenAI shape an object keyed by the
vendor's image_field (Jina: "image"). A plain OpenAI-compatible
endpoint names no image field and takes no images.
- Settings: image_field, max_image_batch_size (default one per request),
max_image_bytes and image_mime_types, filled from vendor docs for the
DashScope multimodal models, gemini-embedding-2 and Jina.
- embedding: an optional ImageEmbedder interface reached through
AsImageEmbedder, so existing Embedder implementations and fakes are
untouched. An embedder takes images only when the catalog (or the
row's spec.input) lists image AND the endpoint can carry one. Images
are checked against the documented limits before any request, then
split at the documented images per request. The concurrency, debug
and Langfuse decorators forward it; logs and traces carry the MIME
type and size, never the bytes.
The migration baseline changes only for the entries whose resolved
embedding settings now carry image limits.
* feat(knowledge): index images by their own vector and recall them
When the knowledge base's embedding model embeds images, each image now
gets a vector of its own next to its caption and OCR chunks, so a query
can find an image by what it shows even where the caption left it out.
Ingestion
- After the caption/OCR chunks are indexed, the multimodal task embeds
the image (embedding.PrepareImage converts and shrinks it to the
vendor's documented limits) and stores it as a new image_vector chunk:
Content is the caption (OCR text when there is none) for reranking and
the answer context; its index row carries the image's vector with
SourceType ImageSourceType. Scanned PDF pages are skipped, their OCR
text serves better. A failure here is logged and never fails the task.
- The image vector gets its own chunk rather than a second index row on
the caption chunk: OpenSearch and Weaviate key documents by ChunkID,
so a second row would overwrite the caption's text vector, and copy,
move and enable/disable all follow chunk IDs.
- CompositeRetrieveEngine.BatchIndexVectors writes precomputed vectors
through each engine's normal BatchIndex, keeping row order (the
composite's de-duplication does not).
- Paths that rebuild vectors from text leave image_vector alone:
syncChunkIndex only syncs its enabled flag, updateChunkVector skips
it, and a caption edit updates its Content only.
- The multimodal task's caption/OCR index rows now set IsEnabled; only
backends with a column default were storing them as enabled.
Retrieval
- A text query scores lower against an image than against text, so when
the model embeds images the document vector search is widened by half
(capped at 500) and run at min(VectorThreshold, 0.1); each store
group's hits are then held to their own threshold before score
normalization: text at VectorThreshold, images at min(VectorThreshold,
0.1). Keyword hits on image rows are dropped, their Content being the
already-indexed caption.
- image_vector hits are searchable and merge like caption hits: parent
context is attached, and the caption is not repeated from ImageInfo.
Adds golang.org/x/image for scaling and WebP/BMP decoding.
* fix(models): report no thinking levels for embedding, rerank and ASR models
Embedding rows now carry capabilities, and the end-to-end run showed them
listing off/auto/minimal/low/medium/high: these references resolve no chat
protocol, and an empty level map reads as "every level supported". A model
without a chat protocol does not think, so its capabilities say so.
* fix(compaction): never read a cancelled turn's stream as a summary
streamSummary selects on the turn's context and the provider stream at
once. When the turn is already cancelled both are ready (providers close
the stream once their context ends), and select picks at random, so a
stopped turn could come back as a completed, partial summary with a nil
error. TestSummarizationIsNotRetriedOnceTheTurnIsCancelled caught it
intermittently in CI and reproduces under -race with several -cpu values.
Check the context before each receive and again when the stream closes.
* feat(knowledge): put image vectors behind a knowledge-base switch, off by default
Image vector indexing and recall used to turn on whenever the knowledge
base's embedding model declared image input. A model that takes images is
often chosen for text alone, so an upgrade would have added an embedding
call per image, widened vector recall 1.5x and lowered the image threshold
to 0.1 for those knowledge bases without anyone asking.
ImageProcessingConfig gains image_vector_enabled (default false; stored
JSON without the key reads as off, so existing knowledge bases stay off
with no migration). Both sides now need the switch and the model:
- indexImageVector returns before touching the model unless the KB opted
in (KnowledgeBase.IsImageVectorEnabled, which also needs vector indexing).
- applyImageRecall resolves the embedding model only when some KB in the
search opted in, and records the opted-in KBs per store group
(storeGroup.ImageKBIDs). Only those groups widen the pool and lower the
threshold; image hits of any other KB, including ones indexed before the
switch was turned off, are dropped.
The KB editor shows the switch in the image processing section, with a
warning when the selected embedding model does not declare image input;
docs describe the switch, its cost and what turning it off does.
* fix(knowledge): leave image vectors behind when cloning into a KB that is off
CloneChunk copied every chunk of the source document and then CopyIndices
for all of them, so copying a document from a knowledge base with image
vectors on into one with the switch off (same model and store) still
created image_vector chunks and their index rows, bypassing the switch
that indexImageVector honours.
Image vector chunks are now dropped from the copy when the target has not
opted in. Every engine's CopyIndices copies only the rows of chunks in the
mapping, so their index rows stay behind too.
* fix(search): refill the vector pool past stale image vectors
Image hits of a knowledge base whose switch is off were dropped only after
the engine had ranked and cut its results at TopK. Image vectors indexed
before the switch was turned off could fill that pool and push out valid
text hits: with MatchCount 5 the pool is 50, and 50 stale image rows
ranked above the text left the search empty.
No engine can filter rows by source type (Postgres and SQLite do not even
take ExcludeChunkIDs), so the exclusion cannot move into the query. When
the document vector pool comes back full and holds stale image rows, that
search alone runs again on a pool twice as large, until it holds TopK rows
that are not stale, the index runs out, or the pool reaches
maxRetrievalPoolSize (at most four extra calls from a pool of 50). The pool
is then cut back to TopK such rows, matching an engine-side exclusion
unless more than maxRetrievalPoolSize-TopK stale rows rank above the text.
* feat(i18n): translate the image vector switch and image input keys into zh-TW
The zh-TW locale arrived from main after this branch added five keys: the
knowledge base's image vector switch (label, description, unsupported-model
warning) and the embedding model's image input flag (label, description).
* fix(knowledge): leave image vectors behind when moving into a KB that is off
A reuse_vectors move relocates a document's chunks and index rows in place:
MoveKnowledgeIndices rebinds the rows by document ID and
MoveChunksByKnowledgeID rebinds the chunks. Moving a document from a
knowledge base with image vectors on into one with the switch off therefore
carried its image_vector chunks and their vectors along, the same bypass the
clone path had.
When the target has not opted in, the image_vector chunks are now deleted
during the move, their index rows first and then the chunks, so a retry
finds the chunks again rather than leaving orphaned rows. A target that
opted in still receives them unchanged.
* fix(retriever): keep image vectors out of keyword-only engines
BatchIndexVectors handed image rows to every engine of the composite,
including one that serves keyword retrieval alone (Elasticsearch v7 next to
a vector store). There the row holds nothing but the image's caption, which
the caption's own chunk already indexes, and retrieval discards every
keyword hit on it.
Image rows now go only to engines that serve vector retrieval. An engine
that serves both keeps one row for both kinds, so its keyword search still
meets them; the search side handles that.
* fix(search): refill the keyword pool past image rows
Every engine that serves both kinds of retrieval stores one row per index
entry and runs its keyword search over that row's content, so an image
vector row, whose content is its caption, is a keyword hit wherever the
caption's own chunk is. filterImageHits drops all of them, but only after
the engine has cut its ranking at TopK, so they crowded out text and caption
hits; a keyword ranking that put the image rows first came back empty.
The keyword pool is now refilled the way the vector pool is refilled past
stale image vectors: when it came back full and held image rows, the
keyword search runs again on a pool twice as large, up to
maxRetrievalPoolSize, and is cut back to TopK hits that are kept.
Leaving the rows out of the keyword index instead would mean writing them
without searchable content (Milvus and Tencent VectorDB derive a BM25
sparse vector from it, and how they take an empty one is unverified) or a
source-type filter in every engine's keyword query (ten engines, and
filterable fields that existing collections lack).
* revert: let image vectors move with the document into a KB that is off
This reverts commit e259977fe915f7d0bc30c4b02bc7af04da1c8dd4.
Deleting the image vector index rows by chunk ID before the move is not
reliable: the Weaviate and Elasticsearch v8 delete wrappers ignore partial
failures, and OpenSearch rejects more than 1000 IDs with ErrBatchTooLarge.
A partial delete followed by the chunk delete leaves index rows whose chunks
are gone, which nothing ever cleans up.
Without it, the image_vector chunks and their index rows move together
with the document. While the target's switch is off they are filtered at
query time (the pool refill keeps them from crowding out text), they are
removed with the document on delete, reparse or knowledge base delete, both
chunks and index rows by knowledge ID, and they become active again if the
target turns the switch on. The clone path keeps dropping them, since there
the rows are copied rather than moved.
* fix(search): keep the image row refill from losing hits it already had
refillPool replaced a pool with the answer to each larger search. A refill
that answered only in part (RetrieveResult.Error, swallowed into a warning
by retainPartialResults, as when some collections of the store did not
answer) could hold fewer text rows than the complete smaller pool, or none:
40 stale image rows above 10 text came back from a partial refill as the
40 image rows alone, and the search lost all 10 text hits. A refill that
failed outright failed the whole store group, discarding a first round that
had succeeded.
A refill is now best effort. A failed or incomplete refill search stops
the refill and leaves the last complete pool in place, cut back to TopK
kept rows as before, so a refill can no longer leave a search worse off
than it was without it. Merging the partial answer into the pool was not
done: its rows would need deduplicating and its ranking is cut at a
different TopK over a different set of collections, while keeping the
previous pool is exact.
* docs(knowledge): say what happens to image vectors in a KB that is off and what the refill costs
The image vector section now says that with the switch off, image vectors
that were indexed before it was turned off, or that came along when the
document was moved in from a KB that has it on, are kept but filtered at
query time, are removed when the document is deleted or reparsed or the
KB is deleted, and are recalled again once the switch is turned on; a copy
into such a KB leaves them behind.
It also states the cost of keeping dropped image rows from crowding text
out of the pools: the vector and keyword pools are searched again with
TopK doubled up to the 500 candidate cap, which adds queries for store
groups that hold such rows, and text can still be missed when more than
about 500 candidates are image rows. A failed or partial refill keeps the
previous complete pool.
* fix(search): log unexpected refill result sets, and describe the refill stop condition precisely
* fix(datasource): omit last_sync_cursor from the data source list response
The settings page polls GET /datasource every few seconds during a sync,
and connector cursors can grow to megabytes. No list consumer reads the
cursor, so NewDataSourceResponses now blanks it; detail, create and
update responses still return it through NewDataSourceResponse.
* feat(datasource): add Seafile connector with fingerprint-based incremental sync
Sync folders and files from one Seafile library per data source. The
connector walks the selected scope one directory level at a time,
downloads files through the link + fileserver flow without credentials,
and reconciles deletions against a per-file oid/mtime/size fingerprint
cursor that is checkpointed every 50 mutations.
- resource ids are "<repo_id>:<path>"; ancestors collapse, one library
- per-file failures become coded error items and keep the path pending
- directory failures and Retry-After > 60s abort without deletions
- forced full runs keep the old cursor as a deletion baseline
- API token never reaches cursors, metadata, logs or error text
- share the supported import extension set through internal/utils
* feat(datasource): add Seafile frontend connector and docs
Wire the Seafile data source into the editor dialog, channel labels
and the five locales, and document how to connect a Seafile 10.0.1
library.
- connector definition with base_url + API token fields and icon
- picker guard: one library per data source, at least one selection
- "library" resource type and icon in the lazy tree
- channel label in the knowledge base filter, document views and chat
- i18n for labels, guide steps and the eight seafile_* sync error codes
- website-docs integration guide, data source and extension point doc
updates, and Seafile in the README data source lists
---------
Co-authored-by: zaqqmm <q-zhang10@qq.com>
* fix(kb): let update API set vlm_config
UpdateKnowledgeBaseRequest only carried name/description/config, so the
PUT /knowledge-bases/:id endpoint could never change a knowledge base's
multimodal (VLM) config, even though CreateKnowledgeBase accepts a
top-level vlm_config. Thread an optional vlm_config through the handler
and service so the update path mirrors create (nil = no change).
Fixes#1723
* test(kb): pass the vlm_config argument in tests added on main
* fix(kb): accept only model-managed vlm_config fields on update
The update API copied the request's VLMConfig wholesale, so a caller
could set the legacy ModelName/BaseURL/APIKey/InterfaceType fields and
have the VLM client call BaseURL directly, outside model management and
its SSRF checks. Only Enabled/ModelID/DescriptionLanguage/
CustomInstructions are taken now; legacy fields keep their stored value.
A referenced model must exist and be a VLLM model (400 otherwise), and
disabling clears ModelID, matching the initialization config endpoint.
* fix(kb): validate and gate vlm_config on the update API
- Enforce the 4000-character custom_instructions cap
(types.ValidateKnowledgeBasePromptInstructions), as create and
/initialization/config already do.
- vlm_config is a KB setting: require kbSettingsAccess (owner workspace
or admin share) like /initialization/config, so share editors get 403.
- Stored legacy model_name/base_url keep IsEnabled() true on their own,
so disabling returned 200 but left VLM on, and managed -> disabled fell
back to the old endpoint. Clear them when the request enables a managed
model_id, or disables VLM without echoing back the stored
model_name+base_url (GET-then-PUT keeps legacy on). Request legacy
fields are only compared, never written. Enabling without model_id on a
non-legacy KB is now a 400 instead of a silent no-op.
- Document the vlm_config semantics in the KB API reference.
* fix(kb): keep the legacy VLM endpoint only for pure legacy knowledge bases
CreateKnowledgeBase stores vlm_config as sent, so a managed model_id can
sit next to legacy model_name/base_url; resolveVLM then uses the managed
model. The update's keep-legacy paths (disable echoing the stored legacy
fields, enabled=true without model_id) cleared model_id but kept the
legacy fields, so IsEnabled() stayed true and image calls moved to the
stored base_url.
Those paths now apply only when no model_id is stored. On a mixed config
a disable clears everything, and enabled=true without model_id is a 400,
the same as on any managed knowledge base.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(frontend): 修复复制模型时模型名称被改写的问题
* fix(model): preserve model name and credentials when copying a config
Copying rewrote the upstream model name and posted redacted parameters, so the duplicate could not authenticate. Clone the stored row, cap the display name, and skip a new download when the local model is already active.
* fix(lint): clear new golangci-lint findings on model copy
Check c.Error results, wrap the long test line, and format the stub. The pull request lint range also includes the new HTTP error test, whose unused request parameter failed revive.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(server): signal shutdown raced Fatalf exit before ResourceCleaner ran
The signal goroutine closed the listener before calling server.Shutdown,
so Serve returned the raw accept error ('use of closed network connection')
instead of ErrServerClosed. The invoke error path then hit logger.Fatalf
(os.Exit(1)) in the same millisecond, killing the process before the
shutdown goroutine reached resourceCleaner.Cleanup. Every registered
cleaner — BrowserSkill daemon, sandbox reapers, IMService, AntsPool,
Langfuse — was skipped on every SIGTERM/SIGINT, leaking the shared bsk
daemon process and its temp home on each restart.
Two changes:
- Drop the manual listener.Close(): Shutdown closes all listeners as its
first step, releasing the port just as fast without racing Serve.
- Give resourceCleaner.Cleanup its own fresh timeout context so a slow
connection drain cannot leave Cleanup with an already-expired context,
which made it skip every entry without running any.
* fix(server): keep signal cleanup inside one grace period
Desktop quit still closed the listener before Shutdown and returned
before ResourceCleaner finished, so Fatalf could skip every hook.
Share ShutdownTimeout between drain and cleanup, run BrowserSkill
first, and do not drop later hooks when the deadline expires.
* fix(server): satisfy errcheck and line-length lint
---------
Co-authored-by: vicfei <vicfei@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Reverts the eleven plugin system PRs merged on 2026-09-26 so they can be
reviewed before landing:
* a829d7cc9 agent tools served by plugins (#3733)
* 4f5d475c3 Jira example plugin (#3730)
* 01222aac8 dynamic form options, OAuth connections and resource limits (#3729)
* cf502ba46 event delivery and webhooks (#3728)
* 996696057 sandboxed plugin pages (#3727)
* da4ef68d9 standalone plugin hosts (#3725)
* 728007ada remote runtime, Python SDK (#3724)
* ff49f9491 Host API and document parsers (#3721)
* 4328ab26a code plugins (#3720)
* ed63e35cc declarative plugins (#3716)
* c91a8d11f plugin system foundation (#3711)
The tree is back to 47773b5bd, migrations included: no environment ran
the plugin migrations (versioned 000115-000119, sqlite 000034-000038).
* feat(plugin): Host API with plugin tokens and a key-value store
* feat(plugin): document parsers from code plugins
* docs(pluginsdk): parsers and the Host API
* feat(pluginsdk): extension protocol v1, Go SDK and conformance suite
* feat(plugin): run code plugins in an embedded plugin host
* feat(plugin): remote adapters for web search and data sources
* fix(plugin): show plugin errors by their message
* feat(frontend): plugin connectors and web search providers in the UI
* docs(pluginsdk): how to write, package, test and install a plugin
* fix(plugin): gofmt two files and read openapi.yaml line-ending agnostic
* fix(plugin): mark a host plugin ready before Activate returns
* feat(plugin): install declarative plugin packages across nodes
* feat(plugin): serve installed plugins' MCP servers in each workspace
* feat(plugin): let installed plugins contribute skills and model vendors
* feat(plugin): register plugin skills into the workspace skill catalog
* feat(frontend): manage installed plugins and their configuration
* test(weaviate): read the class state before the probe barrier
* fix(plugin): keep MCP tool directories reachable behind the plugin wrapper
* test(plugin): wrap a long line
* feat(plugin): add plugin manifest, registry and builtin catalog
* feat(plugin): add config schemas with unified secret handling
* feat(frontend): render config schemas with SchemaForm
* feat(datasource): serve connector credential forms from the backend
* feat(im): describe IM platforms and credentials from the backend
* feat(frontend): render model vendor extra fields from the config schema
* feat(frontend): build the settings modal from an extension registry
* refactor(agent): register builtin tools and their chat views
* feat(plugin): let workspaces enable and disable plugins
* feat(plugin): add the runtime driver interface and gather builtins
* feat(wiki): add cross-KB wiki search POST /wiki-search
Give external callers a knowledge-search counterpart for wiki pages: one
SQL IN query ranked by title/slug/summary/content, slim hits with
match_snippet, and Viewer retrieve auth including shared KBs.
* fix(wiki): satisfy golangci-lint on cross-KB wiki search
Check c.Error returns, wrap lines over 120 columns, and use wikiDialect
instead of the embedded Dialector selector.
* fix(wiki): address review on cross-KB search helpers
Share KB viewer auth and snippet extraction, keep ID limits in the
service layer, and escape SQLite LIKE wildcards.
* fix(lint): wrap knowledge span fixtures over 120 columns
Go Lint still diffs from the original PR base, so later main tests
in knowledge_spans_test.go were reported as new line-length issues.
* fix(wiki): authorize KB access before checking wiki enablement in cross-KB search
Checking IsWikiEnabled before AuthorizeKBAccess let a caller tell apart a
foreign KB with wiki disabled (400) from one it cannot see (404), leaking
the KB's existence and settings. Run the access check first.
---------
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
A stream body that ran out without a finish reason (a proxy cutting the
connection) was closed as a clean stop on the Completions, Responses and
Gemini paths, so a half-written answer was stored as complete. It is now
marked incomplete, as the Anthropic path already did, and both the chat
pipeline and the agent treat an incomplete answer as a stream error.
Retry classification checks typed HTTP/transport errors first and matches
status codes as whole numbers, so "max_tokens 5000" no longer reads as a
500. The retry backoff returns as soon as the turn is cancelled.
* feat(chat): open cited documents at the cited passage
Clicking a citation used to show only the parsed chunk. It now opens the
original file in the references panel, jumps to the cited place and
highlights it, narrowed to the sentence the citation supports.
Parsing records where each chunk sits in the original file
(chunks.source_locators): PDF page and paragraph boxes from the docreader,
MinerU and PaddleOCR-VL layouts; Word paragraphs, slides, sheet rows and
EPUB chapters aligned in Go against any parser engine's Markdown; text
offsets for Markdown/TXT; segment times for audio. Pipeline text rewrites
are followed with a line-diff remapper. Scanned PDF pages get their OCR
text as quotes at search time so a sentence can pick its page.
The frontend renders PDFs with pdf.js and every other previewable type
with the existing preview, then highlights with the CSS Custom Highlight
API in a highlighter yellow that stays visible against the brand green.
* fix(docreader): serialize source blocks outside main so tests import it
test_docx_tables stubs docreader.parser in sys.modules, so importing
docreader.main from a later test failed in the full unittest discover run.
The wire conversion now lives in docreader.source_wire, which needs only
the proto.
* fix(chat): tighten source locators from review
- Remap rewritten-line-for-line stretches by line so scanned PDF pages
no longer drift onto the wrong page.
- Keep the source preview mounted for citations in the file already open.
- Wait for in-flight PDF page renders before looking for the text layer.
- Count image rune offsets in one pass; stream xlsx rows up to maxUnits.
- Start text locators after a UTF-8 BOM, as viewers decode without it.
- Only re-query image children for scanned-page results.
- Seek audio citations without autoplay and drop stale seek listeners.
- Fix the SQLite migration's cross-reference to versioned/000113.
* fix(migrations): renumber chunk source locators after main's chunk_images
main took versioned 000113 and sqlite 000032 for chunk_images, so the
source_locators column moves to versioned 000114 and sqlite 000033.
* feat(image): observe image attributes and let the policy decide OCR
Describe each image with the VLM as before, but ask it to also observe two
attributes about the image in the same round — how much body text it carries
(contain.text) and whether it presents data visually (contain.data_visual) —
then let a small pure function decide whether that image is worth a second,
separate OCR round.
Per-class OCR was the first shape of this idea; a class (chart / photo / logo)
is only an indirect proxy for 'is there text worth reading here', and the class
table has to grow whenever a new kind of image shows up. The decision actually
needs the observed attributes, so it reads them directly, and the attribute set
is a backend registry row rather than a new class.
The observation and the caption share one VLM call; OCR stays a separate call
(prompt isolation, see #3291). An attribute the model does not answer for is
left absent rather than filled with a conservative default, so the trace never
claims an observation that did not happen, and an incomplete observation falls
back to the on_unobserved policy (OCR still runs). The registry also carries the
panel's display text, so a new attribute needs no frontend change.
Off by default: image_attrs_enabled=false keeps today's behaviour (one caption
and one OCR per image). KB config: image_processing_config.image_attrs_enabled /
image_actions; per-upload override via process_config. Registry:
GET /api/v1/image-attrs/schema.
* feat(gallery): add image gallery tab with schema-driven filters
Adds a 4th knowledge-base tab parallel to Documents/Wiki/Graph that
browses every image asset in a KB.
Backend:
- GET /api/v1/knowledge-bases/:id/images lists image assets projected
from chunks.image_info, with keyword (caption/OCR), multi-value
attribute (attr_<name>=<value>), is_enabled and sort filters applied
in memory, plus pagination.
- Resolves the source knowledge item title (SourceName) via a batched
GetKnowledgeBatch so the viewer can show a human-readable source.
Frontend:
- ImageGallery.vue: thumbnail grid + search/sort + schema-driven filter
panel (from GET /image-attrs/schema) + single-image viewer with
prev/next, attributes/caption/OCR/source.
- Wired as the gallery tab in KnowledgeBase.vue; i18n keys in all 5 locales.
* feat(gallery): drive filters/search/sort from a configurable attribute contract
The gallery UI is no longer hardcoded: it renders from a self-describing
contract served by GET /knowledge-bases/:id/gallery-config (live attribute
sources, every attribute's definition with usage flags merged across the
config tiers, and the caller's search activation state).
- Attribute sources form an open registry (types.RegisterGalleryAttrSource);
a builtin source models caption/ocr_text/timestamps/enabled so the gallery
has zero special cases.
- Usage resolution: user > KB > system > source, field by field; the system
tier lives in system_settings (gallery.policy, JSON, no rebuild), the KB
tier slot is wired in the engine but not stored yet.
- Search activation is the user's own record: mode (all|custom) plus
per-field on/off in users.preferences.gallery; source declarations never
set defaults (unrecorded = off in custom mode).
- image_info is read through a gallery-local struct, removing the compile
dependency on the attribute-pipeline types; the gallery builds and runs
with or without that feature deployed.
- ListImages validates search_in / sort_by / attr_filters against the
contract; ineligible fields are dropped, not served.
* fix(frontend): inline the shared graph-drawer stylesheet (@import instead of <style src>)
plugin-vue 6.0.6 keys unscoped src-style descriptors by the imported file
path, so AgentStreamDisplay.vue and WikiBrowser.vue — both importing
components/css/wiki-graph-drawer.less — overwrite each other during the
build and the second one crashes with "reading 'scoped'". Adding the
gallery changed the module order enough to make the collision hit every
time. Inlining the same stylesheet keeps the emitted CSS identical and
removes the shared descriptor key.
* feat(gallery): render attributes from short labels with descriptions in tooltips
A source could only say one thing about a value, so the filter panel had to put
a full sentence inside a checkbox. Attributes and their values now carry two
pieces of text: a short label for the space-constrained controls (checkbox,
search field, sort option) and a description for wherever there is room
(tooltip, detail panel).
The contract's value list becomes an array of {value, label, description}, and
the display helpers resolve each field separately: the gallery stamps the short
name on a control and moves the sentence into a tooltip. Translation overlays
are looked up per field, so a source that ships no gallery-specific text still
picks up the attribute pipeline's own wording.
* fix(gallery): let locale text win over the contract's wording
The filter panel read the backend's own label before asking the locale,
so a translated value never got a chance: the contract ships its own
default-language wording, and it shadowed the overlay for every locale.
Built keys were also off by a segment — imageAttr.<name> has to be
imageAttr.<name>.label, and the gallery's own keys live under
knowledgeEditor.wikiBrowser.gallery.attr — so builtin and system
attributes fell through to English on a Chinese UI.
* feat(gallery): top bar layout, pinned panels and per-value verdicts
The toolbar moves onto its own row above the grid, so the controls no
longer eat into the space the images are there for: search box, the two
scope switches, sort field and direction, and the result count all sit in
one flex row. The settings behind them are no longer inline either — each
opens as a popover under its control and can be pinned to a right-hand
rail, where the two panels take turns as tabs instead of both claiming
width at once.
Filtering gains a third state per attribute value. Where a value could
only be picked or not picked, it can now be "off" (hide those images),
"on" (show them whatever else says) or neutral (leave them alone), which
is what lets one attribute's rules talk about an image another attribute
tried to hide without the pair of them cancelling each other out: a
forced display outranks a forced hide, and anything no rule speaks about
stays visible.
The switch that used to only exist inside the panel now has a toolbar
home as well. "筛选:全显示" genuinely drops the attribute constraint
instead of imposing it against a panel the user cannot see.
Filtering used to empty the list the moment it was switched on. Any
image whose attribute was never observed — every image ingested before
the attribute pipeline existed — read as a mismatch for every filter,
which is not narrowing a list but losing it. Absence now stays silence:
an attribute no image carries places no constraint on it, until the
pipeline catches up and fills the value in.
Co-authored-by: WorkBuddy
* feat(gallery): expose the image attribute pipeline as a gallery source
The gallery's contract resolved to a single source, so the filter panel
could only offer "Enabled" while the attributes the observe-and-describe
pipeline fills in — contain.text, contain.data_visual — never reached the
UI. The registry is a gallery attribute source like any other: every
attribute it declares is offered to the gallery, filterable by default,
with the config tiers able to override the usage flags at runtime.
The registration lives in the attribute pipeline's own commit; this branch
carried an earlier, squapshot of that work which predated the split of an
attribute value into a short label and a description, so the registry's
value shape and this file have been brought in line with it. Both PRs now
hold byte-identical copies of the source hook, which is what lets either
one land first and the other merge cleanly afterwards.
Co-authored-by: WorkBuddy
* fix(gallery): viewer race, pin icon, panel anchoring and scope wording
Opening the viewer rendered the img element with the previous or empty
source while the proxy fetch was still producing a blob, and the error
that empty source fired locked the viewer in its failure state even
after the real image arrived. The resolved source now lands before the
flag is consulted, and a token keeps a slow fetch of one image from
overwriting a faster one that navigated past it.
The pin button drew nothing: TDesign has no "pushpin" glyph, so the
button rendered as an empty shell. It now uses pin / pin-filled, keeps
its place beside the tab bar, and a pinned rail assigns its columns
explicitly — the panel precedes the main section in the DOM, so leaving
the grid to source order handed the wide column to the panel and
squeezed the images into the rail.
The floating panel no longer appears at a hardcoded offset: it anchors
just below the arrow that summoned it, the arrow flips up while its
panel is out, a second press retracts, and any click elsewhere in the
toolbar closes it. When a rail is pinned the arrows turn to the
requested tab instead of spawning a second surface.
The scope words (全部 / 全显示 / 自定义) become plain status text rather
than a second dropdown — they name the current mode, and only the arrow
beside them opens the panel that changes it.
Co-authored-by: WorkBuddy
* fix(web): align the OCR-on-unobserved switch with the other setting rows
The switch sat bare inside its setting-row while every other switch on
the page is wrapped in setting-control, which is what reserves the right
column and centers the control vertically. With the row laid out from
the top, the bare switch hugged the row's upper edge and crowded the
description text — visibly out of place next to its neighbors. Wrapping
it in setting-control gives it the same right-aligned, centered slot as
the rest.
Co-authored-by: WorkBuddy
* fix(web): move the OCR-on-unobserved row out of the vertical schema row
The row lived inside setting-row-vertical, whose descendant rule stretches
every setting-info and setting-control beneath it to 100% width. Two
full-width flex children side by side cannot fit, so the control overflowed
the container and the switch landed past the right edge — visibly further
out than the switch on the row above. The row and its note now sit beside
the vertical block instead of inside it, under their own condition, so the
switch shares the ordinary row layout and its right edge lines up with the
"image attribute observation" switch above.
Co-authored-by: WorkBuddy
* fix(gallery): applying a verdict now activates the filter scope
Setting an off/on verdict while the panel's master switch was off
silently ignored it: buildParams only attaches attr_rules in custom
scope, and setVerdict never flipped the scope. A verdict the user just
expressed now switches the scope to custom; clearing the last meaningful
verdict returns to all, since custom with no rules constrains nothing.
* fix(gallery): search checkboxes stay visible and idle panels read as dimmed
The search panel hid its field checkboxes while the master switch was
on, so the only way to reach them was to flip the switch first. They now
sit next to the switch like the filter panel's verdicts do: dimmed in
all-field mode (where they carry no weight) yet clickable, and touching
one drops the mode to custom. The filter panel's verdict rows dim the
same way while the filter switch is off.
* fix(gallery): bulk verdict row, neutral default mark, panel polish
The clear-filters button never matched what the panel is for, so it is
now a bulk verdict row: one set of -/off/on buttons that applies the
chosen position to every value of every value-typed attribute, with the
default mark turned into a language-neutral dash. The panel head drops
its second divider (the tab bar already draws one) and the tabs follow
the toolbar's order: search left, filter right.
* fix(gallery): drop the all-fields caption and center the order arrow
The caption above the search checkboxes showed and hid with the master
switch, shoving the checkboxes down on every flip; it said nothing the
dimmed state doesn't, so it is gone. The order button's arrow is an
inline svg, which sits on the text baseline and reads as riding high;
it now centers against the label.
* fix(gallery): center the order arrow for real
vertical-align never applied: the icon is a flex item inside the
button's inline-flex text wrapper, and the wrapper's default stretch
pins the explicitly-sized 16px icon to the top of the 22px line box.
align-items: center on the wrapper is what actually centers it.
* fix(gallery): shorten panel tab labels in en and ru
The two-word labels overflowed the 300px panel and forced the tab bar
into scroll arrows; single-word labels fit in every locale that was
affected.
* docs: drop the image pipeline rewrite from this branch
The rewritten 'Go 侧接手后' paragraph belongs to the image attribute observation
work (#3635), which this branch carries as a prerequisite and which already
documents the caption_ocr / observation_driven split. Keeping a competing
rewrite of the same paragraph on the gallery branch makes it collide with
upstream, which extended that paragraph with the custom_instructions and inline
HTML table notes. Restoring the shared wording keeps the gallery branch free of
this file so upstream's edits merge cleanly.
* style: gofmt the gallery attribute types
Struct field tags in the resolved gallery config were one column narrow, so
gofmt rebroke the alignment block. Whitespace only.
* style(gallery): use design tokens in the gallery styles
The style guard ratchet fails on any new violation. ImageGallery.vue shipped
with 47 TDesign token fallbacks, 21 literal font sizes, 4 literal radii and a
0.15s transition; all of them now read the --app-* / --td-* tokens, which is
what the guard asks for and what dark mode depends on.
* lint(gallery): fix the golangci-lint findings in the gallery code
lll: the three over-long swagger @Param lines and one registry row were
shortened. ineffassign: ctx was assigned before the access check and then
reassigned, so the first value was dead; it is now declared once, after the
check. errcheck: the three unchecked c.Error calls in ListImages. revive: the
AttrSpec / AttrValue doc comment block was attached to the wrong type and split
into one block each. staticcheck S1016: the gallery registration converts the
registry value instead of restating its fields.
* lint(gallery): wrap the image query select and check the last c.Error
* style: reformat the wrapped image select in chunk.go
* fix(gallery): address review findings on ordering, KB switch and blob lifetime
- sortImageAssets: swap operands for desc instead of negating, keeping a
strict weak ordering so ties page stably
- ImageGallery: reload on knowledgeBaseId change, drop stale list
responses, fetch thumbnails in parallel with one cached blob per URL,
revoke blobs on unmount, default sort to builtin:created_at
- KnowledgeBase: non-wiki KBs with a stale wiki/graph tab fall back to
documents again
- handler: move ListImages godoc out of ListKnowledgeFolders' block and
document attr_filters / attr_rules
- DecideOCR: a payload with no OCR conditions uses the built-in policy
instead of silently skipping OCR
* docs(parsing): describe the caption_ocr and observation_driven image pipelines
Carried over from #3635, which this PR supersedes.
* fix(gallery): load local:// and other storage handles through the KB file proxy
The gallery's own scheme check only caught resource://, provider:// and
storage://, so local:// (the default storage), minio://, cos:// and s3://
images went straight into <img src> and never rendered. Resolve through
isProviderFileURL, the scheme list chat and wiki already use, and lock the
routing with a test.
* fix(kb-editor): keep API-customised OCR conditions on save and upload
The KB editor always wrote the registry default into image_actions.ocr.on,
so any unrelated edit replaced a knowledge base's custom OCR conditions
with the default table; the upload dialog did the same for the per-document
override. Keep a stored custom list and fall back to the registry default
only when there is none, and show the conditions actually in effect instead
of the default table in both panels.
* perf(gallery): de-duplicate, filter, sort and page images in SQL
Every gallery request used to read every image-bearing chunk of the KB,
OCR text included, decode it in Go and page in memory. A single query now
expands the image_info arrays, keeps one row per image URL, applies the
keyword / attribute / verdict / enabled filters and the sort, and returns
only the page with the filtered total.
The expansion carries narrow scalars only; filters and JSON sort keys are
evaluated on the winning entry of each image, and the page's JSON is read
back for its rows alone. Postgres runs the query with JIT off, since the
fixed 100-rows-per-call estimate of jsonb_array_elements otherwise prices
it ~100x high and compilation dominates.
On 20k images with 2 KB of OCR text each (40k chunks), a page drops from
~750 ms to ~100-200 ms on postgres and ~200 ms on sqlite; keyword search
over OCR text stays a full scan (~400-600 ms) until the text is indexed.
The filter semantics are pinned against both databases.
* perf(gallery): serve the gallery from a trigger-maintained chunk_images table
Reading images out of chunks.image_info meant parsing every image chunk of
the knowledge base on each request, so a page still cost time in
proportion to the KB. chunk_images (versioned/000113, sqlite/000032) keeps
one row per image_info entry with caption, OCR text and attributes as
columns; triggers on chunks keep it in step with every write path, and the
migration backfills existing chunks.
Copies of one image stay separate rows and the query lists the copy on the
most recently updated chunk through an indexed anti-join, so writes decide
nothing and concurrent writers need no locking. On postgres, caption and
OCR text get pg_trgm indexes when the extension is available.
On 20k images with 2 KB of OCR text each (40k chunks):
postgres first page 10 ms, page 500 41 ms, keyword 2 ms, filters 23 ms
sqlite first page 3 ms, page 500 39 ms, keyword 147 ms, filters 51 ms
down from ~100-600 ms. Writing those 40k image chunks costs 7.0 s instead
of 1.3 s on postgres (1.9 s instead of 1.0 s on sqlite), mostly the trigram
indexes; status and enable-flag updates patch rows in place.
---------
Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(retrieval): keep index writes and shared-KB lookups correct
Index writes
- Image OCR/caption chunks were indexed without IsEnabled, so every engine
but Postgres stored them disabled and retrieval never returned them.
The SQLite engine re-enables the affected rows at startup.
- FAQ edits, similar-question additions and import merges re-index under
the old SourceID and relied on the engine overwriting it. Postgres and
SQLite kept the stale row (ON CONFLICT DO NOTHING); ES v8 and Qdrant
mint a new document ID and kept both. FAQ re-indexing now deletes the
old rows first, and Postgres/SQLite BatchSave upserts by
(source_id, source_type).
- Postgres: drop the GORM default tag on is_enabled, which replaced false
with true on insert and indexed disabled chunks as enabled.
- SQLite: BatchSave no longer attaches vectors and FTS rows to the wrong
rows when part of a batch already exists, and vecTables is guarded
against concurrent map writes.
- CopyIndices: Postgres filters to the copied knowledge, pages by id and
keeps is_enabled; SQLite copies every row of a chunk (generated
questions included) with stable source IDs and enabled state.
Shared knowledge bases
- Chat merge: FAQ answer population and neighbor expansion look chunks up
without the caller-tenant filter, like parent resolution (#3342).
- Neighbor expansion never cuts the retrieval hit; the length budget goes
to the text adjacent to it.
- FAQ negative questions are applied whenever an FAQ KB is in scope and
to chunks of shared FAQ KBs.
- Search results get image_info for shared-KB chunks.
- search_knowledge reads FAQ metadata from the search result instead of
one tenant-scoped query per hit, so shared FAQs render their answers.
* fix(retrieval): put every retrieval score on one scale
Scores from different retrieval paths were ranked together while living on
different scales, so ordering depended on which path produced a hit.
Fusion
- RRF ranked a chunk by its position in the concatenation of every
vector (or keyword) list, appended in goroutine completion order. The
second list's best hit ranked behind every hit of the first, and the
order changed between identical requests. Each list is now ranked on
its own by score and a chunk takes its best rank.
- RRF is divided by its maximum, so hybrid scores land in [0, 1] like
vector-only (cosine) and keyword-only (rescaled BM25) results. Raw RRF
topped out near 0.016 and lost to any single-retriever hit it was ranked
against across searches (document targets, model groups, FAQ calls);
rerank's base-score term and MMR's relevance term had nothing to use.
- Keyword-only BM25 is rescaled per list; engines' BM25 is not comparable.
- FAQ iterative retrieval fuses each round like the first pass instead of
sorting raw BM25 and cosine together, starts at twice the depth already
searched, triggers when any single vector list is full (the old summed
count never matched with more than one list) and replaces results per
round.
Rerank and chat pipeline
- The FAQ boost is no longer capped at 1; capping made strong FAQs tie at
1.0 and fall to the tie-breaker. The composite score before boosts is
kept as composite_score.
- The FAQ direct-answer threshold is compared against that pre-boost
score, and the FAQ that cleared it is the one marked match="exact"
rather than the first FAQ.
- Graph hits no longer carry a fabricated retrieval score of 1.0, which
ranked them above every scored hit when rerank did not run; the
composite score uses the model score in its place.
Engines
- OpenSearch and Weaviate return (1 + cos) / 2; their drivers now report
cosine similarity and map thresholds onto the engine scale, so a
threshold means the same on every engine.
- Milvus L2 returned squared distance, ranked farthest-first downstream,
and took the threshold as a radius; scores and radius are converted.
- The multi-store normalizer no longer maps Milvus through (s + 1) / 2,
which put it on a different scale from every other engine and from its
own single-store results.
Docs updated for fusion, normalization, FAQ boost and iterative recall.
* perf(retrieval): bound rerank, agent and pipeline cost; stop failing on slow or missing models
Rerank
- Rerankers report the longest document a request can carry
(max_document_chars / max_request_chars minus the query) and the rerank
stage fits its own passages to it, trimming appended captions, OCR and
generated questions first. One chunk with a large screenshot's OCR used
to exceed the vendor limit and fail the whole batch, so every candidate
lost its score and the threshold stopped applying.
- Rerank HTTP requests default to a 60 s timeout; chat and agent contexts
carry no deadline, so a stalled self-hosted rerank server held the turn
open.
- OCR and caption chunks no longer repeat their own text from image_info in
the passage.
- The chat pipeline and the agent tool rerank at most 200 candidates, best
retrieval score first.
- A rerank model that cannot be loaded degrades to retrieval order instead
of failing every chat turn, as the search APIs already do; without rerank
results the merge stage keeps only RerankTopK hits instead of enriching
every one.
Retrieval APIs
- knowledge-search match_count and rerank.top_k are capped at 200
(unbounded values multiplied into tens of thousands of billed rerank
passages); normalizedMatchCount clamps to the pool size so the
over-retrieval arithmetic cannot overflow.
- hybrid-search with rerank recalls at least rerank.top_k deep.
Agent
- search_knowledge fits its rendered output to four fifths of the tool
output budget, dropping the lowest-ranked rows and telling the model how
many it omitted; the model's view is rebuilt from Data, which registry
truncation never reached.
- Agent searches skip neighbor/parent/related enrichment rows, which were
never retrieved yet tripled the rerank input.
- ParallelToolCalls is enabled. The model was always told it may call tools
in parallel while the engine ran them one by one; only read-only tools
overlap, anything else stays a barrier.
Chat pipeline
- Query expansion searches keywords only (its variants are local rewrites
whose embeddings repeat the main search) and is skipped when keyword
matching is off.
- Entity extraction, a second LLM call, is skipped for intents that need no
KB retrieval and uses the rewritten query.
- Partial-overlap dedup tokenizes each text once instead of per pair, and
only compares token overlap between texts within 3x of each other's size.
Storage
- Migration 000112 adds btree indexes on embeddings(knowledge_id) and
(chunk_id); document deletes, chunk re-index and enable/tag updates
scanned the whole table.
- Shared-KB knowledge rows are fetched in one query instead of one per ID.
- Drop GORM Debug() from the ParadeDB keyword query and the knowledge batch
lookup, which logged SQL and user queries on every search.
Docs updated for the API limits, rerank timeout and passage fitting, agent
parallel tools and output budget, and pipeline changes.
* fix(retrieval): make agent tools and pipeline stages report what they did
Agent tools
- query_knowledge_graph now queries the graph store: entity names from the
query are matched in each KB's graph, relations are returned (and shown to
the model), and the entities' source chunks lead the results, with text
search as a supplement. It used to run text search only, so a graph-only
KB always answered "no relevant graph information".
- search_knowledge reports failures instead of "no matching chunks": when
every search fails the tool fails with the reason; partial failures are
listed. A failed query embedding falls back to keyword search for hybrid
KBs, as the chat pipeline does.
- Keyword mode uses rerank to order hits, not to reject them (identifiers
score low), and the empty-result hint no longer sends keyword mode back to
keyword mode.
- Dedup keeps siblings under one parent, no longer merges every image chunk
of a document through chunk_index 0, and keeps the better-scored copy.
- Search rows omit a <match> excerpt that only repeats the content; snippet
terms are segmented for Chinese queries and skip stopwords, and FAQ
matched questions are chosen by match strength.
- read_document query mode returns next_offset when truncated and resumes
from offset.
- wiki_search over-fetches under document/tag scopes before filtering and
says when every match was out of scope.
- Prompts point to read_document(id=cN, context=k) for surrounding text.
- The agent editor hides the retrieval-strategy section in agent mode,
where search_knowledge uses the global retrieval settings.
Chat pipeline
- Intent labels are normalized; an unknown label retrieves instead of
silently skipping the knowledge base, and web_search without web search
enabled searches the knowledge bases.
- Image turns get a 2048-token rewrite budget, and a reply cut off by the
cap is salvaged field by field instead of being discarded.
- History references are matched by how much of the query they cover;
Jaccard against chunk-sized text never reached its threshold.
- Entity search resolves chunks of org-shared KBs within the turn's KB
scope, skips chunks whose document is gone (it used to panic) and caps
graph chunks at 30.
- Empty history is not fetched twice; fetched pages are cut on a rune
boundary.
Retrieval
- A multi-KB search whose primary KB has no vectors embeds the query with a
vector KB's model instead of failing on an empty model ID.
- Rerank passages include the chunk's heading breadcrumb, which is part of
what was embedded.
- SQLite FTS searches the content column only and turns a lone Han
character into a prefix query; TencentVectorDB ef follows TopK.
Docs updated for the agent tools and pipeline stages.
* fix(retrieval): address review findings on the retrieval fixes
- Restore NeedsRetrieval for web_search without web search enabled: the
intent prompt "web_search" is how the user learns web search is off, and
running KB retrieval instead bypassed it.
- query_knowledge_graph keeps the case of entity terms; Neo4j matches node
names with case-sensitive CONTAINS, so lowercased terms never matched
English entities.
- An empty search_knowledge or query_knowledge_graph result states partial
failures in its text, which is all the model sees for an empty result.
- SQLite BatchSave replaces a source in one transaction, so a failed insert
keeps the old rows instead of leaving the source unindexed.
- Reuse stringSliceValue and an exported searchutil.ContainsChinese instead
of new copies; drop an unused named return and the FAQ obsolete-source
delete that the full per-chunk delete now covers.
* fix(retrieval): address second review of the retrieval fixes
- query_knowledge_graph: under a document/tag scope, keep only relations
whose entities come from an in-scope evidence chunk (the graph namespace
is the whole KB); drop chunks whose document no longer exists; report a
graph or text search failure even when the other source returned hits.
- search_knowledge: record the hybrid -> keyword fallback taken when the
query embedding fails in mode_fallbacks, so an empty result is not read
as "no semantic match".
- query understanding salvage: take a cut-off value only for the image
description; a truncated rewrite_query or intent is dropped instead of
replacing the user's query.
- Graph hits (no retrieval score) are kept past the no-rerank RerankTopK
cut and the rerank candidate cap instead of always falling below them.
- FAQ boost is capped at 1 again so scores stay on the [0, 1] scale; FAQs
tied at the cap are ordered by their pre-boost score.
- Negative-question filtering loads only candidates of FAQ knowledge bases.
- read_document query continuation starts one chunk before offset so the
first match keeps its preceding context at a page boundary.
- graphSearchTerms and searchQueryTokens share one queryTerms splitter.
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
MinerU 4.0 removed the synchronous /file_parse endpoint that the mineru
engine relied on, so a self-hosted 4.0 server answered every parse with
404. The reader now probes GET /v1/health before each parse: a 200 selects
the V1 flow, 404/405 keeps the legacy /file_parse path (AMD and domestic
accelerators must stay on mineru<4), and a 503 error envelope surfaces the
server's model-preload failure instead of falling back.
The V1 flow uploads via /v1/uploads (sha256 dedup, API key only sent to
same-origin upload URLs), creates a zip-only parse job, polls with
exponential backoff (2s to 30s), cancels the job on timeout or caller
cancellation, and reads markdown.md plus images/ from the zip artifact.
The zip extraction is shared with the MinerU Cloud reader and now also
resolves <img src> references and percent-encoded paths.
New tenant settings: mineru_server_api_key (redacted in responses,
preserved on update) and mineru_tier (flash/basic/standard/advanced, empty
for the server default). mineru_parse_method maps to V1 ocr_mode. Backend,
vLLM URL, formula/table toggles and language only apply to MinerU 3.x and
are grouped as legacy options in the settings page. The connection check
also verifies the API key on V1 servers, since /v1/health is
unauthenticated.
* feat(models): add system admin model catalog console
System administrators can now maintain the model catalog from
Settings → System administration → Model catalog instead of editing
models.json on every replica. Administrator changes are stored as a
versioned overlay in the database (new model_catalog_config table),
layered between the deployment file and explicit per-model settings,
and synchronised to other instances within about 5 seconds.
Backend
- GET/POST preview/PUT /system/admin/model-catalog with optimistic
version + deployment baseline checks (409 on conflict), 20-version
history and audit logging.
- Console overlays reject credentials, headers, env interpolation and
non-inline icons; those stay deployment-owned.
- Catalog views include each chat model's resolved thinking levels and
the vendor defaults, so the console never re-implements level
inheritance.
Frontend
- Model list in the shared data-table style with provider/type filters,
"modified only" filter and source tags (built-in / deployment / admin).
- Edit drawer saves and applies immediately: display name, context
window, max output, input modalities, reasoning, thinking levels,
embedding dimension and picker visibility, each with its inherited
default and a layer comparison table.
- Add-model drawer, JSON editor with server-side change check before
publishing, and version history with one-step restore.
- Model settings gets a system-admin shortcut to the catalog; model
editor reloads providers when opened so published catalogs apply.
* fix(models): harden the admin model catalog console
- Reject console base_url, base_urls, url_patterns and auth: moving a
vendor's default URL would send its deployment api_key (and keys of rows
without their own base URL) to another host. With every interpolated
field now deployment-owned, '$' in names and descriptions is plain text.
- Drop deployment base URLs from catalog views; they may carry
interpolated tokens.
- Remember a stored version this replica cannot compile and skip it until
the version changes, instead of re-reading and recompiling every poll.
- Resolve vendor default thinking levels with reasoning forced on, like
per-model levels.
- Compute builtin/deployment views once; preview returns only the
candidate catalog and normalized overlay.
- Share deployment file lookup via runtime.ReadDeploymentOverlay.
- The model editor force-refreshes its own type instead of resetting the
provider store for every type; the refreshed list wins in the id index.
* fix(tenant): stop wiki tasks of soft-deleted tenants from issuing model requests
Tenant deletion only soft-deletes the tenant and its memberships; its
knowledge bases, model configs, and durable task_pending_ops rows survive.
Wiki ingest/finalize tasks restored from those ops only checked KB state,
so a deleted tenant kept accruing new LLM requests (observed as ~11h of
continued synthesis calls in #3593).
Gate every task entry point on tenant liveness:
- EnqueueIfKnowledgeBaseActive now also requires tenants.deleted_at IS NULL
inside the same transaction (fail closed).
- Startup recovery purges pending rows whose tenant is soft-deleted (the
predicate is wrapped in parentheses: NOT EXISTS(...) AND EXISTS(...) would
parse as (NOT EXISTS(...)) AND EXISTS(...) and silently match nothing).
- Wiki ingest and finalize handlers check tenant liveness from the payload
before any model call and drop the KB's queue when the tenant is gone.
Liveness lookup errors fail open; the guarded enqueue and recovery paths
enforce the same invariant on their own DB access.
Adds TaskPendingOpsTenantLiveness as an optional extension interface,
regression tests for the enqueue guard, startup recovery, and the task
entry guards.
Fixes#3593
* style(container): wrap long log line over 120 chars for golangci-lint (lll)
---------
Co-authored-by: LuckVd <LuckVd@users.noreply.github.com>
* feat(knowledge): flag documents that stop making progress
A document stuck in processing looked the same as one still working: a
spinner that polled forever, no hint of where it stopped, and, hours
later, a generic housekeeping failure.
- The batch and spans endpoints return last_activity_at for in-flight
rows: the later of updated_at and the newest span write. updated_at
only moves at stage transitions; span writes keep moving while a long
stage still works.
- After 20 minutes with no progress, the list and card views show
"may be stuck" with a tooltip. The timeline shows the stage it
stopped at and a stop-parsing button, plus a "last progress" stat.
Both pollers back off to 15s while everything in flight is stalled.
- When housekeeping fails a row, the error names the stalled stage and
the time of the last progress. It closes the row's open spans with
TASK_STALLED (the running stage fails, the rest are cancelled), so
the timeline shows where the run stopped instead of a spinner.
* fix(knowledge): locate stalls in finalizing and tell backlog from stuck
- Housekeeping reads the stall site from the latest attempt. It takes
the running stages, or, when none runs (post-process closes its stage
once enrichment is fanned out), the innermost running spans, mapped
back to their stage. Those spans fail with TASK_STALLED and are named
in the message; every stalled stage is listed in pipeline order. The
other open spans are cancelled. Every closed span gets a duration, and
the message leaves out the time when the heartbeat query failed.
- The spans tree prefers a TASK_STALLED failure for last_error over a
later subtask failure. current_stage falls back to the stage owning a
running subspan.
- Batch and spans responses flag rows quiet past the stall hint whose
work is still queued (waiting_in_queue), using the sweep's own probes.
At most 20 rows are probed per request, cached for a minute. The UI
shows these as queued, with no stop button in the banner.
* fix(knowledge): answer stall verdicts from one shared queue scan
- Probing a quiet document for a backlog rescanned the whole queue once
per document. When the document was absent, which is exactly the
stuck case, the scan ran to the end, and it read a Redis error as
"nothing queued". QueuedKnowledgeIDs now collects every referenced
knowledge ID in one pass and returns list errors. HousekeepingService
keeps that snapshot for a minute, shared across requests and
single-flight. A failed scan is not cached.
- With the per-request cap of 20 gone, every quiet row on a page gets a
verdict. The response carries it as stall_state (queued or stalled),
replacing waiting_in_queue. The UI marks a row as stuck only on a
"stalled" verdict. A row with no verdict, because the probe failed,
shows as ordinary processing instead of inviting a stop.
- Abort check: a read failure now stops the pipeline instead of carrying
on. Continuing let a later full-row Save overwrite a cancel or delete
it never saw. processChunks now returns an error, and the handlers
return it, so asynq retries an unknown state instead of acking it.
- Resolve the vector store before deleting the old chunks, and retry
when the lookup itself failed. A failed persist or an interrupted
context no longer acks with the row still in flight.
- Drain and release the wiki slots in one transaction, so a failed
release rolls the delete back for the retry. Skip any document that
still has a fresh claim in the lane: its live batch releases it.
- Re-armed and restart-recovered wiki triggers carry the language the
ops were queued with, rather than the server default.
Housekeeping skips any row that still has a durable wiki ingest op, with
no age limit, so a wiki that can never run left its documents in
"finalizing" forever:
- A KB whose wiki was disabled, has no synthesis model, or whose model
was deleted failed the trigger before claiming anything; after the
retries the trigger was archived and the ops stayed. Treat these as
terminal: drain the unclaimed ingest ops and release each document's
wiki slot. Retract ops stay queued; rows a live batch holds are left
to it.
- A lost or archived trigger left the ops with nothing to wake their
consumer until a restart. Housekeeping now re-arms one trigger per KB
for rows held only by durable ops, at most once per stale threshold.
* feat(sandbox): resolve workspace paths through WorkspaceLayout
Keep remote sessions on the /workspace contract, while file, shell, and
artifact code follow the session layout instead of hardcoded paths.
* fix(sandbox): fail closed on host layout lookup and restore the shell blacklist
Treat host vs remote as an explicit origin, refuse inspect outside ReadRoots,
and skip collection when the output tree is missing or equals the project root.
* fix(sandbox): close leftover fail-open holes in the layout path
Keep host copy, collection, and prompt text from falling back to /workspace when a lookup fails or an adapter returns unclean roots.
* fix(sandbox): satisfy golangci-lint on the layout helpers
Wrap new copy and test lines to 120 columns, and add the exported comments revive requires.
* fix(sandbox): gofmt the read_file layout copy wrap
* feat(sandbox): add host as an OS-enforced backend type
Host is not a named/pinnable config: each command is a fresh local process, unlike Docker, Cube, and E2B.
* feat(session): persist Lite host workspace directory
Empty means an auto-allocated folder. Standard edition leaves the column unused so the shared session model still loads. SQLite also gains the tenant skill tables Lite shell_exec already expects.
* feat(localsandbox): adapt the host OS sandbox to session interfaces
File tools stay outside Seatbelt and are bounded by PathGuard. A revoked project directory fails closed instead of allocating a replacement workspace.
* feat(sandbox): run unpinned Lite sessions on the host backend
Lite opts into host explicitly; web stays disabled when no named remote config exists. Host workspaces are not destroyed or auto-committed.
* feat(desktop): pick and persist host project directories
The native folder dialog is the authorization path; typed paths are rejected. settings.sandbox.host tells the client when this edition can use it.
* fix(sandbox): fail closed on Lite host layout lookup misses
Unpinned host sessions were described as /workspace when pin read or
sandbox resolve failed. Prompts and attachment staging now share the
same fallback so Lite never pretends the remote tree exists.
* fix(frontend): keep settings.sandbox.host in sync with the backend keys
TestDeploymentCapabilityKeysMatchFrontend requires the lists to match. Host stays fail-closed like docker so a missed probe does not show Lite-only UI.
* fix(lint): satisfy golangci-lint on the host sandbox wiring
Check c.Error, wrap lines over 120 columns, put context first on versionsWorkspace, and document exported adapter methods.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Org-shared KB viewers got 403 for extracted images on current main even
though the retrieving chunk text carried the resource:// handle and the
share grant was valid (#3342, second report after #3022/#3069).
Root cause is a mismatch between ingestion and authorization: document
parses store extracted images through SaveBytes, which registers the
resource but never claims it, while both file proxies (ResolveKBFile and
the message fallback via resourceAccessibleViaSharedKB) authorize through
IsReferencedByKnowledgeBase — which accepts explicit resource_bindings
rows only. The binding-only hardening landed without adding the missing
bindings, so every extracted image became invisible cross-workspace.
Ingestion: bindStoredImages claims each stored image for its knowledge
(ResourceRelationExtractedImage), mirroring bindContentResources — best
effort, same-workspace resources only. Wired into both the file-parse
path (knowledge_process) and manual knowledge (triggerManualProcessing).
Merge: resolveParentChunks fetched parents, grandparents, and image_info
children with the caller's tenant, so shared-KB hits from the sharing
workspace resolved none of them and lost parent context and image_info
(the second defect in the #3342 audit). Those lookups now use the
existing *Only unscoped variants, matching their shared-KB precedent.
Historical documents parsed before this change still carry no bindings;
a backfill is tracked as the follow-up slice.
Verified end-to-end on a fresh stack: publishing a manual document with
an embedded image now records kind=image plus an extracted_image binding
alongside the content-derived attachment binding.
resolveAskSession stores the session owner as
MCPEndpointPrincipal(...).StorageID() -> "mcp_endpoint:<tenant>:<endpoint>",
but SessionOwnerIDFromContext has no PrincipalMCPEndpoint case and falls
back to the raw user id set by the MCP auth middleware ("mcp-<endpoint>"),
so every ask call failed with
"failed to create user message: session not found".
Handle PrincipalMCPEndpoint the same way as PrincipalEmbedSession.
* fix(sandbox): resolve a session's pinned sandbox in the workspace that owns it
tenant_sandbox_configs is keyed by (tenant_id, id), so a config id is not an
address on its own. sessions.sandbox_config_id stored only the id and every
reader took the workspace from the ambient request context.
That holds inside a chat turn, which already runs under the agent's workspace
(WithExecutionTenant in the QA handler). It breaks for a shared agent
everywhere else, because a shared agent runs on ITS OWNER's sandbox config
while the session belongs to the borrower, and these paths run from plain
requests whose only tenant is the session owner's:
* DeleteSession resolved nothing and returned after a warning. Cube and E2B
create session sandboxes with onTimeout=pause, so the MicroVM was not
reclaimed later either: it stayed paused, billing, with nobody holding its
id. ReapOrphanSandboxes has no production caller, so nothing recovered it.
* The terminal and desktop panels failed on both branches - attaching to the
pinned sandbox and provisioning a new one - with ErrSandboxConfigNotFound,
which terminalErrorFrame maps to a bare INTERNAL close reason. The panel is
offered unconditionally and already forwards agent_source_tenant_id, so
this was a dead end the user could reach in one click.
* Fork read the pin through PinnedSessionSandbox, got no manager, and
silently degraded to SNAPSHOT_UNSUPPORTED, losing the sandbox state.
* The fork snapshot lease recorded the session's workspace next to the
lending workspace's config id, so the reaper could not resolve it either.
Record the owning workspace beside the pin and make the pair travel together
as SandboxPin. Readers now take the workspace from the pin and fall back to the
session's own, which is what every non-borrowed pin (and every row written
before this column) resolves to - so behaviour for own agents is unchanged.
The provision path has no pin yet, so the WebSocket handler passes the agent's
workspace along with its config: terminalProvisionPin already resolved the
shared agent, it just discarded the tenant resolveAgent handed back.
Backfill in the migration is limited to sessions whose pinned config does not
exist in their own workspace - the only ones that can have borrowed it - using
the lending workspace recorded on their newest shared-agent message. Lite has
no shared spaces, so the SQLite mirror adds the column without a backfill.
* fix(sandbox): backfill the pin owner from the config row and self-heal legacy pins
Three gaps in the session sandbox pin owner:
Backfilling from messages.agent_tenant_id names the wrong workspace. The
pin is sticky to the session's FIRST sandbox, while the newest message
carrying agent_tenant_id may belong to a later shared agent from another
workspace. Writing that tenant leaves teardown resolving a config the
workspace does not own - the very failure the column exists to fix.
tenant_sandbox_configs.id is a global UUID primary key, so joining the
config row itself names the workspace that actually owns the pin.
The column was INTEGER while every other tenant id is BIGINT/uint64.
Postgres INTEGER is 32-bit signed, so a large tenant id would not fit.
Rows the SQL backfill cannot reach keep sandbox_config_tenant_id = 0
forever: the reader falls back to the caller's workspace but never
records what it learned. The next chat turn already runs under the
workspace that can resolve the config, so persisting it there is what
lets a later DELETE, panel open or fork find the sandbox. recordOwner
only writes when the lookup actually succeeded and the stored owner is
still unset - stamping the caller's tenant onto a failed lookup would
brand a sticky pin left by a previous shared agent with the wrong
workspace.
Also renames the lentSandboxConfg typo and corrects the fork degrade
reason in a comment: a lent config that resolves to nil fails
BoundSandboxID, so the fork degrades to SANDBOX_GONE, not
SNAPSHOT_UNSUPPORTED.
* fix(sandbox): require a named backend before recording a pin's owner
The self-heal wants to persist an owner "that actually resolved this config",
but a nil error does not prove that. resolveTenantSandboxForConfig returns a
disabled manager with no error twice before it ever reads the config row: for
the workspace kill switch, and for an empty or deployment-default config id.
That reopens the leak this column exists to close. The SQL backfill correctly
leaves sandbox_config_tenant_id at 0 whenever the pinned config belongs to the
session's own workspace, which is most existing rows. One turn of a shared
agent lent by a workspace whose admin has script execution turned off then
short-circuits on the kill switch, returns nil, and stamps that lending
workspace onto a config it does not own. The next DELETE resolves the pin there,
finds nothing, and strands the paused MicroVM.
The pin branch runs before agentConfigID is read, so the shared agent does not
even need a sandbox config of its own to trigger it.
A named backend (Cube/E2B/Docker) is the proof, and it is the same condition
that is already allowed to CREATE a pin further down: what may write a pin may
update whose it is. A config that exists but resolves to type "disabled" is
skipped as well - ownership is genuine there, but such a config never creates a
sandbox, so no teardown can go missing.
The deployment-default pin was inert either way (both workspaces short-circuit
to a manager with no DestroySession), but it is covered too so a later
relaxation of the proof cannot quietly reopen the kill-switch case.
Both tests fail without the guard.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
* fix(queue): order ClaimBatch by fail_count so retries cannot starve new work
ClaimBatch picks work with ORDER BY id ASC (Postgres) / MIN(id) ASC (GORM),
i.e. oldest-first by primary key.
Retry does not move a row: requeueFailedOps calls IncrFailCount and then
ReleaseByIDs, which deliberately keeps the row in place so the fail_count
budget keeps counting down.
Together those two produce a starvation mode. A document that fails
repeatedly keeps its original, lowest id, so it is always selected first,
while every never-attempted document - enqueued later, hence a higher id -
waits behind it until the stuck op finally exhausts wikiMaxFailRetries and
dead-letters. Observed on a real 87-document KB: 42 ops pending, the four
head ops were all re-runs of documents that already had a root page, and 38
never-attempted documents were never claimed. With 8 concurrent workers and
a drained asynq queue (pending=0, retry=0, active=4), the bottleneck was the
claim ordering, not throughput.
Order by least-failed first, oldest-first within the same fail_count. When
every row has fail_count = 0 this degenerates exactly to the previous
id-ordering, so the existing FIFO tests still pass.
Test: TestTaskPendingOps_ClaimBatch_PrefersLeastFailed replays the real retry
path (claim -> IncrFailCount -> ReleaseByIDs) and asserts that the
never-attempted document is claimed first and the retried one is still
eventually picked up rather than stranded.
* fix(queue): apply the same least-failed PeekBatch order in Lite mode
Lite wiki ingest still peeks rather than claims. Ordering that path by
id alone would keep the same head-of-line starvation this PR already
fixed for ClaimBatch. Align comments with the new selection rule.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* refactor(session): extract shared workspace git reset helper
Fork bootstrap and in-place rewind both need the same /workspace reset
script. Move it out of fork_bootstrapper so rewind can reuse it without
copying ExecShellCommand plumbing.
* feat(message): add DeleteMessagesFrom for rewind truncation
Rewind needs a composite created_at+id cursor so same-millisecond
messages after the fork point are not left behind. Delete inside a
transaction and return the rows so callers can log the count.
* feat(session): add in-place rewind service
Truncate the current session at a user or assistant message and git-reset
the live sandbox when a reachable checkpoint exists. Reset failure is
fail-closed so messages are never deleted ahead of the workspace.
* feat(session): expose POST /sessions/:id/rewind
Wire SessionRewindService through the session handler and container so
the client can rewind in place. Busy sources map to 409; git reset
failures stay 500.
* feat(chat): add rewind API client and i18n copy
Expose rewindSession with the same 180s timeout as fork, and add
chat.rewind strings in all five locales so the confirm dialog is not
hardcoded Chinese.
* feat(chat): add rewind button with confirm dialog
Offer in-place rewind next to fork on user and assistant messages.
Confirm before truncating the thread, then reset locally and prefill
the original question when the cut point is a user message.
* fix(chat): confirm rewind with a compact popover
Replace the full-page confirm dialog with a small popconfirm above the
rewind button, and shorten the swagger description to satisfy line length.
* fix(chat): skip markdown images with an empty destination
Empty ![]() was rendered as <img src="">, which shows as a broken image in the reply.
* fix(chat): clip sandbox panel slide-in and rename open tooltip
Keep the enter animation inside the viewport so it cannot flash a document
scrollbar and jump the page. Rename the header toggle from sandbox terminal
to expand sidebar.
* fix(session): 回滚时完成截断、互斥发送并重拉历史
客户端中断不再让 git reset 成功后留下未删的消息;reset/删消息前复查 busy,并锁住当前会话输入。成功后重拉历史,避免切会话预填串台和 steer 分段截错。
* fix(session): 工作区 git 外置,并让未开箱 fork 的回滚对齐 SHA
checkpoint、回滚和 fork 共用 --git-dir=/var/lib/weknora/workspace.git,避免 rm -rf /workspace 丢掉历史。
未开箱分叉会话回滚时改写或清除 ForkBootstrap,避免第一次开箱仍 reset 到旧切点。
* fix(session): 回滚加会话锁,并堵住发送中回滚与脏 SHA
同一会话并发回滚互斥(进程内 + Redis),冲突返回 409。发送/流式中禁用回滚按钮;历史重拉成功后再改界面。未开箱 fork 的 checkpoint SHA 必须是 40 位 hex。工作区 git 只认外置 GIT_DIR,不再 mv/rm 根上的 .git。
* fix(session): 未开箱 fork 回滚成功时不要提示没有沙箱
ForkBootstrap 已对齐切点后返回 workspace_reset,前端走「已回滚」;清掉引导时改报 NO_CHECKPOINT。纯聊天仍用 NO_SANDBOX。
* fix(chat): stub composerLocked in steer interaction tests
handleSteerMsg now bails when rewind locks the composer. The VM harness
did not provide that ref, so CI threw ReferenceError on every steer case.
* fix(session): allow git-layout mkdir in pinned checkpoint test
SkipWorkspacePrep still omits the input/output bootstrap exec. The
checkpoint script now mkdirs WORK_TREE and GIT_DIR, so the old
"command must not contain mkdir" assertion was stale.
* fix(session): block send while rewind holds the session lock
BeginTurn now fails if the rewind key is held, so AgentQA cannot start
during git reset. Knowledge-chat and executeQA check the same lock
before persisting messages. composerLocked is still page-local.
* fix(session): close send/rewind races and fail closed without a checkpoint
Hold the turn lease before persist so rewind cannot truncate an in-flight
send. TryLockRewind fails while a turn is live; emptying the conversation
resets the workspace through the same git path as a SHA rewind.
* fix(session): wrap rewind reset call to satisfy line length lint
* fix(session): fail closed on a replaced sandbox and close leftover rewind holes
Do not truncate history when the live workspace cannot follow the checkpoint, retry deletes after a successful git reset, and keep send and rewind exclusive even without a sandbox manager.
* fix(session): address rewind review findings
Correctness:
- Adopt a pre-move /workspace/.git into SessionGitDir on first use. Sandboxes
provisioned before the git dir moved out of the work tree still hold their
checkpoints there, so every SHA recorded on those turns stopped resolving:
rewind 500'd instead of truncating and fork's rollback failed on a bad
object. Adoption is guarded on the new dir being absent and on the work-tree
repo carrying the checkpointer's committer identity, so an agent's own
`git init` in /workspace is left alone.
- Only fail a send closed when BeginSessionTurn reports a rewind lock. Every
other lease error degrades to running without a lease, as before rewind
existed; a Redis blip on the lease script was rejecting the user's message.
- Release an abandoned fork snapshot after the truncate lands, not before.
Clearing the bootstrap column is a DB write a retry can redo; deleting the
snapshot is not, and a delete that exhausted its retries left the fork with
no workspace to boot from under a full transcript.
- Re-acquire the rewind lock when a renew call errors, not just when it
reports the lease lost. Sitting the tick out let a short Redis outage expire
the 2m key mid-reset and hand the session to a second rewind.
- Keep the history cursor when a post-rewind reload fails. Clearing it sent
the next scroll-up back to the newest page, which the kept prefix already
holds, instead of to older messages.
Efficiency:
- Add ListAssistantCheckpointsUpTo and use it to answer "which checkpoint does
kept history still reach". Rewind was reading every message body and artifact
row in the session to find one SHA.
- Take the send-side turn lease once per request. The handler already holds it
before persisting; QARequest.TurnLeaseHeld lets the QA services skip their
own, so IM/MCP still take one and HTTP send stops doubling every rewind check
and lease round trip.
- Batch the suggestion delete behind DeleteByMessageIDs instead of one query
per deleted message while the rewind lock is held.
- Resolve the rewind affordance for the whole transcript in one pass rather
than two list scans per rendered row on every streamed token.
Cleanup:
- Drop the SANDBOX_REPLACED skip reason, which the service never produces (it
fail-closes with a 409), along with its copy in all five locales.
- Drop GetMessagesBySession from rewindMessageStore; rewind never calls it.
- Drop the unused chat.rewind.confirmTitle copy from all five locales.
- Restore the chatHeader.toggleSandboxPanel wording this branch had changed
unrelated to rewind.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(agent): end the turn when the completion cap truncates an answer
A round that came back with finish_reason=length and no tool calls used to
fall through analyzeResponse as non-terminal. With no tool calls to run,
appendToolResults pushed the half-written answer back into the history as a
plain assistant message with nothing telling the model to continue, so the
next round rewrote the answer from the top, hit the same cap, and repeated
until the round budget ran out or the user cancelled.
Three guards, each independently load-bearing:
- analyzeResponse gains a truncation case: deliver the partial text and end
the turn. A cap hit that produced no text at all has nothing to hand over
and defers to the existing bounded empty-content retry. No continuation
nudge is sent — it only grows the prompt with the discarded fragment, and
assistant prefill, the reliable form, is unavailable on most
OpenAI-compatible endpoints.
- A consecutive-length breaker stops the loop when truncation keeps landing
inside tool-call arguments, the one shape that still legitimately loops.
- The stuck-loop detector now counts an identical lack of content. A model
burning its whole budget on reasoning returns empty every round, which used
to land in the else branch and reset the counter, so the guard could never
fire on the case that needed it most.
The answer event and its Done marker carry a truncated flag through the SSE
metadata, and the chat UI shows the same style of notice it already uses for
fallback answers.
Fixes#3446
* fix(agent): treat unterminated reasoning as no answer and keep the truncation marker
Review follow-up on three ways the truncation path could still mislead.
An inline <think> block that the cap cut off never gets its closing tag, so
the pair regex left it whole and the round looked like it had produced an
answer. The truncation case then skipped the empty-content retry it exists
for and emitted the chain of thought as the final answer. StripThinkBlocks
now drops a dangling <think> through end of input, which is what the
streaming splitter already does with the same text, so the two agree on what
counts as an answer.
The truncation marker only rode on the SSE metadata, and a finished
conversation is rebuilt from agent_steps rather than replayed from stream
events. A reloaded, shared or re-opened turn therefore lost the notice and
its half-written answer looked complete. AgentStep carries the flag now — it
is a jsonb column, so no migration — and the frontend marks the rebuilt
answer event and message from it.
The consecutive-length guard ran before the round was recorded, so a final
round of "let me look that up" plus a truncated tool call handed that
preamble over as the final answer, with its tool calls never refused and the
preamble never retracted from the answer area. The guard now records the
round and fails its tool calls the way every other truncated round does, and
a response carrying tool calls never donates its text as the answer.
* fix(security): close the low-severity shared-space findings
Verified and fixed from the shared-space permission review:
- Embed channels kept knowledge_ids, tag_ids, mentioned_items, skill_names
and summary_model_id from anonymous visitors. Mentions are only scoped
for shared agents, so a visitor could query any KB (and pick any model)
of the channel workspace by ID. The payload now drops them; the widget
never sends them.
- PUT /knowledgebase/:kb_id/wiki/issues/:issue_id/status updated any issue
by ID. The update is now scoped to the path's KB (404 otherwise).
- A direct org invite let the inviter attach any user of the target
workspace (or any user by ID) as its representative, whose username and
email the roster then showed to all members. Direct adds attach no
representative, and the roster returns emails only for the caller's own
workspace.
- COS and KS3 dropped the bucket (and COS the region) segment when turning
a path into an object key, while the tenant check scans every segment:
cos://42/<region>/prefix/7/... passed as tenant 42 and read tenant 7's
object from a shared bucket. Both must now match the service's config.
- Shared agent listings returned the full agent to receivers. Prompts and
the creator's user ID are withheld; capabilities and scope stay visible.
- Temporary document uploads let the caller pick the parser engine (and
its credentials) of a shared agent's workspace; only the agent's rules
choose one there now.
- Suggested questions used request / message-snapshot knowledge_ids
without a permission check. Each document is now resolved to its KB and
checked like KB IDs, including a shared agent's scope.
- IM channels accepted any knowledge_base_id (and any agent ID on create).
Both must belong to the channel's workspace, the KB also to a restricted
API key's allow-list.
- KB-restricted API keys could list org-shared KBs outside their allow-list
in the organization views; those listings are now filtered.
- With enable_rbac=false, RequireKBAccess let cross-workspace denials
through. That switch is for roles inside a workspace; the guard now
always denies another workspace's KB, since handlers behind it load by
ID without a tenant filter.
- The org member limit was counted before and enforced apart from the
insert; it is now checked in the insert's transaction with the
organization row locked (SQLite serializes writers instead).
- The UpdateOrganization route comment and API doc said owner-only; the
service and UI allow org admins. Comment and doc corrected.
Documented, not changed: issued /r/ and presigned links stay valid until
they expire after a share is revoked.
* fix(security): address review of the low-severity shared-space fixes
- Embed payloads also drop question_origin. It selects a KB like an
@mention for agents that retrieve only when mentioned, and the widget
never sends it; the selection stays inside the agent's scope, but an
anonymous visitor should not pick among its KBs. suggestion_attribution is
kept (the widget sends it and it is validated).
- The receiver view of a shared agent also withholds the follow-up
suggestion instruction, a prompt like the others. Starter questions are
shown to receivers anyway, and the sharer's name is displayed by design.
- Follow-up suggestion generation moves into the agent's workspace with
types.WithExecutionTenant instead of a bare tenant rewrite, so the
caller is pinned before the document permission check runs. The auth
middleware and the shared run already pinned it; this removes the
dependency on that.
- KS3 CopyFile copies only from the service's own bucket (as COS and S3
do); another bucket's path is a cross-backend copy.
Tests cover the dropped question_origin, the follow-up instruction and the
/shared-agents listing, a follow-up context with the tenant switched, a
repeated wiki issue status update, and the KS3 copy source.
* feat(agent): size agent history by the window and calibrate token estimates
Agent history was capped at history_turns (default 5) because compaction
could not survive a turn. With checkpoints persisted, load history by
tokens instead, and let compaction bound the session.
Budget. The loader drops the oldest turns that do not fit, and what it
drops is lost: neither replayed nor summarized. A budget equal to the
compaction threshold (the earlier attempt, since reverted) let the loader
trim first, so whenever one turn was larger than the system prompt plus
the new question the request never crossed the threshold; nothing was
summarized and the session became a sliding window with no checkpoint.
agent.HistoryTokenBudget is now the whole window: overflow reaches the
first round's compaction, which summarizes it into a new checkpoint.
A turn-by-turn regression test drives the real loader, budget and
compactor and fails as soon as a stored turn after the checkpoint is
missing from the loaded history; restoring the old budget fails it.
Summarizer input. History can now reach the window, so the part being
summarized can outgrow one request. An overflowing request degrades to a
raw archive that is never persisted, so every later turn would fail the
same way. Prepare keeps the newest messages that fit, the prompt states
how many were left out, and file paths still come from all of them.
Calibration. cl100k counts Chinese at about a token a character where
Qwen and DeepSeek spend about 0.6. The estimator now carries a scale
(clamped to 0.5-1.5, text only), learned from consecutive requests in a
turn: they share tools, system prompt and earlier messages, so the
provider's growth between them prices exactly the appended messages,
without mixing in how a provider renders tool schemas. The turn's scale
is stored on its usage (context_token_scale); the next turn's loader
prices history with it and hands it to the engine. Loading, the first
compaction check and compaction itself share one estimator, so they
agree; calibrating only the trigger would let the loader drop turns
again, which the regression test covers.
Also: history is priced as the engine sends it (redacted KB/Wiki results
cost one line); a checkpoint older than the rows read is matched by the
assistant rows' (created_at, id) order; a read ending on a turn's
question keeps that turn; a compaction discarded for freeing under 5%
still persists its checkpoint; a checkpoint write matching no row is an
error; a partial index backs the newest-checkpoint lookup. Agent mode no
longer reads history_turns: the field leaves AgentConfig, the editor
hides it in smart-reasoning mode, and quick-answer keeps its turn cap.
* fix(agent): address review of window-sized history loading
Summarization. The summarizer call had a fixed 60s total timeout. With
history loaded up to the window, a request could carry ~175k tokens and be
cut off while prefilling or writing normally; the degraded result left no
checkpoint, so the next turn reloaded the same history and failed the same
way. Summarization now streams and is cancelled only after no output for the
engine's stall timeout (120s by default, LLMCallTimeout overrides), with the
overall ceiling left to the transport, as for the engine's own rounds and as
in Codex and Pi. A stopped turn is not retried. When the summarizer still
fails, the raw archive is persisted as a checkpoint marked degraded so the
loop is broken, and the next compaction refines it as the previous summary.
The archive is held to the summary budget, newest messages first, so a
degraded compaction always frees room.
Loading. The loader priced turns as sent but returned and held them as
stored, so wiki pages stored in full and sent as one line could pile up in
memory. It now returns the as-sent history and releases each turn's stored
rows once replayed, keeping about a page of rows at a time.
Index. The partial checkpoint index is replaced by a composite
(session_id, created_at DESC, id DESC) index serving the backwards page,
the newest-checkpoint lookup and the recent-messages reads. It is built with
CREATE INDEX CONCURRENTLY so writes to messages are not blocked; the
migration holds that single statement, since a multi-statement query runs as
an implicit transaction. A PostgreSQL integration test, skipped without
WEKNORA_MIGRATION_TEST_POSTGRES_DSN, applies the chain, runs 000106 down and
up through golang-migrate, and checks both queries use the index without a
sort; the SQLite migration test checks the same plans.
Calibration. It no longer waits for a total token count, and a usage with
only a scale is persisted. A turn that measures nothing carries forward the
scale it started from, so the newest turn always has the latest one.