* feat(models): expose embedding input modalities
Whether an embedding model maps images into the same space as text is
what will decide if a knowledge base can store image vectors. The
catalog already had the field, but most multimodal embedding models
were missing it and embedding rows never reported it.
- Catalog: declare input modalities for the Doubao and Tongyi vision
embeddings, the Aliyun multimodal / qwen-vl embedding families and
jina-embeddings-v4, as listed in each vendor's documentation. The
migration baseline changes for exactly these nine entries.
- Model responses resolve capabilities for embedding rows too, so
capabilities.input reflects the catalog merged with spec.input.
- Model editor: an "Image input" switch for embedding models. It follows
the catalog and writes spec.input only when the row differs from it;
picking another catalog model or switching vendor drops the
declaration. Model cards show the image badge for such models.
* feat(embedding): embed images with multimodal embedding models
Adds the image side of an embedder, for models that map images into the
same space as their text vectors so a text query can find an image.
- api: EmbedImage (bytes + MIME type, sent inline as base64 because the
object store is usually unreachable from the vendor) and an
ImageEmbedder implemented by all four protocols, each in the image
shape its reference documents: Ark image_url, DashScope contents[].image,
Gemini inlineData parts, and on the OpenAI shape an object keyed by the
vendor's image_field (Jina: "image"). A plain OpenAI-compatible
endpoint names no image field and takes no images.
- Settings: image_field, max_image_batch_size (default one per request),
max_image_bytes and image_mime_types, filled from vendor docs for the
DashScope multimodal models, gemini-embedding-2 and Jina.
- embedding: an optional ImageEmbedder interface reached through
AsImageEmbedder, so existing Embedder implementations and fakes are
untouched. An embedder takes images only when the catalog (or the
row's spec.input) lists image AND the endpoint can carry one. Images
are checked against the documented limits before any request, then
split at the documented images per request. The concurrency, debug
and Langfuse decorators forward it; logs and traces carry the MIME
type and size, never the bytes.
The migration baseline changes only for the entries whose resolved
embedding settings now carry image limits.
* feat(knowledge): index images by their own vector and recall them
When the knowledge base's embedding model embeds images, each image now
gets a vector of its own next to its caption and OCR chunks, so a query
can find an image by what it shows even where the caption left it out.
Ingestion
- After the caption/OCR chunks are indexed, the multimodal task embeds
the image (embedding.PrepareImage converts and shrinks it to the
vendor's documented limits) and stores it as a new image_vector chunk:
Content is the caption (OCR text when there is none) for reranking and
the answer context; its index row carries the image's vector with
SourceType ImageSourceType. Scanned PDF pages are skipped, their OCR
text serves better. A failure here is logged and never fails the task.
- The image vector gets its own chunk rather than a second index row on
the caption chunk: OpenSearch and Weaviate key documents by ChunkID,
so a second row would overwrite the caption's text vector, and copy,
move and enable/disable all follow chunk IDs.
- CompositeRetrieveEngine.BatchIndexVectors writes precomputed vectors
through each engine's normal BatchIndex, keeping row order (the
composite's de-duplication does not).
- Paths that rebuild vectors from text leave image_vector alone:
syncChunkIndex only syncs its enabled flag, updateChunkVector skips
it, and a caption edit updates its Content only.
- The multimodal task's caption/OCR index rows now set IsEnabled; only
backends with a column default were storing them as enabled.
Retrieval
- A text query scores lower against an image than against text, so when
the model embeds images the document vector search is widened by half
(capped at 500) and run at min(VectorThreshold, 0.1); each store
group's hits are then held to their own threshold before score
normalization: text at VectorThreshold, images at min(VectorThreshold,
0.1). Keyword hits on image rows are dropped, their Content being the
already-indexed caption.
- image_vector hits are searchable and merge like caption hits: parent
context is attached, and the caption is not repeated from ImageInfo.
Adds golang.org/x/image for scaling and WebP/BMP decoding.
* fix(models): report no thinking levels for embedding, rerank and ASR models
Embedding rows now carry capabilities, and the end-to-end run showed them
listing off/auto/minimal/low/medium/high: these references resolve no chat
protocol, and an empty level map reads as "every level supported". A model
without a chat protocol does not think, so its capabilities say so.
* fix(compaction): never read a cancelled turn's stream as a summary
streamSummary selects on the turn's context and the provider stream at
once. When the turn is already cancelled both are ready (providers close
the stream once their context ends), and select picks at random, so a
stopped turn could come back as a completed, partial summary with a nil
error. TestSummarizationIsNotRetriedOnceTheTurnIsCancelled caught it
intermittently in CI and reproduces under -race with several -cpu values.
Check the context before each receive and again when the stream closes.
* feat(knowledge): put image vectors behind a knowledge-base switch, off by default
Image vector indexing and recall used to turn on whenever the knowledge
base's embedding model declared image input. A model that takes images is
often chosen for text alone, so an upgrade would have added an embedding
call per image, widened vector recall 1.5x and lowered the image threshold
to 0.1 for those knowledge bases without anyone asking.
ImageProcessingConfig gains image_vector_enabled (default false; stored
JSON without the key reads as off, so existing knowledge bases stay off
with no migration). Both sides now need the switch and the model:
- indexImageVector returns before touching the model unless the KB opted
in (KnowledgeBase.IsImageVectorEnabled, which also needs vector indexing).
- applyImageRecall resolves the embedding model only when some KB in the
search opted in, and records the opted-in KBs per store group
(storeGroup.ImageKBIDs). Only those groups widen the pool and lower the
threshold; image hits of any other KB, including ones indexed before the
switch was turned off, are dropped.
The KB editor shows the switch in the image processing section, with a
warning when the selected embedding model does not declare image input;
docs describe the switch, its cost and what turning it off does.
* fix(knowledge): leave image vectors behind when cloning into a KB that is off
CloneChunk copied every chunk of the source document and then CopyIndices
for all of them, so copying a document from a knowledge base with image
vectors on into one with the switch off (same model and store) still
created image_vector chunks and their index rows, bypassing the switch
that indexImageVector honours.
Image vector chunks are now dropped from the copy when the target has not
opted in. Every engine's CopyIndices copies only the rows of chunks in the
mapping, so their index rows stay behind too.
* fix(search): refill the vector pool past stale image vectors
Image hits of a knowledge base whose switch is off were dropped only after
the engine had ranked and cut its results at TopK. Image vectors indexed
before the switch was turned off could fill that pool and push out valid
text hits: with MatchCount 5 the pool is 50, and 50 stale image rows
ranked above the text left the search empty.
No engine can filter rows by source type (Postgres and SQLite do not even
take ExcludeChunkIDs), so the exclusion cannot move into the query. When
the document vector pool comes back full and holds stale image rows, that
search alone runs again on a pool twice as large, until it holds TopK rows
that are not stale, the index runs out, or the pool reaches
maxRetrievalPoolSize (at most four extra calls from a pool of 50). The pool
is then cut back to TopK such rows, matching an engine-side exclusion
unless more than maxRetrievalPoolSize-TopK stale rows rank above the text.
* feat(i18n): translate the image vector switch and image input keys into zh-TW
The zh-TW locale arrived from main after this branch added five keys: the
knowledge base's image vector switch (label, description, unsupported-model
warning) and the embedding model's image input flag (label, description).
* fix(knowledge): leave image vectors behind when moving into a KB that is off
A reuse_vectors move relocates a document's chunks and index rows in place:
MoveKnowledgeIndices rebinds the rows by document ID and
MoveChunksByKnowledgeID rebinds the chunks. Moving a document from a
knowledge base with image vectors on into one with the switch off therefore
carried its image_vector chunks and their vectors along, the same bypass the
clone path had.
When the target has not opted in, the image_vector chunks are now deleted
during the move, their index rows first and then the chunks, so a retry
finds the chunks again rather than leaving orphaned rows. A target that
opted in still receives them unchanged.
* fix(retriever): keep image vectors out of keyword-only engines
BatchIndexVectors handed image rows to every engine of the composite,
including one that serves keyword retrieval alone (Elasticsearch v7 next to
a vector store). There the row holds nothing but the image's caption, which
the caption's own chunk already indexes, and retrieval discards every
keyword hit on it.
Image rows now go only to engines that serve vector retrieval. An engine
that serves both keeps one row for both kinds, so its keyword search still
meets them; the search side handles that.
* fix(search): refill the keyword pool past image rows
Every engine that serves both kinds of retrieval stores one row per index
entry and runs its keyword search over that row's content, so an image
vector row, whose content is its caption, is a keyword hit wherever the
caption's own chunk is. filterImageHits drops all of them, but only after
the engine has cut its ranking at TopK, so they crowded out text and caption
hits; a keyword ranking that put the image rows first came back empty.
The keyword pool is now refilled the way the vector pool is refilled past
stale image vectors: when it came back full and held image rows, the
keyword search runs again on a pool twice as large, up to
maxRetrievalPoolSize, and is cut back to TopK hits that are kept.
Leaving the rows out of the keyword index instead would mean writing them
without searchable content (Milvus and Tencent VectorDB derive a BM25
sparse vector from it, and how they take an empty one is unverified) or a
source-type filter in every engine's keyword query (ten engines, and
filterable fields that existing collections lack).
* revert: let image vectors move with the document into a KB that is off
This reverts commit e259977fe915f7d0bc30c4b02bc7af04da1c8dd4.
Deleting the image vector index rows by chunk ID before the move is not
reliable: the Weaviate and Elasticsearch v8 delete wrappers ignore partial
failures, and OpenSearch rejects more than 1000 IDs with ErrBatchTooLarge.
A partial delete followed by the chunk delete leaves index rows whose chunks
are gone, which nothing ever cleans up.
Without it, the image_vector chunks and their index rows move together
with the document. While the target's switch is off they are filtered at
query time (the pool refill keeps them from crowding out text), they are
removed with the document on delete, reparse or knowledge base delete, both
chunks and index rows by knowledge ID, and they become active again if the
target turns the switch on. The clone path keeps dropping them, since there
the rows are copied rather than moved.
* fix(search): keep the image row refill from losing hits it already had
refillPool replaced a pool with the answer to each larger search. A refill
that answered only in part (RetrieveResult.Error, swallowed into a warning
by retainPartialResults, as when some collections of the store did not
answer) could hold fewer text rows than the complete smaller pool, or none:
40 stale image rows above 10 text came back from a partial refill as the
40 image rows alone, and the search lost all 10 text hits. A refill that
failed outright failed the whole store group, discarding a first round that
had succeeded.
A refill is now best effort. A failed or incomplete refill search stops
the refill and leaves the last complete pool in place, cut back to TopK
kept rows as before, so a refill can no longer leave a search worse off
than it was without it. Merging the partial answer into the pool was not
done: its rows would need deduplicating and its ranking is cut at a
different TopK over a different set of collections, while keeping the
previous pool is exact.
* docs(knowledge): say what happens to image vectors in a KB that is off and what the refill costs
The image vector section now says that with the switch off, image vectors
that were indexed before it was turned off, or that came along when the
document was moved in from a KB that has it on, are kept but filtered at
query time, are removed when the document is deleted or reparsed or the
KB is deleted, and are recalled again once the switch is turned on; a copy
into such a KB leaves them behind.
It also states the cost of keeping dropped image rows from crowding text
out of the pools: the vector and keyword pools are searched again with
TopK doubled up to the 500 candidate cap, which adds queries for store
groups that hold such rows, and text can still be missed when more than
about 500 candidates are image rows. A failed or partial refill keeps the
previous complete pool.
* fix(search): log unexpected refill result sets, and describe the refill stop condition precisely
#3956 (Hugging Face TEI) and #3957 (Pinecone) added two rerank-only
vendors, but the READMEs, the models page, the extension-points page and
the homepage still said 27, and the vendor table had no rows for them.
The v0.8.2 release notes keep their historical 27.
The rerank limits row said every limit splits the request automatically.
Only max_documents and max_request_chars do; an over-long query or
document (max_query_chars / max_document_chars, or a single document
that does not fit the request budget beside the query) fails the call.
Taken from the documentation part of #3757; max_documents keeps
0 = unlimited.
Co-authored-by: linus-liu-web <11640009+linus-liu-web@users.noreply.github.com>
* fix(datasource): omit last_sync_cursor from the data source list response
The settings page polls GET /datasource every few seconds during a sync,
and connector cursors can grow to megabytes. No list consumer reads the
cursor, so NewDataSourceResponses now blanks it; detail, create and
update responses still return it through NewDataSourceResponse.
* feat(datasource): add Seafile connector with fingerprint-based incremental sync
Sync folders and files from one Seafile library per data source. The
connector walks the selected scope one directory level at a time,
downloads files through the link + fileserver flow without credentials,
and reconciles deletions against a per-file oid/mtime/size fingerprint
cursor that is checkpointed every 50 mutations.
- resource ids are "<repo_id>:<path>"; ancestors collapse, one library
- per-file failures become coded error items and keep the path pending
- directory failures and Retry-After > 60s abort without deletions
- forced full runs keep the old cursor as a deletion baseline
- API token never reaches cursors, metadata, logs or error text
- share the supported import extension set through internal/utils
* feat(datasource): add Seafile frontend connector and docs
Wire the Seafile data source into the editor dialog, channel labels
and the five locales, and document how to connect a Seafile 10.0.1
library.
- connector definition with base_url + API token fields and icon
- picker guard: one library per data source, at least one selection
- "library" resource type and icon in the lazy tree
- channel label in the knowledge base filter, document views and chat
- i18n for labels, guide steps and the eight seafile_* sync error codes
- website-docs integration guide, data source and extension point doc
updates, and Seafile in the README data source lists
---------
Co-authored-by: zaqqmm <q-zhang10@qq.com>
* fix(kb): let update API set vlm_config
UpdateKnowledgeBaseRequest only carried name/description/config, so the
PUT /knowledge-bases/:id endpoint could never change a knowledge base's
multimodal (VLM) config, even though CreateKnowledgeBase accepts a
top-level vlm_config. Thread an optional vlm_config through the handler
and service so the update path mirrors create (nil = no change).
Fixes#1723
* test(kb): pass the vlm_config argument in tests added on main
* fix(kb): accept only model-managed vlm_config fields on update
The update API copied the request's VLMConfig wholesale, so a caller
could set the legacy ModelName/BaseURL/APIKey/InterfaceType fields and
have the VLM client call BaseURL directly, outside model management and
its SSRF checks. Only Enabled/ModelID/DescriptionLanguage/
CustomInstructions are taken now; legacy fields keep their stored value.
A referenced model must exist and be a VLLM model (400 otherwise), and
disabling clears ModelID, matching the initialization config endpoint.
* fix(kb): validate and gate vlm_config on the update API
- Enforce the 4000-character custom_instructions cap
(types.ValidateKnowledgeBasePromptInstructions), as create and
/initialization/config already do.
- vlm_config is a KB setting: require kbSettingsAccess (owner workspace
or admin share) like /initialization/config, so share editors get 403.
- Stored legacy model_name/base_url keep IsEnabled() true on their own,
so disabling returned 200 but left VLM on, and managed -> disabled fell
back to the old endpoint. Clear them when the request enables a managed
model_id, or disables VLM without echoing back the stored
model_name+base_url (GET-then-PUT keeps legacy on). Request legacy
fields are only compared, never written. Enabling without model_id on a
non-legacy KB is now a 400 instead of a silent no-op.
- Document the vlm_config semantics in the KB API reference.
* fix(kb): keep the legacy VLM endpoint only for pure legacy knowledge bases
CreateKnowledgeBase stores vlm_config as sent, so a managed model_id can
sit next to legacy model_name/base_url; resolveVLM then uses the managed
model. The update's keep-legacy paths (disable echoing the stored legacy
fields, enabled=true without model_id) cleared model_id but kept the
legacy fields, so IsEnabled() stayed true and image calls moved to the
stored base_url.
Those paths now apply only when no model_id is stored. On a mixed config
a disable clears everything, and enabled=true without model_id is a 400,
the same as on any managed knowledge base.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(data-analysis): skip style-only worksheets before DuckDB loading
* chore: retrigger CI for XLSX sheet fix
* fix(data-analysis): keep all sheets when the worksheet preflight fails
The style-only sheet preflight is best-effort: if it cannot read the
workbook parts, log a warning and load the sheets DuckDB enumerated
instead of failing the whole workbook. Context cancellation still
aborts the load.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(server): signal shutdown raced Fatalf exit before ResourceCleaner ran
The signal goroutine closed the listener before calling server.Shutdown,
so Serve returned the raw accept error ('use of closed network connection')
instead of ErrServerClosed. The invoke error path then hit logger.Fatalf
(os.Exit(1)) in the same millisecond, killing the process before the
shutdown goroutine reached resourceCleaner.Cleanup. Every registered
cleaner — BrowserSkill daemon, sandbox reapers, IMService, AntsPool,
Langfuse — was skipped on every SIGTERM/SIGINT, leaking the shared bsk
daemon process and its temp home on each restart.
Two changes:
- Drop the manual listener.Close(): Shutdown closes all listeners as its
first step, releasing the port just as fast without racing Serve.
- Give resourceCleaner.Cleanup its own fresh timeout context so a slow
connection drain cannot leave Cleanup with an already-expired context,
which made it skip every entry without running any.
* fix(server): keep signal cleanup inside one grace period
Desktop quit still closed the listener before Shutdown and returned
before ResourceCleaner finished, so Fatalf could skip every hook.
Share ShutdownTimeout between drain and cleanup, run BrowserSkill
first, and do not drop later hooks when the deadline expires.
* fix(server): satisfy errcheck and line-length lint
---------
Co-authored-by: vicfei <vicfei@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
scripts/dev.sh ended with an unconditional `exit 0`, so a failed
`make dev-start` (e.g. an image pull error) printed "服务启动失败" but still
reported success to callers and CI. Drop it so the script exits with the
status of the dispatched command; help still exits 0, unknown commands 1.
MinIO closed anonymous pulls on quay.io on 2026-09-24 (Docker Hub was
already gone), so quay.io/minio/minio now answers 401 for every tag and a
fresh `dev-start` fails. No registry serves RELEASE.2025-09-07T16-13-09Z
anonymously any more. Switch compose and Helm to the community rebuild
pgsty/minio, pinning RELEASE.2026-08-04T00-00-00Z where a tag was pinned
before, and update the docs that named quay.io.
Reverts the eleven plugin system PRs merged on 2026-09-26 so they can be
reviewed before landing:
* a829d7cc9 agent tools served by plugins (#3733)
* 4f5d475c3 Jira example plugin (#3730)
* 01222aac8 dynamic form options, OAuth connections and resource limits (#3729)
* cf502ba46 event delivery and webhooks (#3728)
* 996696057 sandboxed plugin pages (#3727)
* da4ef68d9 standalone plugin hosts (#3725)
* 728007ada remote runtime, Python SDK (#3724)
* ff49f9491 Host API and document parsers (#3721)
* 4328ab26a code plugins (#3720)
* ed63e35cc declarative plugins (#3716)
* c91a8d11f plugin system foundation (#3711)
The tree is back to 47773b5bd, migrations included: no environment ran
the plugin migrations (versioned 000115-000119, sqlite 000034-000038).
* feat(plugin): add plugin manifest, registry and builtin catalog
* feat(plugin): add config schemas with unified secret handling
* feat(frontend): render config schemas with SchemaForm
* feat(datasource): serve connector credential forms from the backend
* feat(im): describe IM platforms and credentials from the backend
* feat(frontend): render model vendor extra fields from the config schema
* feat(frontend): build the settings modal from an extension registry
* refactor(agent): register builtin tools and their chat views
* feat(plugin): let workspaces enable and disable plugins
* feat(plugin): add the runtime driver interface and gather builtins
* feat(datasource/yuque): preserve TOC folder hierarchy for synced documents
The Yuque connector emitted a bare "<title>.md" file name, and folder_path
is derived from FileName alone (SplitKnowledgeRelativePath ->
NormalizeKnowledgeFolderPath). With no "/" in the name the derived folder is
always empty, so every synced document landed at the knowledge base root no
matter where the book's TOC places it.
Read the book's TOC once per sync, fold it into a per-document path by walking
parent_uuid upward, and prepend the result to FileName. The GitLab connector
already works this way; this supplies the missing input rather than changing
the FileName -> folder_path machinery.
Three optional settings, all defaulting to the previous behaviour so the
emitted FileName stays byte-for-byte identical unless folder_mode is set to
"toc". A TOC failure degrades to flat names rather than failing the sync, and
the toc_only filter is skipped entirely when no TOC was fetched.
* feat(datasource/yuque): make toc_only an admission filter, drop include_book_title
Review follow-ups for the TOC folder hierarchy change.
toc_only used to run before the document was recorded for deletion detection,
so a document that was present in the source but not admitted came back as
IsDeleted — and with sync_deletions on, the sync service then deleted it from
the knowledge base. That is the wrong semantics for a key named after
filtering, and on Yuque it is unusually damaging: the web UI does not show
documents created through the API that were never attached to the TOC, so a
knowledge base is often the only place such a document can be reached at all.
A filtered document is now recorded as seen but not ingested, so enabling the
filter removes nothing. It is deliberately kept out of the cursor so that
admitting it later — by attaching it to the TOC, or by switching the filter
off — ingests it instead of dismissing it as unchanged.
include_book_title is removed. Leaving the book segment out silently merges
same-named top-level groups from different books, and the "which book"
dimension has no other carrier: the automatic tag granularity is one tag per
data source, not one per book. Under folder_mode=toc the book segment is now
unconditional; a source still carrying the key keeps syncing with the value
ignored.
Also supplies the UI the feature was missing — a Yuque-only folder_mode select
in the sync-strategy step, with newly created sources opting into the TOC
layout and existing ones left untouched — and documents both settings in the
Yuque section of the datasource guide.
* feat(datasource/yuque): expose the TOC admission filter in the datasource editor
UI follow-ups from review.
The folder-layout control now uses the pill-button group that the neighbouring
sync-mode and conflict-strategy settings already use, instead of a select — a
two-way choice was the only thing on that step rendered as a dropdown.
toc_only is exposed beside it, disabled unless the TOC layout is selected: the
connector only fetches the table of contents under folder_mode=toc, so the flag
does nothing in the flat layout, and the control says so. Its label names the
documents it admits as the ones visible in the Yuque table of contents, which
is what separates them from documents created through the API that were never
attached to one.
Drops the include_book_title paragraph from the datasource guide. That key was
never in a released build, so describing it as removed documents a history no
reader has; the tolerance test for unknown settings keys stays, without the
retired-key framing.
* docs(datasource): note Yuque folder-mode sync limitations
---------
Co-authored-by: niannian <10008660+RichardChengshaojin@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* fix(datasource): preserve schedule state on invalid updates
* docs(datasource): document PUT /datasource/:id as a full replace
The update now always persists sync_schedule, so a partial body clears
the schedule. Show a full-object example and state that omitted fields
are reset rather than preserved.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* feat(wiki): add cross-KB wiki search POST /wiki-search
Give external callers a knowledge-search counterpart for wiki pages: one
SQL IN query ranked by title/slug/summary/content, slim hits with
match_snippet, and Viewer retrieve auth including shared KBs.
* fix(wiki): satisfy golangci-lint on cross-KB wiki search
Check c.Error returns, wrap lines over 120 columns, and use wikiDialect
instead of the embedded Dialector selector.
* fix(wiki): address review on cross-KB search helpers
Share KB viewer auth and snippet extraction, keep ID limits in the
service layer, and escape SQLite LIKE wildcards.
* fix(lint): wrap knowledge span fixtures over 120 columns
Go Lint still diffs from the original PR base, so later main tests
in knowledge_spans_test.go were reported as new line-length issues.
* fix(wiki): authorize KB access before checking wiki enablement in cross-KB search
Checking IsWikiEnabled before AuthorizeKBAccess let a caller tell apart a
foreign KB with wiki disabled (400) from one it cannot see (404), leaking
the KB's existence and settings. Run the access check first.
---------
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
- 07-agent: the retrieved-image output requirement and
internal/agent/image_requirement.go were removed in #3193; the
Observe step no longer appends that message.
- 15-evaluation: internal/handler/dataset.go never existed; dataset
loading lives in internal/application/service/dataset.go.
- 03-document-parsing: the proto generation script is
docreader/scripts/generate_proto.sh (see docker/Dockerfile.docreader).
Self-hosted MinerU capped every parse at a hardcoded 1000s, which fired
before WEKNORA_DOCREADER_CALL_TIMEOUT and WEKNORA_DOCUMENT_PROCESS_TIMEOUT
could matter, so large PDFs always failed at 16m40s (#3091). MinerU Cloud
had the same problem with a hardcoded 600s polling deadline.
- WEKNORA_MINERU_TIMEOUT (default 1000s) bounds a self-hosted parse: the
whole V1 job, or the legacy /file_parse request.
- WEKNORA_MINERU_CLOUD_TIMEOUT (default 600s) bounds how long a MinerU
Cloud batch is polled.
- Both share the env parsing added for WEKNORA_PADDLEOCR_VL_TIMEOUT in
#3537, now in helpers.go: invalid or non-positive values log a warning
and fall back to the default.
Reparse:
- Open the new attempt only after override validation and cleanup pass.
A rejected reparse used to leave the previous run's subtasks treating
themselves as superseded, so the row stayed in finalizing for good.
- Cancel the previous run's queued tasks when it is still in flight.
ProcessDocument, processChunks, post-process, image tasks, wiki ops and
the dead-letter callback now skip work that belongs to a superseded
attempt.
- Summary tasks write only their own columns. A full-row save after the
LLM call could put finalizing over the new run's processing row.
Errors that were acked with the row still in flight:
- A failed post-process enqueue is retried in place and then returned,
so the task retries instead of leaving the row in processing.
- Transient tenant/knowledge reads and the processing write now retry.
- An image whose orphan check fails on its final attempt is still
counted. Empty manual content fails the row.
Counters:
- FinalizeSubtask runs its decrement and promote in one transaction.
Slot releases retry, and the post-process shortfall release keeps
going past a failed release.
Wiki:
- A synthesis model that cannot be built releases the KB's pending
documents on the final attempt instead of retrying forever.
- Panics in map/reduce workers become per-document failures.
- Housekeeping protects a row held by a wiki op for at most 48h.
Lite mode:
- SyncTaskExecutor honours options set on asynq.NewTask (MaxRetry,
Timeout, Deadline, ProcessIn/At), stops on SkipRetry, and runs the
same final-failure callback as the dead-letter middleware.
- The multimodal fan-in counter is kept in process, so post-process
waits for every image.
Also:
- Housekeeping recovers summaries left pending on finished rows. Cancel
and stall recovery close unfinished summaries.
- Housekeeping sweeps no longer overlap.
- Data source URL ingest writes only the metadata column.
- The document list drops cards the batch endpoint no longer returns.
* feat(chat): open cited documents at the cited passage
Clicking a citation used to show only the parsed chunk. It now opens the
original file in the references panel, jumps to the cited place and
highlights it, narrowed to the sentence the citation supports.
Parsing records where each chunk sits in the original file
(chunks.source_locators): PDF page and paragraph boxes from the docreader,
MinerU and PaddleOCR-VL layouts; Word paragraphs, slides, sheet rows and
EPUB chapters aligned in Go against any parser engine's Markdown; text
offsets for Markdown/TXT; segment times for audio. Pipeline text rewrites
are followed with a line-diff remapper. Scanned PDF pages get their OCR
text as quotes at search time so a sentence can pick its page.
The frontend renders PDFs with pdf.js and every other previewable type
with the existing preview, then highlights with the CSS Custom Highlight
API in a highlighter yellow that stays visible against the brand green.
* fix(docreader): serialize source blocks outside main so tests import it
test_docx_tables stubs docreader.parser in sys.modules, so importing
docreader.main from a later test failed in the full unittest discover run.
The wire conversion now lives in docreader.source_wire, which needs only
the proto.
* fix(chat): tighten source locators from review
- Remap rewritten-line-for-line stretches by line so scanned PDF pages
no longer drift onto the wrong page.
- Keep the source preview mounted for citations in the file already open.
- Wait for in-flight PDF page renders before looking for the text layer.
- Count image rune offsets in one pass; stream xlsx rows up to maxUnits.
- Start text locators after a UTF-8 BOM, as viewers decode without it.
- Only re-query image children for scanned-page results.
- Seek audio citations without autoplay and drop stale seek listeners.
- Fix the SQLite migration's cross-reference to versioned/000113.
* fix(migrations): renumber chunk source locators after main's chunk_images
main took versioned 000113 and sqlite 000032 for chunk_images, so the
source_locators column moves to versioned 000114 and sqlite 000033.
* feat(image): observe image attributes and let the policy decide OCR
Describe each image with the VLM as before, but ask it to also observe two
attributes about the image in the same round — how much body text it carries
(contain.text) and whether it presents data visually (contain.data_visual) —
then let a small pure function decide whether that image is worth a second,
separate OCR round.
Per-class OCR was the first shape of this idea; a class (chart / photo / logo)
is only an indirect proxy for 'is there text worth reading here', and the class
table has to grow whenever a new kind of image shows up. The decision actually
needs the observed attributes, so it reads them directly, and the attribute set
is a backend registry row rather than a new class.
The observation and the caption share one VLM call; OCR stays a separate call
(prompt isolation, see #3291). An attribute the model does not answer for is
left absent rather than filled with a conservative default, so the trace never
claims an observation that did not happen, and an incomplete observation falls
back to the on_unobserved policy (OCR still runs). The registry also carries the
panel's display text, so a new attribute needs no frontend change.
Off by default: image_attrs_enabled=false keeps today's behaviour (one caption
and one OCR per image). KB config: image_processing_config.image_attrs_enabled /
image_actions; per-upload override via process_config. Registry:
GET /api/v1/image-attrs/schema.
* feat(gallery): add image gallery tab with schema-driven filters
Adds a 4th knowledge-base tab parallel to Documents/Wiki/Graph that
browses every image asset in a KB.
Backend:
- GET /api/v1/knowledge-bases/:id/images lists image assets projected
from chunks.image_info, with keyword (caption/OCR), multi-value
attribute (attr_<name>=<value>), is_enabled and sort filters applied
in memory, plus pagination.
- Resolves the source knowledge item title (SourceName) via a batched
GetKnowledgeBatch so the viewer can show a human-readable source.
Frontend:
- ImageGallery.vue: thumbnail grid + search/sort + schema-driven filter
panel (from GET /image-attrs/schema) + single-image viewer with
prev/next, attributes/caption/OCR/source.
- Wired as the gallery tab in KnowledgeBase.vue; i18n keys in all 5 locales.
* feat(gallery): drive filters/search/sort from a configurable attribute contract
The gallery UI is no longer hardcoded: it renders from a self-describing
contract served by GET /knowledge-bases/:id/gallery-config (live attribute
sources, every attribute's definition with usage flags merged across the
config tiers, and the caller's search activation state).
- Attribute sources form an open registry (types.RegisterGalleryAttrSource);
a builtin source models caption/ocr_text/timestamps/enabled so the gallery
has zero special cases.
- Usage resolution: user > KB > system > source, field by field; the system
tier lives in system_settings (gallery.policy, JSON, no rebuild), the KB
tier slot is wired in the engine but not stored yet.
- Search activation is the user's own record: mode (all|custom) plus
per-field on/off in users.preferences.gallery; source declarations never
set defaults (unrecorded = off in custom mode).
- image_info is read through a gallery-local struct, removing the compile
dependency on the attribute-pipeline types; the gallery builds and runs
with or without that feature deployed.
- ListImages validates search_in / sort_by / attr_filters against the
contract; ineligible fields are dropped, not served.
* fix(frontend): inline the shared graph-drawer stylesheet (@import instead of <style src>)
plugin-vue 6.0.6 keys unscoped src-style descriptors by the imported file
path, so AgentStreamDisplay.vue and WikiBrowser.vue — both importing
components/css/wiki-graph-drawer.less — overwrite each other during the
build and the second one crashes with "reading 'scoped'". Adding the
gallery changed the module order enough to make the collision hit every
time. Inlining the same stylesheet keeps the emitted CSS identical and
removes the shared descriptor key.
* feat(gallery): render attributes from short labels with descriptions in tooltips
A source could only say one thing about a value, so the filter panel had to put
a full sentence inside a checkbox. Attributes and their values now carry two
pieces of text: a short label for the space-constrained controls (checkbox,
search field, sort option) and a description for wherever there is room
(tooltip, detail panel).
The contract's value list becomes an array of {value, label, description}, and
the display helpers resolve each field separately: the gallery stamps the short
name on a control and moves the sentence into a tooltip. Translation overlays
are looked up per field, so a source that ships no gallery-specific text still
picks up the attribute pipeline's own wording.
* fix(gallery): let locale text win over the contract's wording
The filter panel read the backend's own label before asking the locale,
so a translated value never got a chance: the contract ships its own
default-language wording, and it shadowed the overlay for every locale.
Built keys were also off by a segment — imageAttr.<name> has to be
imageAttr.<name>.label, and the gallery's own keys live under
knowledgeEditor.wikiBrowser.gallery.attr — so builtin and system
attributes fell through to English on a Chinese UI.
* feat(gallery): top bar layout, pinned panels and per-value verdicts
The toolbar moves onto its own row above the grid, so the controls no
longer eat into the space the images are there for: search box, the two
scope switches, sort field and direction, and the result count all sit in
one flex row. The settings behind them are no longer inline either — each
opens as a popover under its control and can be pinned to a right-hand
rail, where the two panels take turns as tabs instead of both claiming
width at once.
Filtering gains a third state per attribute value. Where a value could
only be picked or not picked, it can now be "off" (hide those images),
"on" (show them whatever else says) or neutral (leave them alone), which
is what lets one attribute's rules talk about an image another attribute
tried to hide without the pair of them cancelling each other out: a
forced display outranks a forced hide, and anything no rule speaks about
stays visible.
The switch that used to only exist inside the panel now has a toolbar
home as well. "筛选:全显示" genuinely drops the attribute constraint
instead of imposing it against a panel the user cannot see.
Filtering used to empty the list the moment it was switched on. Any
image whose attribute was never observed — every image ingested before
the attribute pipeline existed — read as a mismatch for every filter,
which is not narrowing a list but losing it. Absence now stays silence:
an attribute no image carries places no constraint on it, until the
pipeline catches up and fills the value in.
Co-authored-by: WorkBuddy
* feat(gallery): expose the image attribute pipeline as a gallery source
The gallery's contract resolved to a single source, so the filter panel
could only offer "Enabled" while the attributes the observe-and-describe
pipeline fills in — contain.text, contain.data_visual — never reached the
UI. The registry is a gallery attribute source like any other: every
attribute it declares is offered to the gallery, filterable by default,
with the config tiers able to override the usage flags at runtime.
The registration lives in the attribute pipeline's own commit; this branch
carried an earlier, squapshot of that work which predated the split of an
attribute value into a short label and a description, so the registry's
value shape and this file have been brought in line with it. Both PRs now
hold byte-identical copies of the source hook, which is what lets either
one land first and the other merge cleanly afterwards.
Co-authored-by: WorkBuddy
* fix(gallery): viewer race, pin icon, panel anchoring and scope wording
Opening the viewer rendered the img element with the previous or empty
source while the proxy fetch was still producing a blob, and the error
that empty source fired locked the viewer in its failure state even
after the real image arrived. The resolved source now lands before the
flag is consulted, and a token keeps a slow fetch of one image from
overwriting a faster one that navigated past it.
The pin button drew nothing: TDesign has no "pushpin" glyph, so the
button rendered as an empty shell. It now uses pin / pin-filled, keeps
its place beside the tab bar, and a pinned rail assigns its columns
explicitly — the panel precedes the main section in the DOM, so leaving
the grid to source order handed the wide column to the panel and
squeezed the images into the rail.
The floating panel no longer appears at a hardcoded offset: it anchors
just below the arrow that summoned it, the arrow flips up while its
panel is out, a second press retracts, and any click elsewhere in the
toolbar closes it. When a rail is pinned the arrows turn to the
requested tab instead of spawning a second surface.
The scope words (全部 / 全显示 / 自定义) become plain status text rather
than a second dropdown — they name the current mode, and only the arrow
beside them opens the panel that changes it.
Co-authored-by: WorkBuddy
* fix(web): align the OCR-on-unobserved switch with the other setting rows
The switch sat bare inside its setting-row while every other switch on
the page is wrapped in setting-control, which is what reserves the right
column and centers the control vertically. With the row laid out from
the top, the bare switch hugged the row's upper edge and crowded the
description text — visibly out of place next to its neighbors. Wrapping
it in setting-control gives it the same right-aligned, centered slot as
the rest.
Co-authored-by: WorkBuddy
* fix(web): move the OCR-on-unobserved row out of the vertical schema row
The row lived inside setting-row-vertical, whose descendant rule stretches
every setting-info and setting-control beneath it to 100% width. Two
full-width flex children side by side cannot fit, so the control overflowed
the container and the switch landed past the right edge — visibly further
out than the switch on the row above. The row and its note now sit beside
the vertical block instead of inside it, under their own condition, so the
switch shares the ordinary row layout and its right edge lines up with the
"image attribute observation" switch above.
Co-authored-by: WorkBuddy
* fix(gallery): applying a verdict now activates the filter scope
Setting an off/on verdict while the panel's master switch was off
silently ignored it: buildParams only attaches attr_rules in custom
scope, and setVerdict never flipped the scope. A verdict the user just
expressed now switches the scope to custom; clearing the last meaningful
verdict returns to all, since custom with no rules constrains nothing.
* fix(gallery): search checkboxes stay visible and idle panels read as dimmed
The search panel hid its field checkboxes while the master switch was
on, so the only way to reach them was to flip the switch first. They now
sit next to the switch like the filter panel's verdicts do: dimmed in
all-field mode (where they carry no weight) yet clickable, and touching
one drops the mode to custom. The filter panel's verdict rows dim the
same way while the filter switch is off.
* fix(gallery): bulk verdict row, neutral default mark, panel polish
The clear-filters button never matched what the panel is for, so it is
now a bulk verdict row: one set of -/off/on buttons that applies the
chosen position to every value of every value-typed attribute, with the
default mark turned into a language-neutral dash. The panel head drops
its second divider (the tab bar already draws one) and the tabs follow
the toolbar's order: search left, filter right.
* fix(gallery): drop the all-fields caption and center the order arrow
The caption above the search checkboxes showed and hid with the master
switch, shoving the checkboxes down on every flip; it said nothing the
dimmed state doesn't, so it is gone. The order button's arrow is an
inline svg, which sits on the text baseline and reads as riding high;
it now centers against the label.
* fix(gallery): center the order arrow for real
vertical-align never applied: the icon is a flex item inside the
button's inline-flex text wrapper, and the wrapper's default stretch
pins the explicitly-sized 16px icon to the top of the 22px line box.
align-items: center on the wrapper is what actually centers it.
* fix(gallery): shorten panel tab labels in en and ru
The two-word labels overflowed the 300px panel and forced the tab bar
into scroll arrows; single-word labels fit in every locale that was
affected.
* docs: drop the image pipeline rewrite from this branch
The rewritten 'Go 侧接手后' paragraph belongs to the image attribute observation
work (#3635), which this branch carries as a prerequisite and which already
documents the caption_ocr / observation_driven split. Keeping a competing
rewrite of the same paragraph on the gallery branch makes it collide with
upstream, which extended that paragraph with the custom_instructions and inline
HTML table notes. Restoring the shared wording keeps the gallery branch free of
this file so upstream's edits merge cleanly.
* style: gofmt the gallery attribute types
Struct field tags in the resolved gallery config were one column narrow, so
gofmt rebroke the alignment block. Whitespace only.
* style(gallery): use design tokens in the gallery styles
The style guard ratchet fails on any new violation. ImageGallery.vue shipped
with 47 TDesign token fallbacks, 21 literal font sizes, 4 literal radii and a
0.15s transition; all of them now read the --app-* / --td-* tokens, which is
what the guard asks for and what dark mode depends on.
* lint(gallery): fix the golangci-lint findings in the gallery code
lll: the three over-long swagger @Param lines and one registry row were
shortened. ineffassign: ctx was assigned before the access check and then
reassigned, so the first value was dead; it is now declared once, after the
check. errcheck: the three unchecked c.Error calls in ListImages. revive: the
AttrSpec / AttrValue doc comment block was attached to the wrong type and split
into one block each. staticcheck S1016: the gallery registration converts the
registry value instead of restating its fields.
* lint(gallery): wrap the image query select and check the last c.Error
* style: reformat the wrapped image select in chunk.go
* fix(gallery): address review findings on ordering, KB switch and blob lifetime
- sortImageAssets: swap operands for desc instead of negating, keeping a
strict weak ordering so ties page stably
- ImageGallery: reload on knowledgeBaseId change, drop stale list
responses, fetch thumbnails in parallel with one cached blob per URL,
revoke blobs on unmount, default sort to builtin:created_at
- KnowledgeBase: non-wiki KBs with a stale wiki/graph tab fall back to
documents again
- handler: move ListImages godoc out of ListKnowledgeFolders' block and
document attr_filters / attr_rules
- DecideOCR: a payload with no OCR conditions uses the built-in policy
instead of silently skipping OCR
* docs(parsing): describe the caption_ocr and observation_driven image pipelines
Carried over from #3635, which this PR supersedes.
* fix(gallery): load local:// and other storage handles through the KB file proxy
The gallery's own scheme check only caught resource://, provider:// and
storage://, so local:// (the default storage), minio://, cos:// and s3://
images went straight into <img src> and never rendered. Resolve through
isProviderFileURL, the scheme list chat and wiki already use, and lock the
routing with a test.
* fix(kb-editor): keep API-customised OCR conditions on save and upload
The KB editor always wrote the registry default into image_actions.ocr.on,
so any unrelated edit replaced a knowledge base's custom OCR conditions
with the default table; the upload dialog did the same for the per-document
override. Keep a stored custom list and fall back to the registry default
only when there is none, and show the conditions actually in effect instead
of the default table in both panels.
* perf(gallery): de-duplicate, filter, sort and page images in SQL
Every gallery request used to read every image-bearing chunk of the KB,
OCR text included, decode it in Go and page in memory. A single query now
expands the image_info arrays, keeps one row per image URL, applies the
keyword / attribute / verdict / enabled filters and the sort, and returns
only the page with the filtered total.
The expansion carries narrow scalars only; filters and JSON sort keys are
evaluated on the winning entry of each image, and the page's JSON is read
back for its rows alone. Postgres runs the query with JIT off, since the
fixed 100-rows-per-call estimate of jsonb_array_elements otherwise prices
it ~100x high and compilation dominates.
On 20k images with 2 KB of OCR text each (40k chunks), a page drops from
~750 ms to ~100-200 ms on postgres and ~200 ms on sqlite; keyword search
over OCR text stays a full scan (~400-600 ms) until the text is indexed.
The filter semantics are pinned against both databases.
* perf(gallery): serve the gallery from a trigger-maintained chunk_images table
Reading images out of chunks.image_info meant parsing every image chunk of
the knowledge base on each request, so a page still cost time in
proportion to the KB. chunk_images (versioned/000113, sqlite/000032) keeps
one row per image_info entry with caption, OCR text and attributes as
columns; triggers on chunks keep it in step with every write path, and the
migration backfills existing chunks.
Copies of one image stay separate rows and the query lists the copy on the
most recently updated chunk through an indexed anti-join, so writes decide
nothing and concurrent writers need no locking. On postgres, caption and
OCR text get pg_trgm indexes when the extension is available.
On 20k images with 2 KB of OCR text each (40k chunks):
postgres first page 10 ms, page 500 41 ms, keyword 2 ms, filters 23 ms
sqlite first page 3 ms, page 500 39 ms, keyword 147 ms, filters 51 ms
down from ~100-600 ms. Writing those 40k image chunks costs 7.0 s instead
of 1.3 s on postgres (1.9 s instead of 1.0 s on sqlite), mostly the trigram
indexes; status and enable-flag updates patch rows in place.
---------
Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(retrieval): keep index writes and shared-KB lookups correct
Index writes
- Image OCR/caption chunks were indexed without IsEnabled, so every engine
but Postgres stored them disabled and retrieval never returned them.
The SQLite engine re-enables the affected rows at startup.
- FAQ edits, similar-question additions and import merges re-index under
the old SourceID and relied on the engine overwriting it. Postgres and
SQLite kept the stale row (ON CONFLICT DO NOTHING); ES v8 and Qdrant
mint a new document ID and kept both. FAQ re-indexing now deletes the
old rows first, and Postgres/SQLite BatchSave upserts by
(source_id, source_type).
- Postgres: drop the GORM default tag on is_enabled, which replaced false
with true on insert and indexed disabled chunks as enabled.
- SQLite: BatchSave no longer attaches vectors and FTS rows to the wrong
rows when part of a batch already exists, and vecTables is guarded
against concurrent map writes.
- CopyIndices: Postgres filters to the copied knowledge, pages by id and
keeps is_enabled; SQLite copies every row of a chunk (generated
questions included) with stable source IDs and enabled state.
Shared knowledge bases
- Chat merge: FAQ answer population and neighbor expansion look chunks up
without the caller-tenant filter, like parent resolution (#3342).
- Neighbor expansion never cuts the retrieval hit; the length budget goes
to the text adjacent to it.
- FAQ negative questions are applied whenever an FAQ KB is in scope and
to chunks of shared FAQ KBs.
- Search results get image_info for shared-KB chunks.
- search_knowledge reads FAQ metadata from the search result instead of
one tenant-scoped query per hit, so shared FAQs render their answers.
* fix(retrieval): put every retrieval score on one scale
Scores from different retrieval paths were ranked together while living on
different scales, so ordering depended on which path produced a hit.
Fusion
- RRF ranked a chunk by its position in the concatenation of every
vector (or keyword) list, appended in goroutine completion order. The
second list's best hit ranked behind every hit of the first, and the
order changed between identical requests. Each list is now ranked on
its own by score and a chunk takes its best rank.
- RRF is divided by its maximum, so hybrid scores land in [0, 1] like
vector-only (cosine) and keyword-only (rescaled BM25) results. Raw RRF
topped out near 0.016 and lost to any single-retriever hit it was ranked
against across searches (document targets, model groups, FAQ calls);
rerank's base-score term and MMR's relevance term had nothing to use.
- Keyword-only BM25 is rescaled per list; engines' BM25 is not comparable.
- FAQ iterative retrieval fuses each round like the first pass instead of
sorting raw BM25 and cosine together, starts at twice the depth already
searched, triggers when any single vector list is full (the old summed
count never matched with more than one list) and replaces results per
round.
Rerank and chat pipeline
- The FAQ boost is no longer capped at 1; capping made strong FAQs tie at
1.0 and fall to the tie-breaker. The composite score before boosts is
kept as composite_score.
- The FAQ direct-answer threshold is compared against that pre-boost
score, and the FAQ that cleared it is the one marked match="exact"
rather than the first FAQ.
- Graph hits no longer carry a fabricated retrieval score of 1.0, which
ranked them above every scored hit when rerank did not run; the
composite score uses the model score in its place.
Engines
- OpenSearch and Weaviate return (1 + cos) / 2; their drivers now report
cosine similarity and map thresholds onto the engine scale, so a
threshold means the same on every engine.
- Milvus L2 returned squared distance, ranked farthest-first downstream,
and took the threshold as a radius; scores and radius are converted.
- The multi-store normalizer no longer maps Milvus through (s + 1) / 2,
which put it on a different scale from every other engine and from its
own single-store results.
Docs updated for fusion, normalization, FAQ boost and iterative recall.
* perf(retrieval): bound rerank, agent and pipeline cost; stop failing on slow or missing models
Rerank
- Rerankers report the longest document a request can carry
(max_document_chars / max_request_chars minus the query) and the rerank
stage fits its own passages to it, trimming appended captions, OCR and
generated questions first. One chunk with a large screenshot's OCR used
to exceed the vendor limit and fail the whole batch, so every candidate
lost its score and the threshold stopped applying.
- Rerank HTTP requests default to a 60 s timeout; chat and agent contexts
carry no deadline, so a stalled self-hosted rerank server held the turn
open.
- OCR and caption chunks no longer repeat their own text from image_info in
the passage.
- The chat pipeline and the agent tool rerank at most 200 candidates, best
retrieval score first.
- A rerank model that cannot be loaded degrades to retrieval order instead
of failing every chat turn, as the search APIs already do; without rerank
results the merge stage keeps only RerankTopK hits instead of enriching
every one.
Retrieval APIs
- knowledge-search match_count and rerank.top_k are capped at 200
(unbounded values multiplied into tens of thousands of billed rerank
passages); normalizedMatchCount clamps to the pool size so the
over-retrieval arithmetic cannot overflow.
- hybrid-search with rerank recalls at least rerank.top_k deep.
Agent
- search_knowledge fits its rendered output to four fifths of the tool
output budget, dropping the lowest-ranked rows and telling the model how
many it omitted; the model's view is rebuilt from Data, which registry
truncation never reached.
- Agent searches skip neighbor/parent/related enrichment rows, which were
never retrieved yet tripled the rerank input.
- ParallelToolCalls is enabled. The model was always told it may call tools
in parallel while the engine ran them one by one; only read-only tools
overlap, anything else stays a barrier.
Chat pipeline
- Query expansion searches keywords only (its variants are local rewrites
whose embeddings repeat the main search) and is skipped when keyword
matching is off.
- Entity extraction, a second LLM call, is skipped for intents that need no
KB retrieval and uses the rewritten query.
- Partial-overlap dedup tokenizes each text once instead of per pair, and
only compares token overlap between texts within 3x of each other's size.
Storage
- Migration 000112 adds btree indexes on embeddings(knowledge_id) and
(chunk_id); document deletes, chunk re-index and enable/tag updates
scanned the whole table.
- Shared-KB knowledge rows are fetched in one query instead of one per ID.
- Drop GORM Debug() from the ParadeDB keyword query and the knowledge batch
lookup, which logged SQL and user queries on every search.
Docs updated for the API limits, rerank timeout and passage fitting, agent
parallel tools and output budget, and pipeline changes.
* fix(retrieval): make agent tools and pipeline stages report what they did
Agent tools
- query_knowledge_graph now queries the graph store: entity names from the
query are matched in each KB's graph, relations are returned (and shown to
the model), and the entities' source chunks lead the results, with text
search as a supplement. It used to run text search only, so a graph-only
KB always answered "no relevant graph information".
- search_knowledge reports failures instead of "no matching chunks": when
every search fails the tool fails with the reason; partial failures are
listed. A failed query embedding falls back to keyword search for hybrid
KBs, as the chat pipeline does.
- Keyword mode uses rerank to order hits, not to reject them (identifiers
score low), and the empty-result hint no longer sends keyword mode back to
keyword mode.
- Dedup keeps siblings under one parent, no longer merges every image chunk
of a document through chunk_index 0, and keeps the better-scored copy.
- Search rows omit a <match> excerpt that only repeats the content; snippet
terms are segmented for Chinese queries and skip stopwords, and FAQ
matched questions are chosen by match strength.
- read_document query mode returns next_offset when truncated and resumes
from offset.
- wiki_search over-fetches under document/tag scopes before filtering and
says when every match was out of scope.
- Prompts point to read_document(id=cN, context=k) for surrounding text.
- The agent editor hides the retrieval-strategy section in agent mode,
where search_knowledge uses the global retrieval settings.
Chat pipeline
- Intent labels are normalized; an unknown label retrieves instead of
silently skipping the knowledge base, and web_search without web search
enabled searches the knowledge bases.
- Image turns get a 2048-token rewrite budget, and a reply cut off by the
cap is salvaged field by field instead of being discarded.
- History references are matched by how much of the query they cover;
Jaccard against chunk-sized text never reached its threshold.
- Entity search resolves chunks of org-shared KBs within the turn's KB
scope, skips chunks whose document is gone (it used to panic) and caps
graph chunks at 30.
- Empty history is not fetched twice; fetched pages are cut on a rune
boundary.
Retrieval
- A multi-KB search whose primary KB has no vectors embeds the query with a
vector KB's model instead of failing on an empty model ID.
- Rerank passages include the chunk's heading breadcrumb, which is part of
what was embedded.
- SQLite FTS searches the content column only and turns a lone Han
character into a prefix query; TencentVectorDB ef follows TopK.
Docs updated for the agent tools and pipeline stages.
* fix(retrieval): address review findings on the retrieval fixes
- Restore NeedsRetrieval for web_search without web search enabled: the
intent prompt "web_search" is how the user learns web search is off, and
running KB retrieval instead bypassed it.
- query_knowledge_graph keeps the case of entity terms; Neo4j matches node
names with case-sensitive CONTAINS, so lowercased terms never matched
English entities.
- An empty search_knowledge or query_knowledge_graph result states partial
failures in its text, which is all the model sees for an empty result.
- SQLite BatchSave replaces a source in one transaction, so a failed insert
keeps the old rows instead of leaving the source unindexed.
- Reuse stringSliceValue and an exported searchutil.ContainsChinese instead
of new copies; drop an unused named return and the FAQ obsolete-source
delete that the full per-chunk delete now covers.
* fix(retrieval): address second review of the retrieval fixes
- query_knowledge_graph: under a document/tag scope, keep only relations
whose entities come from an in-scope evidence chunk (the graph namespace
is the whole KB); drop chunks whose document no longer exists; report a
graph or text search failure even when the other source returned hits.
- search_knowledge: record the hybrid -> keyword fallback taken when the
query embedding fails in mode_fallbacks, so an empty result is not read
as "no semantic match".
- query understanding salvage: take a cut-off value only for the image
description; a truncated rewrite_query or intent is dropped instead of
replacing the user's query.
- Graph hits (no retrieval score) are kept past the no-rerank RerankTopK
cut and the rerank candidate cap instead of always falling below them.
- FAQ boost is capped at 1 again so scores stay on the [0, 1] scale; FAQs
tied at the cap are ordered by their pre-boost score.
- Negative-question filtering loads only candidates of FAQ knowledge bases.
- read_document query continuation starts one chunk before offset so the
first match keeps its preceding context at a page boundary.
- graphSearchTerms and searchQueryTokens share one queryTerms splitter.
* docs(readme): name skills, sandbox, browser and MCP in the overview
The overview only mentioned the built-in MCP Server; the agent's own
tools (skill sandboxes, the local browser, external MCP services) first
appeared far down in What's New and the feature tables.
Add an "Agent tools and execution" bullet to all four READMEs, and
update the Agent capability card (README artwork and the homepage it is
copied from) to mention the local browser, with concrete tags instead
of "tool calling / skills".
* docs(readme): condense the English feature and client tables
The four feature tables had 42 rows, many written as release notes
(removed backends, deprecated packages) or UI details, and repeated
what the overview, product tour and What's New already cover.
Replace them with a seven-row feature table linked to the docs and a
supported-backends table that keeps every compatibility list. Drop the
MCP Server subsection and the three client detail blocks, which repeat
the clients table, and shorten the v0.7.x entries in earlier releases.
* docs(readme): drop the v0.8.2 screenshot row
The three screenshots rendered at a third of the column width, too
small to read, and the local browser shot already appears in the
product tour.
* docs(readme): move the agent toolbox out of "beyond the three modes"
Skills, sandboxes, the local browser and MCP tools are part of the
agent mode, not something beside it. Describe them in a short paragraph
right under the capability cards and keep the list for platform
capabilities.
* docs(readme): condense the CN, JA and KO feature and client tables
Bring the translated READMEs to the same structure as the English one:
a seven-row feature table linked to the docs, a supported-backends table
with every compatibility list, no MCP Server subsection or client detail
blocks, and one-line v0.7.x entries.
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
MinerU 4.0 removed the synchronous /file_parse endpoint that the mineru
engine relied on, so a self-hosted 4.0 server answered every parse with
404. The reader now probes GET /v1/health before each parse: a 200 selects
the V1 flow, 404/405 keeps the legacy /file_parse path (AMD and domestic
accelerators must stay on mineru<4), and a 503 error envelope surfaces the
server's model-preload failure instead of falling back.
The V1 flow uploads via /v1/uploads (sha256 dedup, API key only sent to
same-origin upload URLs), creates a zip-only parse job, polls with
exponential backoff (2s to 30s), cancels the job on timeout or caller
cancellation, and reads markdown.md plus images/ from the zip artifact.
The zip extraction is shared with the MinerU Cloud reader and now also
resolves <img src> references and percent-encoded paths.
New tenant settings: mineru_server_api_key (redacted in responses,
preserved on update) and mineru_tier (flash/basic/standard/advanced, empty
for the server default). mineru_parse_method maps to V1 ocr_mode. Backend,
vLLM URL, formula/table toggles and language only apply to MinerU 3.x and
are grouped as legacy options in the settings page. The connection check
also verifies the API key on V1 servers, since /v1/health is
unauthenticated.
Deployments upgraded from the old .env.example keep the published example
SYSTEM_AES_KEY, which SystemHMACKey rejects since v0.8.2. Embed sessions
then fail at request time with no hint at boot.
- Log a [startup-env] warning when no usable signing key is found, and
surface SYSTEM_SIGNING_KEY in the startup env banner.
- Name SYSTEM_SIGNING_KEY in the embed session 503 error.
- Provision the desktop signing key before printing the banner so the
desktop build does not warn about a key it is about to create.
- Document the symptoms and fix in troubleshooting.
* feat(models): add system admin model catalog console
System administrators can now maintain the model catalog from
Settings → System administration → Model catalog instead of editing
models.json on every replica. Administrator changes are stored as a
versioned overlay in the database (new model_catalog_config table),
layered between the deployment file and explicit per-model settings,
and synchronised to other instances within about 5 seconds.
Backend
- GET/POST preview/PUT /system/admin/model-catalog with optimistic
version + deployment baseline checks (409 on conflict), 20-version
history and audit logging.
- Console overlays reject credentials, headers, env interpolation and
non-inline icons; those stay deployment-owned.
- Catalog views include each chat model's resolved thinking levels and
the vendor defaults, so the console never re-implements level
inheritance.
Frontend
- Model list in the shared data-table style with provider/type filters,
"modified only" filter and source tags (built-in / deployment / admin).
- Edit drawer saves and applies immediately: display name, context
window, max output, input modalities, reasoning, thinking levels,
embedding dimension and picker visibility, each with its inherited
default and a layer comparison table.
- Add-model drawer, JSON editor with server-side change check before
publishing, and version history with one-step restore.
- Model settings gets a system-admin shortcut to the catalog; model
editor reloads providers when opened so published catalogs apply.
* fix(models): harden the admin model catalog console
- Reject console base_url, base_urls, url_patterns and auth: moving a
vendor's default URL would send its deployment api_key (and keys of rows
without their own base URL) to another host. With every interpolated
field now deployment-owned, '$' in names and descriptions is plain text.
- Drop deployment base URLs from catalog views; they may carry
interpolated tokens.
- Remember a stored version this replica cannot compile and skip it until
the version changes, instead of re-reading and recompiling every poll.
- Resolve vendor default thinking levels with reasoning forced on, like
per-model levels.
- Compute builtin/deployment views once; preview returns only the
candidate catalog and normalized overlay.
- Share deployment file lookup via runtime.ReadDeploymentOverlay.
- The model editor force-refreshes its own type instead of resetting the
provider store for every type; the refreshed list wins in the id index.
The docs header gained a '版本' entry, and the homepage release gallery
now ends with the 对话控制 slide linking to 03-features/18-chat-experience
instead of the v0.8.0 memory slide. Expect the new header entry, and
assert the release link lands on the chat-experience page by path and
heading. The memory doc stays covered by the search and direct-route
checks.
The site is deployed behind an upstream router that forwards only `/` and
`/docs/`. The homepage export also claimed /_next/, /brand/, /brands/,
/product/, /favicon.ico and Next's root RSC payloads, so behind that router
the page loaded without its scripts, styles or images.
Next's assetPrefix and the homepage's public assets now live under
/docs/_home/ (one `homeAssets` constant shared with the docs header), the
favicon reuses /docs/favicon.ico, and build-site.mjs leaves only index.html,
404.html and docs/ at the root. The homepage logo links are plain anchors
because <Link> navigation fetches RSC payloads from the root. check-site.mjs
now fails the build if a page references anything outside `/` and `/docs/`
or if anything else appears at the site root.
Move the CHANGELOG and release-notes date to 2026-09-24. In all four
READMEs, add the v0.8.2 entry to Latest Updates and bring the overview
and feature tables up to date: local browser, built-in MCP Server,
conversation control, artifacts library, sandbox terminal and desktop,
Lite host sandbox, model catalog, Confluence, Bocha / Serply, UI
languages and whitelist-only outbound. The MCP Server section now points
to the built-in server and marks the Python one deprecated.
* fix(mcp): return accessible image URLs from existing tools
* fix(mcp): resolve resource URLs via binding lookup and cover shared KBs
- Look up the KBs that bind each reference instead of scanning every
allowed KB, and cache per-KB grants for the call. Unrestricted
endpoints accept any KB the caller can view, so ask answers drawing
on agent-selected shared KBs get links too.
- Share grant/owner-tenant resolution with scopedKBContext via
authorizeKB; log non-denial authorization errors.
- Drop the redundant resource-catalog re-wrap; both sources are
already wrapped.
- Skip the structured-content round trip when no storage reference is
present, rewrite non-object results, and log marshal/decode failures.
- Quote ask image lines with strconv.Quote so & stays literal, and
never cut an excerpt inside a storage reference.
* docs: sync website-docs with changes since v0.8.0
Audit every change between v0.8.0 and the v0.8.2 release against the
documentation site, adding what users and operators need and correcting
statements that no longer match the code.
- Add local browser (BrowserSkill), built-in MCP Server endpoints, the
toolbox, conversation steer/fork/rewind and reasoning effort, the
artifacts library, sandbox terminal and desktop, the Lite host sandbox,
the rebuilt model catalog (including a protocol-compat field reference),
Confluence and DingTalk Docs sources, Bocha/Serply search, per-channel IM
locale, whitelist-only outbound, admin user creation, new configuration
variables and migrations 000091-000110.
- Fix wrong API examples and parameters (model type values, knowledge
search params, FAQ match_count cap, Notion credentials), tenant deletion
semantics, default timeouts, parser engine claims, removed tables and
commands, stale navigation paths, and route locations after the router
split into routes_*.go.
- Add screenshots for the new pages and refresh outdated ones.
* fix(frontend): link the protocol-compat field to its documentation
The model editor told users to read catalog/compat.go on the backend, a
path that no longer exists after the model catalog refactor. The hint now
says when the override is needed and links to the field reference in
website-docs/03-features/06-models.md; the config examples and the API
type comment point to the same place.
* feat(website): showcase v0.8.2 on the homepage
- Add a v0.8.2 section for the local browser (BrowserSkill), the built-in
MCP Server and conversation control, and turn the v0.8.0 release block
into a standing skills & sandbox topic with the sandbox desktop and
terminal.
- Replace the three-up galleries with a feature list beside one large
screenshot: every item's text stays visible, clicking an item or one of
its views switches the image with a cross-fade, and the list plays itself
while on screen until the visitor picks something.
- Add Confluence, DingTalk, BrowserSkill and more model vendors to the
integrations, bump the header badge to v0.8.2 and link the changelog.
- check-site reports missing docs screenshots as pending instead of
failing, since the Screenshot component renders a placeholder for them.
* docs(changelog): add v0.8.2 release notes
* docs(website): add per-version release notes pages
Add a "版本发布" docs section with one page per release, starting with
v0.8.2 and a backfilled v0.8.0. Each page walks through the highlights
with the existing product screenshots, the other improvements and fixes,
and an upgrade checklist with the actions each breaking change needs.
The sidebar orders these pages by version, newest first; the docs header
gains a 版本 entry, and the homepage's v0.8.2 section now links to the
release notes page instead of the raw CHANGELOG.
* feat(website): let the site image listen on a configurable port
The Docker image hard-coded `listen 80`, so running it with host
networking or on a platform that assigns the container port required
editing the Nginx config and rebuilding. The image now renders its
config from the official nginx template directory at startup and reads
the listen port from `PORT` (default 80); deploy/nginx.conf stays a plain
config for static deployments.
test-docker starts a second container with PORT=8080 and checks the
rendered config and that the site answers on that port; the README
documents the variable.
* fix(website): namespace the nginx listen port variable
* fix(website): support bash-wrapped container startup
* fix(website): align nginx startup with frontend entrypoint
* fix(website): pin nginx runtime for older host compatibility
* fix(website): bake the configured nginx port into the image
- Updated the BROWSERSKILL_EXTENSION_PATH in .env.example and Dockerfile.app to reference the new extension version 0.3.1.
- Modified localization files for English, Japanese, Korean, Russian, and Chinese to reflect the support for both Chrome and Edge browsers.
- Added checks for extension version compatibility in the BrowserConnectionSettings component to improve user experience.
This commit ensures that the application is aligned with the latest BrowserSkill extension and enhances the localization for better user guidance.
* feat(lite): let new chats bind a host project directory
Unpinned Lite sessions already run on the host sandbox. The new-chat page can now pick an approved project, and the header shows that workspace instead of a full path.
* docs(lite): publish the host sandbox note on the site
The published docs live in website-docs, so the host backend entry belongs in the sandbox guide. Session creation also stops rewriting project_dir inside the shared API helper.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
* docs(ollama): document embedding model config and RAM guidance (#3480)
Local embedding and generation share the Ollama process at OLLAMA_BASE_URL.
The embedding model name is the local Embedding model's name; EMBEDDING_MODEL_NAME
is read only when builtin_models.yaml references it. The published 8 GB
starting point does not include Ollama weights or optional Neo4j.
Fixes#3480
* docs(ollama): shrink README/docs scope per maintainer review (#3480)
* feat(security): add a whitelist-only outbound mode that refuses before DNS
A penetration test of a private deployment found that a host outside
SSRF_WHITELIST still produces a DNS query: every outbound guard resolves
the name first and judges the addresses afterwards, which is what makes
it immune to DNS rebinding, but it also means an attacker-chosen name
reaches the resolver and can carry data out of an otherwise offline
network (#3378).
SSRF_DNS_WHITELIST_ONLY makes the whitelist the whole egress policy, and
the check is placed wherever a name is still in hand.
Where a URL is available, CheckSSRFWhitelistOnly refuses a host the
whitelist does not name, IP literals included: URL validation, the
sandbox outbound policy, and the web-fetch target resolver.
Where only a dial address is available, CheckDialWhitelistOnly judges
hostnames and leaves addresses to the checks that already exist, because
an IP arriving at a dialer is either a name something else resolved or a
literal the URL layer already refused. That covers the shared SSRF-safe
dialer, the web-fetch dialer, the Yunzhijia webhook dialer, and every
sandbox transport: the HTTP and websocket data planes, the guarded
transport and the Docker engine now share one GuardedDialContext instead
of each building its own dialer.
Where the client resolves its own target, the name is judged before the
client exists. gRPC's resolver runs before any dialer it was given, so
the env-configured qdrant and milvus clients are built through small
constructors that refuse a non-whitelisted host outright; a whitelisted
one is resolved by gRPC as before.
docreader enforces the same switch in its own validator, and compose
passes the variable to both services.
Matching is by name only in this mode: the whitelist's CIDR entries
still match an IP literal, but a hostname is no longer resolved to see
whether its addresses fall inside one, because that lookup is the query
the mode exists to prevent.
The value is parsed with ParseBool, and a non-empty value that parses as
neither enables the mode: silently leaving an egress lockdown off
because an operator wrote "yes" is the one failure mode a security
control must not have.
Still not covered, and documented as such: the Langfuse OTLP exporter
builds its own client and never reaches these guards; and with
HTTP(S)_PROXY set the proxy host is dialled unguarded when the dial
address equals the proxy URL's host exactly, while a mismatch makes the
proxy host itself subject to the whitelist.
The switch defaults to off, so nothing changes for deployments that do
not set it.
* docs(security): document SSRF_DNS_WHITELIST_ONLY in website-docs
docs/ has been retired in favour of website-docs/ (#3580), so the notes that
lived in docs/QA.md move to the configuration table: the new switch, what to
whitelist before turning it on, and the three outbound paths it still does
not cover.
The existing SSRF_WHITELIST row was also stale: compose gives only the app its
SSRF_WHITELIST_EXTRA default (which now includes minio), and docreader's is
empty, so "shared by app and docreader" held for SSRF_WHITELIST alone.