* feat(models): expose embedding input modalities
Whether an embedding model maps images into the same space as text is
what will decide if a knowledge base can store image vectors. The
catalog already had the field, but most multimodal embedding models
were missing it and embedding rows never reported it.
- Catalog: declare input modalities for the Doubao and Tongyi vision
embeddings, the Aliyun multimodal / qwen-vl embedding families and
jina-embeddings-v4, as listed in each vendor's documentation. The
migration baseline changes for exactly these nine entries.
- Model responses resolve capabilities for embedding rows too, so
capabilities.input reflects the catalog merged with spec.input.
- Model editor: an "Image input" switch for embedding models. It follows
the catalog and writes spec.input only when the row differs from it;
picking another catalog model or switching vendor drops the
declaration. Model cards show the image badge for such models.
* feat(embedding): embed images with multimodal embedding models
Adds the image side of an embedder, for models that map images into the
same space as their text vectors so a text query can find an image.
- api: EmbedImage (bytes + MIME type, sent inline as base64 because the
object store is usually unreachable from the vendor) and an
ImageEmbedder implemented by all four protocols, each in the image
shape its reference documents: Ark image_url, DashScope contents[].image,
Gemini inlineData parts, and on the OpenAI shape an object keyed by the
vendor's image_field (Jina: "image"). A plain OpenAI-compatible
endpoint names no image field and takes no images.
- Settings: image_field, max_image_batch_size (default one per request),
max_image_bytes and image_mime_types, filled from vendor docs for the
DashScope multimodal models, gemini-embedding-2 and Jina.
- embedding: an optional ImageEmbedder interface reached through
AsImageEmbedder, so existing Embedder implementations and fakes are
untouched. An embedder takes images only when the catalog (or the
row's spec.input) lists image AND the endpoint can carry one. Images
are checked against the documented limits before any request, then
split at the documented images per request. The concurrency, debug
and Langfuse decorators forward it; logs and traces carry the MIME
type and size, never the bytes.
The migration baseline changes only for the entries whose resolved
embedding settings now carry image limits.
* feat(knowledge): index images by their own vector and recall them
When the knowledge base's embedding model embeds images, each image now
gets a vector of its own next to its caption and OCR chunks, so a query
can find an image by what it shows even where the caption left it out.
Ingestion
- After the caption/OCR chunks are indexed, the multimodal task embeds
the image (embedding.PrepareImage converts and shrinks it to the
vendor's documented limits) and stores it as a new image_vector chunk:
Content is the caption (OCR text when there is none) for reranking and
the answer context; its index row carries the image's vector with
SourceType ImageSourceType. Scanned PDF pages are skipped, their OCR
text serves better. A failure here is logged and never fails the task.
- The image vector gets its own chunk rather than a second index row on
the caption chunk: OpenSearch and Weaviate key documents by ChunkID,
so a second row would overwrite the caption's text vector, and copy,
move and enable/disable all follow chunk IDs.
- CompositeRetrieveEngine.BatchIndexVectors writes precomputed vectors
through each engine's normal BatchIndex, keeping row order (the
composite's de-duplication does not).
- Paths that rebuild vectors from text leave image_vector alone:
syncChunkIndex only syncs its enabled flag, updateChunkVector skips
it, and a caption edit updates its Content only.
- The multimodal task's caption/OCR index rows now set IsEnabled; only
backends with a column default were storing them as enabled.
Retrieval
- A text query scores lower against an image than against text, so when
the model embeds images the document vector search is widened by half
(capped at 500) and run at min(VectorThreshold, 0.1); each store
group's hits are then held to their own threshold before score
normalization: text at VectorThreshold, images at min(VectorThreshold,
0.1). Keyword hits on image rows are dropped, their Content being the
already-indexed caption.
- image_vector hits are searchable and merge like caption hits: parent
context is attached, and the caption is not repeated from ImageInfo.
Adds golang.org/x/image for scaling and WebP/BMP decoding.
* fix(models): report no thinking levels for embedding, rerank and ASR models
Embedding rows now carry capabilities, and the end-to-end run showed them
listing off/auto/minimal/low/medium/high: these references resolve no chat
protocol, and an empty level map reads as "every level supported". A model
without a chat protocol does not think, so its capabilities say so.
* fix(compaction): never read a cancelled turn's stream as a summary
streamSummary selects on the turn's context and the provider stream at
once. When the turn is already cancelled both are ready (providers close
the stream once their context ends), and select picks at random, so a
stopped turn could come back as a completed, partial summary with a nil
error. TestSummarizationIsNotRetriedOnceTheTurnIsCancelled caught it
intermittently in CI and reproduces under -race with several -cpu values.
Check the context before each receive and again when the stream closes.
* feat(knowledge): put image vectors behind a knowledge-base switch, off by default
Image vector indexing and recall used to turn on whenever the knowledge
base's embedding model declared image input. A model that takes images is
often chosen for text alone, so an upgrade would have added an embedding
call per image, widened vector recall 1.5x and lowered the image threshold
to 0.1 for those knowledge bases without anyone asking.
ImageProcessingConfig gains image_vector_enabled (default false; stored
JSON without the key reads as off, so existing knowledge bases stay off
with no migration). Both sides now need the switch and the model:
- indexImageVector returns before touching the model unless the KB opted
in (KnowledgeBase.IsImageVectorEnabled, which also needs vector indexing).
- applyImageRecall resolves the embedding model only when some KB in the
search opted in, and records the opted-in KBs per store group
(storeGroup.ImageKBIDs). Only those groups widen the pool and lower the
threshold; image hits of any other KB, including ones indexed before the
switch was turned off, are dropped.
The KB editor shows the switch in the image processing section, with a
warning when the selected embedding model does not declare image input;
docs describe the switch, its cost and what turning it off does.
* fix(knowledge): leave image vectors behind when cloning into a KB that is off
CloneChunk copied every chunk of the source document and then CopyIndices
for all of them, so copying a document from a knowledge base with image
vectors on into one with the switch off (same model and store) still
created image_vector chunks and their index rows, bypassing the switch
that indexImageVector honours.
Image vector chunks are now dropped from the copy when the target has not
opted in. Every engine's CopyIndices copies only the rows of chunks in the
mapping, so their index rows stay behind too.
* fix(search): refill the vector pool past stale image vectors
Image hits of a knowledge base whose switch is off were dropped only after
the engine had ranked and cut its results at TopK. Image vectors indexed
before the switch was turned off could fill that pool and push out valid
text hits: with MatchCount 5 the pool is 50, and 50 stale image rows
ranked above the text left the search empty.
No engine can filter rows by source type (Postgres and SQLite do not even
take ExcludeChunkIDs), so the exclusion cannot move into the query. When
the document vector pool comes back full and holds stale image rows, that
search alone runs again on a pool twice as large, until it holds TopK rows
that are not stale, the index runs out, or the pool reaches
maxRetrievalPoolSize (at most four extra calls from a pool of 50). The pool
is then cut back to TopK such rows, matching an engine-side exclusion
unless more than maxRetrievalPoolSize-TopK stale rows rank above the text.
* feat(i18n): translate the image vector switch and image input keys into zh-TW
The zh-TW locale arrived from main after this branch added five keys: the
knowledge base's image vector switch (label, description, unsupported-model
warning) and the embedding model's image input flag (label, description).
* fix(knowledge): leave image vectors behind when moving into a KB that is off
A reuse_vectors move relocates a document's chunks and index rows in place:
MoveKnowledgeIndices rebinds the rows by document ID and
MoveChunksByKnowledgeID rebinds the chunks. Moving a document from a
knowledge base with image vectors on into one with the switch off therefore
carried its image_vector chunks and their vectors along, the same bypass the
clone path had.
When the target has not opted in, the image_vector chunks are now deleted
during the move, their index rows first and then the chunks, so a retry
finds the chunks again rather than leaving orphaned rows. A target that
opted in still receives them unchanged.
* fix(retriever): keep image vectors out of keyword-only engines
BatchIndexVectors handed image rows to every engine of the composite,
including one that serves keyword retrieval alone (Elasticsearch v7 next to
a vector store). There the row holds nothing but the image's caption, which
the caption's own chunk already indexes, and retrieval discards every
keyword hit on it.
Image rows now go only to engines that serve vector retrieval. An engine
that serves both keeps one row for both kinds, so its keyword search still
meets them; the search side handles that.
* fix(search): refill the keyword pool past image rows
Every engine that serves both kinds of retrieval stores one row per index
entry and runs its keyword search over that row's content, so an image
vector row, whose content is its caption, is a keyword hit wherever the
caption's own chunk is. filterImageHits drops all of them, but only after
the engine has cut its ranking at TopK, so they crowded out text and caption
hits; a keyword ranking that put the image rows first came back empty.
The keyword pool is now refilled the way the vector pool is refilled past
stale image vectors: when it came back full and held image rows, the
keyword search runs again on a pool twice as large, up to
maxRetrievalPoolSize, and is cut back to TopK hits that are kept.
Leaving the rows out of the keyword index instead would mean writing them
without searchable content (Milvus and Tencent VectorDB derive a BM25
sparse vector from it, and how they take an empty one is unverified) or a
source-type filter in every engine's keyword query (ten engines, and
filterable fields that existing collections lack).
* revert: let image vectors move with the document into a KB that is off
This reverts commit e259977fe915f7d0bc30c4b02bc7af04da1c8dd4.
Deleting the image vector index rows by chunk ID before the move is not
reliable: the Weaviate and Elasticsearch v8 delete wrappers ignore partial
failures, and OpenSearch rejects more than 1000 IDs with ErrBatchTooLarge.
A partial delete followed by the chunk delete leaves index rows whose chunks
are gone, which nothing ever cleans up.
Without it, the image_vector chunks and their index rows move together
with the document. While the target's switch is off they are filtered at
query time (the pool refill keeps them from crowding out text), they are
removed with the document on delete, reparse or knowledge base delete, both
chunks and index rows by knowledge ID, and they become active again if the
target turns the switch on. The clone path keeps dropping them, since there
the rows are copied rather than moved.
* fix(search): keep the image row refill from losing hits it already had
refillPool replaced a pool with the answer to each larger search. A refill
that answered only in part (RetrieveResult.Error, swallowed into a warning
by retainPartialResults, as when some collections of the store did not
answer) could hold fewer text rows than the complete smaller pool, or none:
40 stale image rows above 10 text came back from a partial refill as the
40 image rows alone, and the search lost all 10 text hits. A refill that
failed outright failed the whole store group, discarding a first round that
had succeeded.
A refill is now best effort. A failed or incomplete refill search stops
the refill and leaves the last complete pool in place, cut back to TopK
kept rows as before, so a refill can no longer leave a search worse off
than it was without it. Merging the partial answer into the pool was not
done: its rows would need deduplicating and its ranking is cut at a
different TopK over a different set of collections, while keeping the
previous pool is exact.
* docs(knowledge): say what happens to image vectors in a KB that is off and what the refill costs
The image vector section now says that with the switch off, image vectors
that were indexed before it was turned off, or that came along when the
document was moved in from a KB that has it on, are kept but filtered at
query time, are removed when the document is deleted or reparsed or the
KB is deleted, and are recalled again once the switch is turned on; a copy
into such a KB leaves them behind.
It also states the cost of keeping dropped image rows from crowding text
out of the pools: the vector and keyword pools are searched again with
TopK doubled up to the 500 candidate cap, which adds queries for store
groups that hold such rows, and text can still be missed when more than
about 500 candidates are image rows. A failed or partial refill keeps the
previous complete pool.
* fix(search): log unexpected refill result sets, and describe the refill stop condition precisely
* fix(kb): let update API set vlm_config
UpdateKnowledgeBaseRequest only carried name/description/config, so the
PUT /knowledge-bases/:id endpoint could never change a knowledge base's
multimodal (VLM) config, even though CreateKnowledgeBase accepts a
top-level vlm_config. Thread an optional vlm_config through the handler
and service so the update path mirrors create (nil = no change).
Fixes#1723
* test(kb): pass the vlm_config argument in tests added on main
* fix(kb): accept only model-managed vlm_config fields on update
The update API copied the request's VLMConfig wholesale, so a caller
could set the legacy ModelName/BaseURL/APIKey/InterfaceType fields and
have the VLM client call BaseURL directly, outside model management and
its SSRF checks. Only Enabled/ModelID/DescriptionLanguage/
CustomInstructions are taken now; legacy fields keep their stored value.
A referenced model must exist and be a VLLM model (400 otherwise), and
disabling clears ModelID, matching the initialization config endpoint.
* fix(kb): validate and gate vlm_config on the update API
- Enforce the 4000-character custom_instructions cap
(types.ValidateKnowledgeBasePromptInstructions), as create and
/initialization/config already do.
- vlm_config is a KB setting: require kbSettingsAccess (owner workspace
or admin share) like /initialization/config, so share editors get 403.
- Stored legacy model_name/base_url keep IsEnabled() true on their own,
so disabling returned 200 but left VLM on, and managed -> disabled fell
back to the old endpoint. Clear them when the request enables a managed
model_id, or disables VLM without echoing back the stored
model_name+base_url (GET-then-PUT keeps legacy on). Request legacy
fields are only compared, never written. Enabling without model_id on a
non-legacy KB is now a 400 instead of a silent no-op.
- Document the vlm_config semantics in the KB API reference.
* fix(kb): keep the legacy VLM endpoint only for pure legacy knowledge bases
CreateKnowledgeBase stores vlm_config as sent, so a managed model_id can
sit next to legacy model_name/base_url; resolveVLM then uses the managed
model. The update's keep-legacy paths (disable echoing the stored legacy
fields, enabled=true without model_id) cleared model_id but kept the
legacy fields, so IsEnabled() stayed true and image calls moved to the
stored base_url.
Those paths now apply only when no model_id is stored. On a mixed config
a disable clears everything, and enabled=true without model_id is a 400,
the same as on any managed knowledge base.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(data-analysis): skip style-only worksheets before DuckDB loading
* chore: retrigger CI for XLSX sheet fix
* fix(data-analysis): keep all sheets when the worksheet preflight fails
The style-only sheet preflight is best-effort: if it cannot read the
workbook parts, log a warning and load the sheets DuckDB enumerated
instead of failing the whole workbook. Context cancellation still
aborts the load.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
CallTool kept only text and image content from a tool result. A tool
can also return audio, a link to a resource, or a resource embedded in
the result, and GitHub's MCP server returns files that way:
get_file_contents answers with a short message and the file as an
embedded resource. The agent only ever saw the message.
Convert audio, embedded resources (text or blob) and resource links,
recording the resource URI. The agent now follows the reference to a
text resource with its text, and shows links and binary resources as
references. Base64 data of any kind, not only images, is redacted from
the logged content items.
Endpoint.Do turns a non-2xx reply into &HTTPError{StatusCode, Body} and
drops the response headers. A caller that retries a 429 or 503 therefore
cannot see the provider's Retry-After and falls back to its own guess: the
agent retry loop waits 1s, 2s, 3s, which against a provider asking for, say,
20s spends every attempt inside the rate-limit window and then fails the
turn.
- HTTPError carries the reply's Header, and RetryAfter() parses the header
in both forms (delay-seconds and HTTP-date); absent, malformed or past
values give zero.
- The agent retry loop waits max(linear backoff, Retry-After), with the
vendor's value capped at 30s so a large Retry-After cannot hold a turn
indefinitely.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Reverts the eleven plugin system PRs merged on 2026-09-26 so they can be
reviewed before landing:
* a829d7cc9 agent tools served by plugins (#3733)
* 4f5d475c3 Jira example plugin (#3730)
* 01222aac8 dynamic form options, OAuth connections and resource limits (#3729)
* cf502ba46 event delivery and webhooks (#3728)
* 996696057 sandboxed plugin pages (#3727)
* da4ef68d9 standalone plugin hosts (#3725)
* 728007ada remote runtime, Python SDK (#3724)
* ff49f9491 Host API and document parsers (#3721)
* 4328ab26a code plugins (#3720)
* ed63e35cc declarative plugins (#3716)
* c91a8d11f plugin system foundation (#3711)
The tree is back to 47773b5bd, migrations included: no environment ran
the plugin migrations (versioned 000115-000119, sqlite 000034-000038).
* feat(plugin): add plugin manifest, registry and builtin catalog
* feat(plugin): add config schemas with unified secret handling
* feat(frontend): render config schemas with SchemaForm
* feat(datasource): serve connector credential forms from the backend
* feat(im): describe IM platforms and credentials from the backend
* feat(frontend): render model vendor extra fields from the config schema
* feat(frontend): build the settings modal from an extension registry
* refactor(agent): register builtin tools and their chat views
* feat(plugin): let workspaces enable and disable plugins
* feat(plugin): add the runtime driver interface and gather builtins
* feat(wiki): add cross-KB wiki search POST /wiki-search
Give external callers a knowledge-search counterpart for wiki pages: one
SQL IN query ranked by title/slug/summary/content, slim hits with
match_snippet, and Viewer retrieve auth including shared KBs.
* fix(wiki): satisfy golangci-lint on cross-KB wiki search
Check c.Error returns, wrap lines over 120 columns, and use wikiDialect
instead of the embedded Dialector selector.
* fix(wiki): address review on cross-KB search helpers
Share KB viewer auth and snippet extraction, keep ID limits in the
service layer, and escape SQLite LIKE wildcards.
* fix(lint): wrap knowledge span fixtures over 120 columns
Go Lint still diffs from the original PR base, so later main tests
in knowledge_spans_test.go were reported as new line-length issues.
* fix(wiki): authorize KB access before checking wiki enablement in cross-KB search
Checking IsWikiEnabled before AuthorizeKBAccess let a caller tell apart a
foreign KB with wiki disabled (400) from one it cannot see (404), leaking
the KB's existence and settings. Run the access check first.
---------
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
A stream body that ran out without a finish reason (a proxy cutting the
connection) was closed as a clean stop on the Completions, Responses and
Gemini paths, so a half-written answer was stored as complete. It is now
marked incomplete, as the Anthropic path already did, and both the chat
pipeline and the agent treat an incomplete answer as a stream error.
Retry classification checks typed HTTP/transport errors first and matches
status codes as whole numbers, so "max_tokens 5000" no longer reads as a
500. The retry backoff returns as soon as the turn is cancelled.
Parallel tool calls run on errgroup goroutines, and errgroup does not carry
a panic back to Wait, so one panicking tool (MCP client, sandbox, SDK) took
down the whole process. The registry now turns a panic into a failed tool
result, and the parallel path guards its own bookkeeping as a backstop.
* fix(retrieval): keep index writes and shared-KB lookups correct
Index writes
- Image OCR/caption chunks were indexed without IsEnabled, so every engine
but Postgres stored them disabled and retrieval never returned them.
The SQLite engine re-enables the affected rows at startup.
- FAQ edits, similar-question additions and import merges re-index under
the old SourceID and relied on the engine overwriting it. Postgres and
SQLite kept the stale row (ON CONFLICT DO NOTHING); ES v8 and Qdrant
mint a new document ID and kept both. FAQ re-indexing now deletes the
old rows first, and Postgres/SQLite BatchSave upserts by
(source_id, source_type).
- Postgres: drop the GORM default tag on is_enabled, which replaced false
with true on insert and indexed disabled chunks as enabled.
- SQLite: BatchSave no longer attaches vectors and FTS rows to the wrong
rows when part of a batch already exists, and vecTables is guarded
against concurrent map writes.
- CopyIndices: Postgres filters to the copied knowledge, pages by id and
keeps is_enabled; SQLite copies every row of a chunk (generated
questions included) with stable source IDs and enabled state.
Shared knowledge bases
- Chat merge: FAQ answer population and neighbor expansion look chunks up
without the caller-tenant filter, like parent resolution (#3342).
- Neighbor expansion never cuts the retrieval hit; the length budget goes
to the text adjacent to it.
- FAQ negative questions are applied whenever an FAQ KB is in scope and
to chunks of shared FAQ KBs.
- Search results get image_info for shared-KB chunks.
- search_knowledge reads FAQ metadata from the search result instead of
one tenant-scoped query per hit, so shared FAQs render their answers.
* fix(retrieval): put every retrieval score on one scale
Scores from different retrieval paths were ranked together while living on
different scales, so ordering depended on which path produced a hit.
Fusion
- RRF ranked a chunk by its position in the concatenation of every
vector (or keyword) list, appended in goroutine completion order. The
second list's best hit ranked behind every hit of the first, and the
order changed between identical requests. Each list is now ranked on
its own by score and a chunk takes its best rank.
- RRF is divided by its maximum, so hybrid scores land in [0, 1] like
vector-only (cosine) and keyword-only (rescaled BM25) results. Raw RRF
topped out near 0.016 and lost to any single-retriever hit it was ranked
against across searches (document targets, model groups, FAQ calls);
rerank's base-score term and MMR's relevance term had nothing to use.
- Keyword-only BM25 is rescaled per list; engines' BM25 is not comparable.
- FAQ iterative retrieval fuses each round like the first pass instead of
sorting raw BM25 and cosine together, starts at twice the depth already
searched, triggers when any single vector list is full (the old summed
count never matched with more than one list) and replaces results per
round.
Rerank and chat pipeline
- The FAQ boost is no longer capped at 1; capping made strong FAQs tie at
1.0 and fall to the tie-breaker. The composite score before boosts is
kept as composite_score.
- The FAQ direct-answer threshold is compared against that pre-boost
score, and the FAQ that cleared it is the one marked match="exact"
rather than the first FAQ.
- Graph hits no longer carry a fabricated retrieval score of 1.0, which
ranked them above every scored hit when rerank did not run; the
composite score uses the model score in its place.
Engines
- OpenSearch and Weaviate return (1 + cos) / 2; their drivers now report
cosine similarity and map thresholds onto the engine scale, so a
threshold means the same on every engine.
- Milvus L2 returned squared distance, ranked farthest-first downstream,
and took the threshold as a radius; scores and radius are converted.
- The multi-store normalizer no longer maps Milvus through (s + 1) / 2,
which put it on a different scale from every other engine and from its
own single-store results.
Docs updated for fusion, normalization, FAQ boost and iterative recall.
* perf(retrieval): bound rerank, agent and pipeline cost; stop failing on slow or missing models
Rerank
- Rerankers report the longest document a request can carry
(max_document_chars / max_request_chars minus the query) and the rerank
stage fits its own passages to it, trimming appended captions, OCR and
generated questions first. One chunk with a large screenshot's OCR used
to exceed the vendor limit and fail the whole batch, so every candidate
lost its score and the threshold stopped applying.
- Rerank HTTP requests default to a 60 s timeout; chat and agent contexts
carry no deadline, so a stalled self-hosted rerank server held the turn
open.
- OCR and caption chunks no longer repeat their own text from image_info in
the passage.
- The chat pipeline and the agent tool rerank at most 200 candidates, best
retrieval score first.
- A rerank model that cannot be loaded degrades to retrieval order instead
of failing every chat turn, as the search APIs already do; without rerank
results the merge stage keeps only RerankTopK hits instead of enriching
every one.
Retrieval APIs
- knowledge-search match_count and rerank.top_k are capped at 200
(unbounded values multiplied into tens of thousands of billed rerank
passages); normalizedMatchCount clamps to the pool size so the
over-retrieval arithmetic cannot overflow.
- hybrid-search with rerank recalls at least rerank.top_k deep.
Agent
- search_knowledge fits its rendered output to four fifths of the tool
output budget, dropping the lowest-ranked rows and telling the model how
many it omitted; the model's view is rebuilt from Data, which registry
truncation never reached.
- Agent searches skip neighbor/parent/related enrichment rows, which were
never retrieved yet tripled the rerank input.
- ParallelToolCalls is enabled. The model was always told it may call tools
in parallel while the engine ran them one by one; only read-only tools
overlap, anything else stays a barrier.
Chat pipeline
- Query expansion searches keywords only (its variants are local rewrites
whose embeddings repeat the main search) and is skipped when keyword
matching is off.
- Entity extraction, a second LLM call, is skipped for intents that need no
KB retrieval and uses the rewritten query.
- Partial-overlap dedup tokenizes each text once instead of per pair, and
only compares token overlap between texts within 3x of each other's size.
Storage
- Migration 000112 adds btree indexes on embeddings(knowledge_id) and
(chunk_id); document deletes, chunk re-index and enable/tag updates
scanned the whole table.
- Shared-KB knowledge rows are fetched in one query instead of one per ID.
- Drop GORM Debug() from the ParadeDB keyword query and the knowledge batch
lookup, which logged SQL and user queries on every search.
Docs updated for the API limits, rerank timeout and passage fitting, agent
parallel tools and output budget, and pipeline changes.
* fix(retrieval): make agent tools and pipeline stages report what they did
Agent tools
- query_knowledge_graph now queries the graph store: entity names from the
query are matched in each KB's graph, relations are returned (and shown to
the model), and the entities' source chunks lead the results, with text
search as a supplement. It used to run text search only, so a graph-only
KB always answered "no relevant graph information".
- search_knowledge reports failures instead of "no matching chunks": when
every search fails the tool fails with the reason; partial failures are
listed. A failed query embedding falls back to keyword search for hybrid
KBs, as the chat pipeline does.
- Keyword mode uses rerank to order hits, not to reject them (identifiers
score low), and the empty-result hint no longer sends keyword mode back to
keyword mode.
- Dedup keeps siblings under one parent, no longer merges every image chunk
of a document through chunk_index 0, and keeps the better-scored copy.
- Search rows omit a <match> excerpt that only repeats the content; snippet
terms are segmented for Chinese queries and skip stopwords, and FAQ
matched questions are chosen by match strength.
- read_document query mode returns next_offset when truncated and resumes
from offset.
- wiki_search over-fetches under document/tag scopes before filtering and
says when every match was out of scope.
- Prompts point to read_document(id=cN, context=k) for surrounding text.
- The agent editor hides the retrieval-strategy section in agent mode,
where search_knowledge uses the global retrieval settings.
Chat pipeline
- Intent labels are normalized; an unknown label retrieves instead of
silently skipping the knowledge base, and web_search without web search
enabled searches the knowledge bases.
- Image turns get a 2048-token rewrite budget, and a reply cut off by the
cap is salvaged field by field instead of being discarded.
- History references are matched by how much of the query they cover;
Jaccard against chunk-sized text never reached its threshold.
- Entity search resolves chunks of org-shared KBs within the turn's KB
scope, skips chunks whose document is gone (it used to panic) and caps
graph chunks at 30.
- Empty history is not fetched twice; fetched pages are cut on a rune
boundary.
Retrieval
- A multi-KB search whose primary KB has no vectors embeds the query with a
vector KB's model instead of failing on an empty model ID.
- Rerank passages include the chunk's heading breadcrumb, which is part of
what was embedded.
- SQLite FTS searches the content column only and turns a lone Han
character into a prefix query; TencentVectorDB ef follows TopK.
Docs updated for the agent tools and pipeline stages.
* fix(retrieval): address review findings on the retrieval fixes
- Restore NeedsRetrieval for web_search without web search enabled: the
intent prompt "web_search" is how the user learns web search is off, and
running KB retrieval instead bypassed it.
- query_knowledge_graph keeps the case of entity terms; Neo4j matches node
names with case-sensitive CONTAINS, so lowercased terms never matched
English entities.
- An empty search_knowledge or query_knowledge_graph result states partial
failures in its text, which is all the model sees for an empty result.
- SQLite BatchSave replaces a source in one transaction, so a failed insert
keeps the old rows instead of leaving the source unindexed.
- Reuse stringSliceValue and an exported searchutil.ContainsChinese instead
of new copies; drop an unused named return and the FAQ obsolete-source
delete that the full per-chunk delete now covers.
* fix(retrieval): address second review of the retrieval fixes
- query_knowledge_graph: under a document/tag scope, keep only relations
whose entities come from an in-scope evidence chunk (the graph namespace
is the whole KB); drop chunks whose document no longer exists; report a
graph or text search failure even when the other source returned hits.
- search_knowledge: record the hybrid -> keyword fallback taken when the
query embedding fails in mode_fallbacks, so an empty result is not read
as "no semantic match".
- query understanding salvage: take a cut-off value only for the image
description; a truncated rewrite_query or intent is dropped instead of
replacing the user's query.
- Graph hits (no retrieval score) are kept past the no-rerank RerankTopK
cut and the rerank candidate cap instead of always falling below them.
- FAQ boost is capped at 1 again so scores stay on the [0, 1] scale; FAQs
tied at the cap are ordered by their pre-boost score.
- Negative-question filtering loads only candidates of FAQ knowledge bases.
- read_document query continuation starts one chunk before offset so the
first match keeps its preceding context at a page boundary.
- graphSearchTerms and searchQueryTokens share one queryTerms splitter.
* fix(desktop): keep Lite signed in with a stable JWT secret
Auto-setup fell through to the login form, and a newly generated signing key replaced the HMAC already used for presigned links. Reuse an existing HMAC, fill only a placeholder JWT secret, and clear a sticky auto-setup failure so the desktop window can enter the local account.
* fix(desktop): make wails dev start locally and store uploads at home
The dev app has no copied config, and the repo env asks for Redis and the Docker path /data/files. Stay on the repo config, use the in-memory stream manager when Redis is unset, write uploads to ~/.weknora/data/files, and allow dragging the top of the window.
* feat(lite): install and run skills on the host instead of a remote sandbox
Lite has no sandbox image, so skills are stored under ~/.weknora/skills as version directories with a live symlink, and chats execute them in the host Seatbelt. The settings UI no longer asks which sandbox to install into. The base profile opens /tmp, chat commands may use the network, and a failed ready-write rolls the symlink back to the previous version.
* fix(ci): align Lite skill checks with the JWT split and lint limits
The desktop signing test still expected the session secret to equal the AES key, and the skill settings source test still looked for the old i18n keys. Wrap the lines golangci-lint rejects so the PR checks can pass.
* fix(ci): separate multi-line test helpers for gofumpt
gofumpt requires a blank line after a multi-line function. The host skill fakes were still packed together, so lint failed on the first one.
* fix(lite): address review findings on host skill install
- Ask mode denies network again, and /tmp writes open only with a
writable root, so nothing runs unapproved.
- Lite still reports and accepts the workspace script switch; the
Skills page offers to re-enable it when it is off.
- The host process refuses symlinks the installer planted before it
clears the install cache or reads the env declaration.
- An existing signing or AES key stays the signer and is never copied
into signing.key; sessions then get their own jwt.key.
- Skill env capture writes under the host target on Lite.
- Sweep keeps the previous version and skips skills another process
holds the lock for.
- Terminal and desktop panels never resolve remote sandboxes on Lite.
- A detached install stream resubscribes instead of freezing.
- Upload migration follows the bundle name and the old relative path.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
A shared agent runs in its owner's workspace, so the execution tenant in
the turn context is the owner's. The local_browser tool built its device
scope from that tenant, while the browser is paired under the asking
member's own workspace, so every call from a shared agent found no device
and told the user to pair again.
Resolve the browser scope from the authenticated caller instead, both
when registering the tool and in its owner check. Agents in the caller's
own workspace are unaffected: without a tenant switch the caller and the
execution tenant are the same.
- Updated the BROWSERSKILL_EXTENSION_PATH in .env.example and Dockerfile.app to reference the new extension version 0.3.1.
- Modified localization files for English, Japanese, Korean, Russian, and Chinese to reflect the support for both Chrome and Edge browsers.
- Added checks for extension version compatibility in the BrowserConnectionSettings component to improve user experience.
This commit ensures that the application is aligned with the latest BrowserSkill extension and enhances the localization for better user guidance.
The flat schema advertised both max_tokens and max_text_chars, so models
sent the log field to observe. Keep a single character budget, map page
reads onto the extension token cap, and list each method's arguments.
* fix(browser): move BrowserSkill baseline to upstream main with merged patches
Upstream merged the navigation response grace (#291), the preview/focus UI
channel (#296) and popup attribution (#297). Pin both ends to commit c61eb7e4
and drop patches 02, 04 and 06. WeKnora now calls the official ui.task_preview
and ui.task_focus methods.
Patch 05 (last-tab lifecycle) is replaced host-side: before tool.tab_close the
gateway lists the Agent Window's tabs and, when the target is the only one,
creates an agent-owned about:blank tab through tool.tab_create so Chrome does
not remove the window and report a user interruption. Patch 01 (bounded CDP
reads and screenshot fence) is rebased onto the new baseline and remains the
only downstream extension change.
The real-extension test follows the upstream borrow semantics: an unowned tab
inside the Agent Window is rejected in place and borrowed after being moved to
a regular window.
* fix(browser): narrow the BrowserSkill read-reliability patch
Keep isAbortError strict and add isCaptureTerminalError for the capture
pipeline, so a renderer read timeout stops fallback reads but tool handlers
report it as cdp_failed instead of cancelled; the gateway no longer pauses the
task on a read timeout.
Drop the CDP-level screenshot fence and the 3 s screenshot deadline: upstream
PR #296 already bounds preview captures and fences them per attachment, and
tool screenshots are serialized by the daemon queue. Stop routing the focus
emulation toggle through the deadline so upstream's raw-command serialization
keeps holding. Remove the duplicate background rendering browser test; the
upstream background-execution and background-screenshot tests cover it.
* docs(browser): link the read-reliability patch to upstream PR #318
* fix(browser): keep BrowserSkill frame discovery consistent after a read timeout
Sync patch 01 with the follow-up pushed to upstream PR #318: a timed-out child
Target.setAutoAttach keeps its session and is retried from getFrameGraph, and
timed-out frame-tree, frame-owner and root auto-attach reads reject the frame
graph instead of letting the coordinator queue a snapshot behind them.
* style(browser): gofmt focus.go
* fix(browser): roll back the last-tab placeholder and surface renderer read timeouts
The placeholder about:blank tab created before tool.tab_close stayed behind
when the extension refused the close (unauthorized or borrowed tab) or the
call was interrupted. After a failed close the gateway lists the Agent Window
again and removes the placeholder while the target tab still exists; when the
extension already closed the target before the reply was lost, the placeholder
is what keeps the window open and stays.
Sync patch 01 with upstream PR #318: renderer reads on a tab are refused while
a timed-out read is still running in Chrome, and observation errors carry
data.reason = renderer_read_timeout. Map that reason to a model hint that
stops repeated reads on the unresponsive tab instead of the generic
inspect-and-retry advice for cdp_failed.
* fix(browser): remove remaining BrowserSkill patches
* fix(browser): add official extension link and setup guidance
* feat(sandbox): resolve workspace paths through WorkspaceLayout
Keep remote sessions on the /workspace contract, while file, shell, and
artifact code follow the session layout instead of hardcoded paths.
* fix(sandbox): fail closed on host layout lookup and restore the shell blacklist
Treat host vs remote as an explicit origin, refuse inspect outside ReadRoots,
and skip collection when the output tree is missing or equals the project root.
* fix(sandbox): close leftover fail-open holes in the layout path
Keep host copy, collection, and prompt text from falling back to /workspace when a lookup fails or an adapter returns unclean roots.
* fix(sandbox): satisfy golangci-lint on the layout helpers
Wrap new copy and test lines to 120 columns, and add the exported comments revive requires.
* fix(sandbox): gofmt the read_file layout copy wrap
* feat(sandbox): add host as an OS-enforced backend type
Host is not a named/pinnable config: each command is a fresh local process, unlike Docker, Cube, and E2B.
* feat(session): persist Lite host workspace directory
Empty means an auto-allocated folder. Standard edition leaves the column unused so the shared session model still loads. SQLite also gains the tenant skill tables Lite shell_exec already expects.
* feat(localsandbox): adapt the host OS sandbox to session interfaces
File tools stay outside Seatbelt and are bounded by PathGuard. A revoked project directory fails closed instead of allocating a replacement workspace.
* feat(sandbox): run unpinned Lite sessions on the host backend
Lite opts into host explicitly; web stays disabled when no named remote config exists. Host workspaces are not destroyed or auto-committed.
* feat(desktop): pick and persist host project directories
The native folder dialog is the authorization path; typed paths are rejected. settings.sandbox.host tells the client when this edition can use it.
* fix(sandbox): fail closed on Lite host layout lookup misses
Unpinned host sessions were described as /workspace when pin read or
sandbox resolve failed. Prompts and attachment staging now share the
same fallback so Lite never pretends the remote tree exists.
* fix(frontend): keep settings.sandbox.host in sync with the backend keys
TestDeploymentCapabilityKeysMatchFrontend requires the lists to match. Host stays fail-closed like docker so a missed probe does not show Lite-only UI.
* fix(lint): satisfy golangci-lint on the host sandbox wiring
Check c.Error, wrap lines over 120 columns, put context first on versionsWorkspace, and document exported adapter methods.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
* feat(localsandbox): add host OS sandbox core and macOS Seatbelt backend
Introduce a standalone localsandbox package so Lite can run agent
commands on the user's machine under Seatbelt, without linking this
into cmd/server or changing remote sandbox behavior.
* fix(localsandbox): stop post-wait Kill, deny credential files, and refuse swapped roots
Seatbelt was SIGKILL-ing the process group after every successful Wait,
credential write denies omitted literal matches for files, PathGuard
followed a replaced workspace root, and the login-shell test never ran -lc.
* fix(localsandbox): tighten app-data deny, env inherit, and post-wait Kill
Keep ask/full from silently widening to auto, stop inheriting host secrets,
and make session workspaces survive midnight so the Seatbelt kernel stays
fail-closed before adapter wiring.
* fix(localsandbox): pass gofmt, errcheck, and revive on the new package
CI now lints from the merge base, so the host sandbox needs formatted tests, checked Close() results, and exported comments.
* fix(localsandbox): refuse broad workspaces and stop leaking host secrets
Keep the host sandbox fail-closed before adapter wiring: home/Users/Volumes
cannot be the workspace, login shells no longer re-import rc secrets, and
Service applies the env allowlist on every Run. Also close the Lstat TOCTOU
and post-wait Kill window.
* fix(localsandbox): check Close() in seatbelt spawn tests
* fix(localsandbox): deny host temp dirs and refuse leftover wide workspaces
file-read* still left /tmp and /var/folders readable, and Build/Relax
accepted /private/var, /System/Volumes/Data, and ~/Library/** as a
workspace, which re-opened the private deny. Also re-open only ~/.local/bin.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(agent): end the turn when the completion cap truncates an answer
A round that came back with finish_reason=length and no tool calls used to
fall through analyzeResponse as non-terminal. With no tool calls to run,
appendToolResults pushed the half-written answer back into the history as a
plain assistant message with nothing telling the model to continue, so the
next round rewrote the answer from the top, hit the same cap, and repeated
until the round budget ran out or the user cancelled.
Three guards, each independently load-bearing:
- analyzeResponse gains a truncation case: deliver the partial text and end
the turn. A cap hit that produced no text at all has nothing to hand over
and defers to the existing bounded empty-content retry. No continuation
nudge is sent — it only grows the prompt with the discarded fragment, and
assistant prefill, the reliable form, is unavailable on most
OpenAI-compatible endpoints.
- A consecutive-length breaker stops the loop when truncation keeps landing
inside tool-call arguments, the one shape that still legitimately loops.
- The stuck-loop detector now counts an identical lack of content. A model
burning its whole budget on reasoning returns empty every round, which used
to land in the else branch and reset the counter, so the guard could never
fire on the case that needed it most.
The answer event and its Done marker carry a truncated flag through the SSE
metadata, and the chat UI shows the same style of notice it already uses for
fallback answers.
Fixes#3446
* fix(agent): treat unterminated reasoning as no answer and keep the truncation marker
Review follow-up on three ways the truncation path could still mislead.
An inline <think> block that the cap cut off never gets its closing tag, so
the pair regex left it whole and the round looked like it had produced an
answer. The truncation case then skipped the empty-content retry it exists
for and emitted the chain of thought as the final answer. StripThinkBlocks
now drops a dangling <think> through end of input, which is what the
streaming splitter already does with the same text, so the two agree on what
counts as an answer.
The truncation marker only rode on the SSE metadata, and a finished
conversation is rebuilt from agent_steps rather than replayed from stream
events. A reloaded, shared or re-opened turn therefore lost the notice and
its half-written answer looked complete. AgentStep carries the flag now — it
is a jsonb column, so no migration — and the frontend marks the rebuilt
answer event and message from it.
The consecutive-length guard ran before the round was recorded, so a final
round of "let me look that up" plus a truncated tool call handed that
preamble over as the final answer, with its tool calls never refused and the
preamble never retracted from the answer area. The guard now records the
round and fails its tool calls the way every other truncated round does, and
a response carrying tool calls never donates its text as the answer.
* fix(security): close the low-severity shared-space findings
Verified and fixed from the shared-space permission review:
- Embed channels kept knowledge_ids, tag_ids, mentioned_items, skill_names
and summary_model_id from anonymous visitors. Mentions are only scoped
for shared agents, so a visitor could query any KB (and pick any model)
of the channel workspace by ID. The payload now drops them; the widget
never sends them.
- PUT /knowledgebase/:kb_id/wiki/issues/:issue_id/status updated any issue
by ID. The update is now scoped to the path's KB (404 otherwise).
- A direct org invite let the inviter attach any user of the target
workspace (or any user by ID) as its representative, whose username and
email the roster then showed to all members. Direct adds attach no
representative, and the roster returns emails only for the caller's own
workspace.
- COS and KS3 dropped the bucket (and COS the region) segment when turning
a path into an object key, while the tenant check scans every segment:
cos://42/<region>/prefix/7/... passed as tenant 42 and read tenant 7's
object from a shared bucket. Both must now match the service's config.
- Shared agent listings returned the full agent to receivers. Prompts and
the creator's user ID are withheld; capabilities and scope stay visible.
- Temporary document uploads let the caller pick the parser engine (and
its credentials) of a shared agent's workspace; only the agent's rules
choose one there now.
- Suggested questions used request / message-snapshot knowledge_ids
without a permission check. Each document is now resolved to its KB and
checked like KB IDs, including a shared agent's scope.
- IM channels accepted any knowledge_base_id (and any agent ID on create).
Both must belong to the channel's workspace, the KB also to a restricted
API key's allow-list.
- KB-restricted API keys could list org-shared KBs outside their allow-list
in the organization views; those listings are now filtered.
- With enable_rbac=false, RequireKBAccess let cross-workspace denials
through. That switch is for roles inside a workspace; the guard now
always denies another workspace's KB, since handlers behind it load by
ID without a tenant filter.
- The org member limit was counted before and enforced apart from the
insert; it is now checked in the insert's transaction with the
organization row locked (SQLite serializes writers instead).
- The UpdateOrganization route comment and API doc said owner-only; the
service and UI allow org admins. Comment and doc corrected.
Documented, not changed: issued /r/ and presigned links stay valid until
they expire after a share is revoked.
* fix(security): address review of the low-severity shared-space fixes
- Embed payloads also drop question_origin. It selects a KB like an
@mention for agents that retrieve only when mentioned, and the widget
never sends it; the selection stays inside the agent's scope, but an
anonymous visitor should not pick among its KBs. suggestion_attribution is
kept (the widget sends it and it is validated).
- The receiver view of a shared agent also withholds the follow-up
suggestion instruction, a prompt like the others. Starter questions are
shown to receivers anyway, and the sharer's name is displayed by design.
- Follow-up suggestion generation moves into the agent's workspace with
types.WithExecutionTenant instead of a bare tenant rewrite, so the
caller is pinned before the document permission check runs. The auth
middleware and the shared run already pinned it; this removes the
dependency on that.
- KS3 CopyFile copies only from the service's own bucket (as COS and S3
do); another bucket's path is a cross-backend copy.
Tests cover the dropped question_origin, the follow-up instruction and the
/shared-agents listing, a follow-up context with the tenant switched, a
repeated wiki issue status update, and the KS3 copy source.
* feat(agent): size agent history by the window and calibrate token estimates
Agent history was capped at history_turns (default 5) because compaction
could not survive a turn. With checkpoints persisted, load history by
tokens instead, and let compaction bound the session.
Budget. The loader drops the oldest turns that do not fit, and what it
drops is lost: neither replayed nor summarized. A budget equal to the
compaction threshold (the earlier attempt, since reverted) let the loader
trim first, so whenever one turn was larger than the system prompt plus
the new question the request never crossed the threshold; nothing was
summarized and the session became a sliding window with no checkpoint.
agent.HistoryTokenBudget is now the whole window: overflow reaches the
first round's compaction, which summarizes it into a new checkpoint.
A turn-by-turn regression test drives the real loader, budget and
compactor and fails as soon as a stored turn after the checkpoint is
missing from the loaded history; restoring the old budget fails it.
Summarizer input. History can now reach the window, so the part being
summarized can outgrow one request. An overflowing request degrades to a
raw archive that is never persisted, so every later turn would fail the
same way. Prepare keeps the newest messages that fit, the prompt states
how many were left out, and file paths still come from all of them.
Calibration. cl100k counts Chinese at about a token a character where
Qwen and DeepSeek spend about 0.6. The estimator now carries a scale
(clamped to 0.5-1.5, text only), learned from consecutive requests in a
turn: they share tools, system prompt and earlier messages, so the
provider's growth between them prices exactly the appended messages,
without mixing in how a provider renders tool schemas. The turn's scale
is stored on its usage (context_token_scale); the next turn's loader
prices history with it and hands it to the engine. Loading, the first
compaction check and compaction itself share one estimator, so they
agree; calibrating only the trigger would let the loader drop turns
again, which the regression test covers.
Also: history is priced as the engine sends it (redacted KB/Wiki results
cost one line); a checkpoint older than the rows read is matched by the
assistant rows' (created_at, id) order; a read ending on a turn's
question keeps that turn; a compaction discarded for freeing under 5%
still persists its checkpoint; a checkpoint write matching no row is an
error; a partial index backs the newest-checkpoint lookup. Agent mode no
longer reads history_turns: the field leaves AgentConfig, the editor
hides it in smart-reasoning mode, and quick-answer keeps its turn cap.
* fix(agent): address review of window-sized history loading
Summarization. The summarizer call had a fixed 60s total timeout. With
history loaded up to the window, a request could carry ~175k tokens and be
cut off while prefilling or writing normally; the degraded result left no
checkpoint, so the next turn reloaded the same history and failed the same
way. Summarization now streams and is cancelled only after no output for the
engine's stall timeout (120s by default, LLMCallTimeout overrides), with the
overall ceiling left to the transport, as for the engine's own rounds and as
in Codex and Pi. A stopped turn is not retried. When the summarizer still
fails, the raw archive is persisted as a checkpoint marked degraded so the
loop is broken, and the next compaction refines it as the previous summary.
The archive is held to the summary budget, newest messages first, so a
degraded compaction always frees room.
Loading. The loader priced turns as sent but returned and held them as
stored, so wiki pages stored in full and sent as one line could pile up in
memory. It now returns the as-sent history and releases each turn's stored
rows once replayed, keeping about a page of rows at a time.
Index. The partial checkpoint index is replaced by a composite
(session_id, created_at DESC, id DESC) index serving the backwards page,
the newest-checkpoint lookup and the recent-messages reads. It is built with
CREATE INDEX CONCURRENTLY so writes to messages are not blocked; the
migration holds that single statement, since a multi-statement query runs as
an implicit transaction. A PostgreSQL integration test, skipped without
WEKNORA_MIGRATION_TEST_POSTGRES_DSN, applies the chain, runs 000106 down and
up through golang-migrate, and checks both queries use the index without a
sort; the SQLite migration test checks the same plans.
Calibration. It no longer waits for a total token count, and a usage with
only a scale is persisted. A turn that measures nothing carries forward the
scale it started from, so the newest turn always has the latest one.
* feat(agent): persist compaction checkpoints across turns
Agent compaction only lived inside the turn that ran it: the summary was
discarded at turn end and LoadAgentHistory rebuilt raw history from the
messages table, so every later turn past the threshold paid for another
summarization of nearly the same history.
A compaction whose history summary ends exactly on a stored turn is now
written onto that turn's assistant message (messages.context_checkpoint).
LoadAgentHistory starts from the newest checkpoint: the summary replaces
that turn and everything before it, and later turns are replayed verbatim.
The compactor already treats a leading summary as the previous summary, so
checkpoints fold forward incrementally.
History messages carry their turn's assistant message ID (chat.Message
TurnID, engine-internal) so compaction can tell whether its cut lands on a
stored-turn boundary. Nothing is persisted when the cut ends inside a stored
turn, covers only the live turn, or the history summary degraded to a raw
archive; a split turn's prefix summary is never persisted because that turn
is replayed whole next time. Writes are column-scoped and best-effort, and a
failed lookup falls back to history without a checkpoint.
Storing the checkpoint on the covered turn keeps the boundary implicit, so
session forks copy it with the turn and deleting the turn drops it. The
HistoryTurns cap still applies, now to the turns after the checkpoint.
* test(skill): wait for the background install in the name-conflict test
InstallSkill queues the install on a goroutine, and the fixture row already
starts at installing, so reading the status right after the call raced the
mock install and failed whenever it landed on ready first. Wait for the run
to finish on the reused row instead, as the other install tests do.
* feat(agent): size agent history by tokens instead of turn count
Agent history was capped at history_turns (default 5) because every turn
used to rebuild raw history and compaction could not persist. With
compaction checkpoints persisted, the cap is no longer what keeps a session
inside the window, and it threw away context that would have fit.
LoadAgentHistory now loads the newest checkpoint summary plus the newest
turns after it that fit a token budget: the run's compaction threshold
(agent.HistoryTokenBudget, the window less the reply reserve). History
that large already triggers the first round's compaction, which persists a
new checkpoint, so compaction rather than a turn count bounds the session.
Loading more would only hand that compaction a summarizer input larger
than the model can read.
Rows are paged backwards by (created_at, id) and the read stops at the
checkpoint or once the budget is full, so a long session is never read
whole. Kept turns stay contiguous, the newest turn is kept even when it
alone exceeds the budget (compaction splits it), and a turn cut by the read
is left out rather than replayed without its original question.
Agent mode no longer reads history_turns: AgentConfig.HistoryTurns is
removed, the editor hides the input in smart-reasoning mode and explains
the automatic behavior, and the smart-reasoning builtins drop the setting.
Quick-answer (KnowledgeQA) keeps its turn cap since it has no compaction.
* Revert "feat(agent): size agent history by tokens instead of turn count"
This reverts commit d05fb6e5f1f161813a88e9127609f7a059f38f12.
Loading history up to the compaction threshold lets the loader drop the
oldest turns before compaction can trigger: whenever a single turn's
replay is larger than the system prompt plus the new question, the
trimmed history stays under the threshold, nothing is summarized, and the
session degrades into a large sliding window that never gets a
checkpoint. A turn-by-turn simulation of LoadAgentHistory with the real
compactor showed zero compactions over 60 turns in those cases.
Keep the persisted checkpoints with the HistoryTurns cap for now; token
budgeted history returns in a separate change with a budget that leaves
the overflow to compaction.
* fix(agent): search bound knowledge bases before answering, and pass suggested-question origins
A user picked the suggested question "为什么在比较图形时需要谨慎处理?"
and the agent answered in round 1 with no tool call, claiming it had
searched the knowledge base and found nothing, then explained misleading
statistical charts. The question had been generated from "Corners Don't
Look Like That: Regarding Screenspace Ambient Occlusion" in a bound base,
where "comparing graphs" means SSAO falloff curves.
Two things let that happen: the grounding prompt said "Stable general
explanations need no lookup", and nothing told the agent where the
question came from - the base in question had no profile, only two
recent titles in the directory, and the click sent the text alone.
Prompt
- With knowledge tools available, search bound bases before answering a
question about a topic, even one that reads like general knowledge;
directory profiles and recent titles cannot prove a base lacks it.
- The general-explanation exemption applies only without bound bases.
- Say you searched or found nothing only when a search tool ran.
Question origin
- Suggested questions from FAQ/document chunks also return knowledge_id.
- Picking one on the home or chat page sends question_origin
{knowledge_base_id, knowledge_id} with that question only.
- The service keeps the origin only when its base is inside the turn's
resolved scope and the document belongs to that base; it never
changes the scope (unlike knowledge_base_ids, which narrows it).
- The engine renders it in runtime_context as <question_origin> with the
base and document handles and a note to search it before answering.
* fix(agent): keep question origins across file/tag scopes and bind them to the send
Review follow-up for the suggested-question origin.
- Scope check: the origin was kept only when its base was in the resolved
KB list, which is empty for file-only and tag-only scopes even though
their bases are searched. It is now checked against SearchTargets after
they are built; the document is kept only when a target for that base
covers it (unfiltered base, or a document scope listing it).
resolveQuestionOriginInfo re-checks SearchTargets, looks up the name of
a base that is not in kbInfos, and the renderer omits an unknown name.
- Frontend: the origin travelled through module-level "pending" state, so
a send that returned early (agent not ready, upload in progress,
steering) left it behind and a second pick overwrote it. It now rides
with the one send: triggerSend(text, options) -> createSession ->
send-msg -> sendMsg / changeFirstQuery.
- RetrieveKBOnlyWhenMentioned: suggestions are still generated from the
agent's bases, but picking one retrieved nothing. The origin now
selects its base, as an @mention would, when the agent may read it.
- API sync: question_origin in docs/api/chat.md and swagger;
QuestionOrigin on the Go SDK's KnowledgeQARequest/AgentQARequest and
knowledge_id on SuggestedQuestion.
Repeating the same query in another mode is usually wasted, but keyword
mode can still recall identifiers that hybrid never surfaced. Stop telling
the model that switching mode cannot help.
The agent searched seven times for "Echo 开放权重模型 降低成本". Hybrid
retrieval ranked the right chunk first (RRF rank 0) every time, but the
rerank model saw only the chunk text, which never names Echo or cost;
it scored 0.0025 and all 18 candidates fell below the 0.15 fallback
floor. The tool reported "No matching chunks", and the model cycled
hybrid -> semantic -> keyword, which cannot help since every mode shares
the same rerank check. Scoring the same chunk with its document title
("Show HN: Echo - ... at 1/3 the cost using open-weight models") gives
0.45, and 0.55 for the query written as a question.
- Rerank passages are title + enriched chunk (rerankPassage); FAQ chunks
keep their own question, and the pattern filter still sees chunk text
only.
- When retrieval found candidates but the rerank model rejected all of
them, the empty statement says so and that switching mode will not
help, instead of "No matching chunks".
- The tool description, query schema and agent system prompt ask for a
complete natural-language question in hybrid/semantic mode rather than
a keyword list, and drop "switch mode" as the retry advice. Every
keyword engine matches any token (OR + BM25), so a question costs the
keyword leg no recall; on the same 20 bases the Echo chunks rank the
same for both query styles.
read_document's in-document query was matched as one literal string, so a
keyword query such as "sunflower f(n,k)" found nothing in a 708-chunk
document even when both words were there. The zero-match result gave the
model no next step, and the UI rendered it as "0 / 708 chunks loaded",
which reads like a failed load.
Paging mode also had no size bound: the registry truncates
ToolResult.Output, but the model sees the result rendered from Data, so a
limit=100 page reached ~68k characters in the model context.
- A plain query is split on whitespace; a chunk matches when it contains
every word, in any order. regex=true still matches one expression.
- A zero-match result carries a <hint> suggesting fewer/different words
or reading a chunk from search_knowledge results.
- A page stops adding chunks at 80% of the output budget (counting
per-chunk rendering overhead); next_offset resumes at the first chunk
not returned, and an oversized first chunk is still returned.
- The UI summary shows in-document search results ("no matches" /
"N matches") and, for offset paging, the chunk range instead of a page
number that no longer holds.
* refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents
The agent's retrieval surface had grown to five overlapping tools whose
behaviour was scattered across long descriptions, interleaved "tips" in
tool output (most of which modelcontext discarded before the model saw
them), and a grep_chunks implementation that ran an unindexed regex scan
over the chunks table ordered by created_at, so large knowledge bases lost
recall and paid a full-table scan on every call.
Tool surface
- search_knowledge replaces knowledge_search and grep_chunks: one query,
mode = hybrid | semantic | keyword, optional knowledge_base_ids, limit.
Keyword mode runs through the retrieval engine's keyword index instead of
the DB regex scan. Modes unavailable for the scope are refused with a
reason. Rerank, MMR and de-duplication are kept; already_seen markers,
retrieval_statistics and the zero-result "CRITICAL" block are gone.
- read_document replaces list_knowledge_chunks, get_document_info and
wiki_read_source_doc: id accepts a dN or cN handle, always returns the
document header, pages by exact offset (fixing the old floor-to-page
bug), finds passages with query (literal by default, regex opt-in) and
reads neighbours around a chunk with context.
- list_documents pages through one knowledge base, honouring @file/@tag
scope.
- wiki_search takes a single query plus regex, knowledge_base_ids, limit;
literal queries are escaped so the model never reasons about JSON
escaping. Legacy queries / knowledge_base_id are still accepted.
- query_knowledge_graph is registered only when a graph-enabled KB is in
scope and its description/tips were trimmed.
Compatibility
- Retired names in stored allowlists, presets and API calls are mapped to
their successors by NormalizeAllowedTools at registration time; no data
migration. Stored histories with old names still decode, render and
redact. Calls to retired names return a replacement hint.
- modelcontext gains an "id" key space that decodes against both the chunk
and document handle tables (decode-only), plus rendering for the
document header, chunk roles, in-document matches and next_offset.
Prompts and surfaces
- The RAG, wiki researcher and hybrid templates now name the tools and the
search -> read -> cite order; tool descriptions are short and contain no
workflow instructions.
- Engine display names, history redaction, grounding prompt, IM rendering,
the built-in MCP server (search_knowledge gains mode/limit, grep_chunks
becomes keyword search, read_document gains query, wiki_search gains
regex), frontend editor/renderers/i18n, presets, website docs and the
changelog are updated accordingly.
Tests cover name mapping, mode validation and refusal, output shape,
read_document paging/context/query/scope, list_documents, wiki_search
escaping and legacy arguments, modelcontext decoding and rendering, and
registration-time aliasing and graph gating.
* fix(agent): address review findings on read_document, list_documents and model rendering
- read_document: fetch the second page whenever the first page did not
reach the end of the document, so an unaligned window near the end no
longer drops the trailing chunks (also fixes context windows around a
chunk near the end); a query passed together with a cN handle now
searches the owning document instead of being ignored; query mode stops
collecting once the output budget is nearly spent and reports truncated;
the document row is authorized in place instead of being fetched twice;
the context read takes the total from the window fetch instead of a
separate count query.
- list_documents: resolve the page's document scope in one batched tag
lookup instead of one call per row.
- modelcontext: render read_document output once and read next_offset
through the existing intValue helper.
- search_knowledge: point title/file-name lookups at list_documents.
- Tests for the boundary window, chunk context near the end, cN + query,
and budget-bound query results.
* fix(agent): close compatibility gaps in the consolidated retrieval tools
- Wiki-only scopes: read_document and list_documents are registered
whenever a wiki KB is in scope, since every KB stores chunks. Their
capability requirement includes wiki but is marked auxiliary, so they do
not widen a RAG agent's derived "all knowledge bases" filter to wiki-only
bases (Go and frontend derivations both honour it).
- search_knowledge resolves the retrieval mode per knowledge base instead
of refusing the call: FAQ and vector-only bases answer keyword requests
semantically, keyword-only bases answer semantic requests by keyword.
Results carry requested_mode and mode_fallbacks, the empty-result text
names the fallback, and the call is refused only when no base in scope
has a chunk index.
- The model rendering uses the mode the search actually ran with (it was
always labelled semantic) and emits <mode_fallback> per degraded base.
- list_documents applies the @file/@tag scope before paging: tags are
pushed into the database filter, pinned documents are loaded by ID, and
total/next_page count only readable documents.
- read_document resolves chunk context by chunk_index through a new
ChunkRepository.ListChunkNeighbors, so parent, summary and image chunks
sharing the index sequence no longer shift the neighbourhood; offset is
documented as a reading-order position, not a chunk index.
- wiki_search (agent and MCP) keeps its regular-expression default and
only matches literally when the text is not a valid expression;
regex=false forces a literal match and regex=true rejects invalid input.
- MCP grep_chunks keeps grep semantics: literal terms from the pattern
fetch candidates from the keyword index (semantic for bases without
one), and every returned chunk is verified against the pattern.
* docs(changelog): align the consolidation entry with regex wiki_search and grep_chunks semantics
* docs(mcp): note that grep_chunks recall is bounded by index candidates
* feat(knowledge): derive knowledge base descriptions from document profiles
Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.
Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.
Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.
Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.
Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.
Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.
* fix(knowledge): keep generated descriptions out of duplicates and refresh clones
DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.
CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.
* fix(knowledge): refresh the loaded clone target and carry document profiles
ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.
cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
* refactor(wiki): consolidate source_refs parsing and resolution
source_refs entries ("uuid" or "uuid|title") were parsed by ten
independent implementations that disagreed on whitespace handling, on
whether a model-supplied title could be trusted, and on how a leading
pipe was treated. The scope-enforced and legacy agent resolvers also
duplicated the lookup and the routing step looked every document up a
second time.
- types.ParseWikiSourceRef / FormatWikiSourceRef / WikiSourceRefMatchesKnowledge
are now the only readers and writer of the format.
- resolveWikiSourceDocuments replaces resolveAuthorizedSourceRefs,
resolveSourceRefs and wikiKnowledgeBasesForSourceRefs: one lookup per
document, titles always rebuilt from server data, dedupe by canonical ID,
unknown documents fail closed in every mode.
- removeSourceRef now matches padded or titled refs of a deleted document.
* refactor(data_analysis): expose the selected document as a fixed table name
The tool required the document ID inside the SQL text and blindly
string-replaced it with the physical DuckDB table name, which forced the
model-context codec to rewrite handles inside quoted SQL and let the
UUID-derived k_<uuid> table name reach the model through stored schema
summaries.
The document is now selected only by knowledge_id. Every query runs on a
dedicated connection where "dataset" is a temporary view over the
physical table (DuckDB temporary objects are connection-local, so
concurrent sessions stay isolated). The physical name remains accepted
for internal callers; any other table fails validation with an error
naming the table the model should use. Schema summaries, the schema
tool, the analyst prompt and the tool schema all name the same table, so
data_analysis.sql no longer carries source handles and its text codec
policy is removed.
* feat(modelcontext): report raw identifiers that survive encoding
Text-level compaction exists because raw UUIDs reach model context through
producers the registry never sees: error strings, slugs, results of
unknown shape. Retiring it requires knowing where those producers are.
Registry.LeakedIdentifiers scans an encoded message list for UUID-shaped
values in every application-composed field (user messages are skipped),
classifies each as registered (codec missed a rewrite) or unregistered
(producer emitted a raw ID), and SummarizeLeaks renders one line per
field. The agent engine and the three RAG pipeline call sites log the
report as a warning after EncodeMessages.
* fix(chat_pipeline): align RAG data analysis with the fixed table contract
The RAG data-analysis plugin still asked the planning model to fill in
knowledge_id and to write SQL from a prompt that exposed the raw
knowledge UUID, then passed the model's JSON straight to Execute. Under
the new data_analysis contract a query that names anything but the
dataset table is rejected, and the plugin logged that rejection at error
level and moved on, so the analysis vanished without any signal. A
"no analysis needed" plan took the same error path.
The pipeline now owns document selection: the model returns only
needs_analysis and sql, the prompt names the dataset table and never
shows the knowledge ID or physical table, and the plugin builds the tool
input itself. A rejected query is fed back to the model once for repair;
a query that still fails is reported through pipelineWarn with the SQL
and the tool error, and "not needed" is logged as such instead of as a
failure.
Covered by unit tests with a scripted model and executor: not-needed,
pipeline-owned knowledge_id, one-shot repair, persistent failure
surfaced, unparseable plan rejected.