* fix(data-analysis): skip style-only worksheets before DuckDB loading
* chore: retrigger CI for XLSX sheet fix
* fix(data-analysis): keep all sheets when the worksheet preflight fails
The style-only sheet preflight is best-effort: if it cannot read the
workbook parts, log a warning and load the sheets DuckDB enumerated
instead of failing the whole workbook. Context cancellation still
aborts the load.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
CallTool kept only text and image content from a tool result. A tool
can also return audio, a link to a resource, or a resource embedded in
the result, and GitHub's MCP server returns files that way:
get_file_contents answers with a short message and the file as an
embedded resource. The agent only ever saw the message.
Convert audio, embedded resources (text or blob) and resource links,
recording the resource URI. The agent now follows the reference to a
text resource with its text, and shows links and binary resources as
references. Base64 data of any kind, not only images, is redacted from
the logged content items.
Endpoint.Do turns a non-2xx reply into &HTTPError{StatusCode, Body} and
drops the response headers. A caller that retries a 429 or 503 therefore
cannot see the provider's Retry-After and falls back to its own guess: the
agent retry loop waits 1s, 2s, 3s, which against a provider asking for, say,
20s spends every attempt inside the rate-limit window and then fails the
turn.
- HTTPError carries the reply's Header, and RetryAfter() parses the header
in both forms (delay-seconds and HTTP-date); absent, malformed or past
values give zero.
- The agent retry loop waits max(linear backoff, Retry-After), with the
vendor's value capped at 30s so a large Retry-After cannot hold a turn
indefinitely.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Reverts the eleven plugin system PRs merged on 2026-09-26 so they can be
reviewed before landing:
* a829d7cc9 agent tools served by plugins (#3733)
* 4f5d475c3 Jira example plugin (#3730)
* 01222aac8 dynamic form options, OAuth connections and resource limits (#3729)
* cf502ba46 event delivery and webhooks (#3728)
* 996696057 sandboxed plugin pages (#3727)
* da4ef68d9 standalone plugin hosts (#3725)
* 728007ada remote runtime, Python SDK (#3724)
* ff49f9491 Host API and document parsers (#3721)
* 4328ab26a code plugins (#3720)
* ed63e35cc declarative plugins (#3716)
* c91a8d11f plugin system foundation (#3711)
The tree is back to 47773b5bd, migrations included: no environment ran
the plugin migrations (versioned 000115-000119, sqlite 000034-000038).
* feat(plugin): add plugin manifest, registry and builtin catalog
* feat(plugin): add config schemas with unified secret handling
* feat(frontend): render config schemas with SchemaForm
* feat(datasource): serve connector credential forms from the backend
* feat(im): describe IM platforms and credentials from the backend
* feat(frontend): render model vendor extra fields from the config schema
* feat(frontend): build the settings modal from an extension registry
* refactor(agent): register builtin tools and their chat views
* feat(plugin): let workspaces enable and disable plugins
* feat(plugin): add the runtime driver interface and gather builtins
* feat(wiki): add cross-KB wiki search POST /wiki-search
Give external callers a knowledge-search counterpart for wiki pages: one
SQL IN query ranked by title/slug/summary/content, slim hits with
match_snippet, and Viewer retrieve auth including shared KBs.
* fix(wiki): satisfy golangci-lint on cross-KB wiki search
Check c.Error returns, wrap lines over 120 columns, and use wikiDialect
instead of the embedded Dialector selector.
* fix(wiki): address review on cross-KB search helpers
Share KB viewer auth and snippet extraction, keep ID limits in the
service layer, and escape SQLite LIKE wildcards.
* fix(lint): wrap knowledge span fixtures over 120 columns
Go Lint still diffs from the original PR base, so later main tests
in knowledge_spans_test.go were reported as new line-length issues.
* fix(wiki): authorize KB access before checking wiki enablement in cross-KB search
Checking IsWikiEnabled before AuthorizeKBAccess let a caller tell apart a
foreign KB with wiki disabled (400) from one it cannot see (404), leaking
the KB's existence and settings. Run the access check first.
---------
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
A stream body that ran out without a finish reason (a proxy cutting the
connection) was closed as a clean stop on the Completions, Responses and
Gemini paths, so a half-written answer was stored as complete. It is now
marked incomplete, as the Anthropic path already did, and both the chat
pipeline and the agent treat an incomplete answer as a stream error.
Retry classification checks typed HTTP/transport errors first and matches
status codes as whole numbers, so "max_tokens 5000" no longer reads as a
500. The retry backoff returns as soon as the turn is cancelled.
Parallel tool calls run on errgroup goroutines, and errgroup does not carry
a panic back to Wait, so one panicking tool (MCP client, sandbox, SDK) took
down the whole process. The registry now turns a panic into a failed tool
result, and the parallel path guards its own bookkeeping as a backstop.
* fix(retrieval): keep index writes and shared-KB lookups correct
Index writes
- Image OCR/caption chunks were indexed without IsEnabled, so every engine
but Postgres stored them disabled and retrieval never returned them.
The SQLite engine re-enables the affected rows at startup.
- FAQ edits, similar-question additions and import merges re-index under
the old SourceID and relied on the engine overwriting it. Postgres and
SQLite kept the stale row (ON CONFLICT DO NOTHING); ES v8 and Qdrant
mint a new document ID and kept both. FAQ re-indexing now deletes the
old rows first, and Postgres/SQLite BatchSave upserts by
(source_id, source_type).
- Postgres: drop the GORM default tag on is_enabled, which replaced false
with true on insert and indexed disabled chunks as enabled.
- SQLite: BatchSave no longer attaches vectors and FTS rows to the wrong
rows when part of a batch already exists, and vecTables is guarded
against concurrent map writes.
- CopyIndices: Postgres filters to the copied knowledge, pages by id and
keeps is_enabled; SQLite copies every row of a chunk (generated
questions included) with stable source IDs and enabled state.
Shared knowledge bases
- Chat merge: FAQ answer population and neighbor expansion look chunks up
without the caller-tenant filter, like parent resolution (#3342).
- Neighbor expansion never cuts the retrieval hit; the length budget goes
to the text adjacent to it.
- FAQ negative questions are applied whenever an FAQ KB is in scope and
to chunks of shared FAQ KBs.
- Search results get image_info for shared-KB chunks.
- search_knowledge reads FAQ metadata from the search result instead of
one tenant-scoped query per hit, so shared FAQs render their answers.
* fix(retrieval): put every retrieval score on one scale
Scores from different retrieval paths were ranked together while living on
different scales, so ordering depended on which path produced a hit.
Fusion
- RRF ranked a chunk by its position in the concatenation of every
vector (or keyword) list, appended in goroutine completion order. The
second list's best hit ranked behind every hit of the first, and the
order changed between identical requests. Each list is now ranked on
its own by score and a chunk takes its best rank.
- RRF is divided by its maximum, so hybrid scores land in [0, 1] like
vector-only (cosine) and keyword-only (rescaled BM25) results. Raw RRF
topped out near 0.016 and lost to any single-retriever hit it was ranked
against across searches (document targets, model groups, FAQ calls);
rerank's base-score term and MMR's relevance term had nothing to use.
- Keyword-only BM25 is rescaled per list; engines' BM25 is not comparable.
- FAQ iterative retrieval fuses each round like the first pass instead of
sorting raw BM25 and cosine together, starts at twice the depth already
searched, triggers when any single vector list is full (the old summed
count never matched with more than one list) and replaces results per
round.
Rerank and chat pipeline
- The FAQ boost is no longer capped at 1; capping made strong FAQs tie at
1.0 and fall to the tie-breaker. The composite score before boosts is
kept as composite_score.
- The FAQ direct-answer threshold is compared against that pre-boost
score, and the FAQ that cleared it is the one marked match="exact"
rather than the first FAQ.
- Graph hits no longer carry a fabricated retrieval score of 1.0, which
ranked them above every scored hit when rerank did not run; the
composite score uses the model score in its place.
Engines
- OpenSearch and Weaviate return (1 + cos) / 2; their drivers now report
cosine similarity and map thresholds onto the engine scale, so a
threshold means the same on every engine.
- Milvus L2 returned squared distance, ranked farthest-first downstream,
and took the threshold as a radius; scores and radius are converted.
- The multi-store normalizer no longer maps Milvus through (s + 1) / 2,
which put it on a different scale from every other engine and from its
own single-store results.
Docs updated for fusion, normalization, FAQ boost and iterative recall.
* perf(retrieval): bound rerank, agent and pipeline cost; stop failing on slow or missing models
Rerank
- Rerankers report the longest document a request can carry
(max_document_chars / max_request_chars minus the query) and the rerank
stage fits its own passages to it, trimming appended captions, OCR and
generated questions first. One chunk with a large screenshot's OCR used
to exceed the vendor limit and fail the whole batch, so every candidate
lost its score and the threshold stopped applying.
- Rerank HTTP requests default to a 60 s timeout; chat and agent contexts
carry no deadline, so a stalled self-hosted rerank server held the turn
open.
- OCR and caption chunks no longer repeat their own text from image_info in
the passage.
- The chat pipeline and the agent tool rerank at most 200 candidates, best
retrieval score first.
- A rerank model that cannot be loaded degrades to retrieval order instead
of failing every chat turn, as the search APIs already do; without rerank
results the merge stage keeps only RerankTopK hits instead of enriching
every one.
Retrieval APIs
- knowledge-search match_count and rerank.top_k are capped at 200
(unbounded values multiplied into tens of thousands of billed rerank
passages); normalizedMatchCount clamps to the pool size so the
over-retrieval arithmetic cannot overflow.
- hybrid-search with rerank recalls at least rerank.top_k deep.
Agent
- search_knowledge fits its rendered output to four fifths of the tool
output budget, dropping the lowest-ranked rows and telling the model how
many it omitted; the model's view is rebuilt from Data, which registry
truncation never reached.
- Agent searches skip neighbor/parent/related enrichment rows, which were
never retrieved yet tripled the rerank input.
- ParallelToolCalls is enabled. The model was always told it may call tools
in parallel while the engine ran them one by one; only read-only tools
overlap, anything else stays a barrier.
Chat pipeline
- Query expansion searches keywords only (its variants are local rewrites
whose embeddings repeat the main search) and is skipped when keyword
matching is off.
- Entity extraction, a second LLM call, is skipped for intents that need no
KB retrieval and uses the rewritten query.
- Partial-overlap dedup tokenizes each text once instead of per pair, and
only compares token overlap between texts within 3x of each other's size.
Storage
- Migration 000112 adds btree indexes on embeddings(knowledge_id) and
(chunk_id); document deletes, chunk re-index and enable/tag updates
scanned the whole table.
- Shared-KB knowledge rows are fetched in one query instead of one per ID.
- Drop GORM Debug() from the ParadeDB keyword query and the knowledge batch
lookup, which logged SQL and user queries on every search.
Docs updated for the API limits, rerank timeout and passage fitting, agent
parallel tools and output budget, and pipeline changes.
* fix(retrieval): make agent tools and pipeline stages report what they did
Agent tools
- query_knowledge_graph now queries the graph store: entity names from the
query are matched in each KB's graph, relations are returned (and shown to
the model), and the entities' source chunks lead the results, with text
search as a supplement. It used to run text search only, so a graph-only
KB always answered "no relevant graph information".
- search_knowledge reports failures instead of "no matching chunks": when
every search fails the tool fails with the reason; partial failures are
listed. A failed query embedding falls back to keyword search for hybrid
KBs, as the chat pipeline does.
- Keyword mode uses rerank to order hits, not to reject them (identifiers
score low), and the empty-result hint no longer sends keyword mode back to
keyword mode.
- Dedup keeps siblings under one parent, no longer merges every image chunk
of a document through chunk_index 0, and keeps the better-scored copy.
- Search rows omit a <match> excerpt that only repeats the content; snippet
terms are segmented for Chinese queries and skip stopwords, and FAQ
matched questions are chosen by match strength.
- read_document query mode returns next_offset when truncated and resumes
from offset.
- wiki_search over-fetches under document/tag scopes before filtering and
says when every match was out of scope.
- Prompts point to read_document(id=cN, context=k) for surrounding text.
- The agent editor hides the retrieval-strategy section in agent mode,
where search_knowledge uses the global retrieval settings.
Chat pipeline
- Intent labels are normalized; an unknown label retrieves instead of
silently skipping the knowledge base, and web_search without web search
enabled searches the knowledge bases.
- Image turns get a 2048-token rewrite budget, and a reply cut off by the
cap is salvaged field by field instead of being discarded.
- History references are matched by how much of the query they cover;
Jaccard against chunk-sized text never reached its threshold.
- Entity search resolves chunks of org-shared KBs within the turn's KB
scope, skips chunks whose document is gone (it used to panic) and caps
graph chunks at 30.
- Empty history is not fetched twice; fetched pages are cut on a rune
boundary.
Retrieval
- A multi-KB search whose primary KB has no vectors embeds the query with a
vector KB's model instead of failing on an empty model ID.
- Rerank passages include the chunk's heading breadcrumb, which is part of
what was embedded.
- SQLite FTS searches the content column only and turns a lone Han
character into a prefix query; TencentVectorDB ef follows TopK.
Docs updated for the agent tools and pipeline stages.
* fix(retrieval): address review findings on the retrieval fixes
- Restore NeedsRetrieval for web_search without web search enabled: the
intent prompt "web_search" is how the user learns web search is off, and
running KB retrieval instead bypassed it.
- query_knowledge_graph keeps the case of entity terms; Neo4j matches node
names with case-sensitive CONTAINS, so lowercased terms never matched
English entities.
- An empty search_knowledge or query_knowledge_graph result states partial
failures in its text, which is all the model sees for an empty result.
- SQLite BatchSave replaces a source in one transaction, so a failed insert
keeps the old rows instead of leaving the source unindexed.
- Reuse stringSliceValue and an exported searchutil.ContainsChinese instead
of new copies; drop an unused named return and the FAQ obsolete-source
delete that the full per-chunk delete now covers.
* fix(retrieval): address second review of the retrieval fixes
- query_knowledge_graph: under a document/tag scope, keep only relations
whose entities come from an in-scope evidence chunk (the graph namespace
is the whole KB); drop chunks whose document no longer exists; report a
graph or text search failure even when the other source returned hits.
- search_knowledge: record the hybrid -> keyword fallback taken when the
query embedding fails in mode_fallbacks, so an empty result is not read
as "no semantic match".
- query understanding salvage: take a cut-off value only for the image
description; a truncated rewrite_query or intent is dropped instead of
replacing the user's query.
- Graph hits (no retrieval score) are kept past the no-rerank RerankTopK
cut and the rerank candidate cap instead of always falling below them.
- FAQ boost is capped at 1 again so scores stay on the [0, 1] scale; FAQs
tied at the cap are ordered by their pre-boost score.
- Negative-question filtering loads only candidates of FAQ knowledge bases.
- read_document query continuation starts one chunk before offset so the
first match keeps its preceding context at a page boundary.
- graphSearchTerms and searchQueryTokens share one queryTerms splitter.
* fix(desktop): keep Lite signed in with a stable JWT secret
Auto-setup fell through to the login form, and a newly generated signing key replaced the HMAC already used for presigned links. Reuse an existing HMAC, fill only a placeholder JWT secret, and clear a sticky auto-setup failure so the desktop window can enter the local account.
* fix(desktop): make wails dev start locally and store uploads at home
The dev app has no copied config, and the repo env asks for Redis and the Docker path /data/files. Stay on the repo config, use the in-memory stream manager when Redis is unset, write uploads to ~/.weknora/data/files, and allow dragging the top of the window.
* feat(lite): install and run skills on the host instead of a remote sandbox
Lite has no sandbox image, so skills are stored under ~/.weknora/skills as version directories with a live symlink, and chats execute them in the host Seatbelt. The settings UI no longer asks which sandbox to install into. The base profile opens /tmp, chat commands may use the network, and a failed ready-write rolls the symlink back to the previous version.
* fix(ci): align Lite skill checks with the JWT split and lint limits
The desktop signing test still expected the session secret to equal the AES key, and the skill settings source test still looked for the old i18n keys. Wrap the lines golangci-lint rejects so the PR checks can pass.
* fix(ci): separate multi-line test helpers for gofumpt
gofumpt requires a blank line after a multi-line function. The host skill fakes were still packed together, so lint failed on the first one.
* fix(lite): address review findings on host skill install
- Ask mode denies network again, and /tmp writes open only with a
writable root, so nothing runs unapproved.
- Lite still reports and accepts the workspace script switch; the
Skills page offers to re-enable it when it is off.
- The host process refuses symlinks the installer planted before it
clears the install cache or reads the env declaration.
- An existing signing or AES key stays the signer and is never copied
into signing.key; sessions then get their own jwt.key.
- Skill env capture writes under the host target on Lite.
- Sweep keeps the previous version and skips skills another process
holds the lock for.
- Terminal and desktop panels never resolve remote sandboxes on Lite.
- A detached install stream resubscribes instead of freezing.
- Upload migration follows the bundle name and the old relative path.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* feat(retrieval): unify rerank across search APIs and expose diagnostics
The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.
- internal/reranking: one rerank stage (passage = title + cleaned body +
captions/OCR + generated questions, threshold with degradation and
top-1 fallback, composite score, MMR) used by the chat pipeline, the
agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
rerank model; an unknown requested model is a 400. Load or call
failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.
Refs #2976
* docs(retrieval): lead with which search API to use and when
knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
A shared agent runs in its owner's workspace, so the execution tenant in
the turn context is the owner's. The local_browser tool built its device
scope from that tenant, while the browser is paired under the asking
member's own workspace, so every call from a shared agent found no device
and told the user to pair again.
Resolve the browser scope from the authenticated caller instead, both
when registering the tool and in its owner check. Agents in the caller's
own workspace are unaffected: without a tenant switch the caller and the
execution tenant are the same.
- Updated the BROWSERSKILL_EXTENSION_PATH in .env.example and Dockerfile.app to reference the new extension version 0.3.1.
- Modified localization files for English, Japanese, Korean, Russian, and Chinese to reflect the support for both Chrome and Edge browsers.
- Added checks for extension version compatibility in the BrowserConnectionSettings component to improve user experience.
This commit ensures that the application is aligned with the latest BrowserSkill extension and enhances the localization for better user guidance.
The flat schema advertised both max_tokens and max_text_chars, so models
sent the log field to observe. Keep a single character budget, map page
reads onto the extension token cap, and list each method's arguments.
* fix(browser): move BrowserSkill baseline to upstream main with merged patches
Upstream merged the navigation response grace (#291), the preview/focus UI
channel (#296) and popup attribution (#297). Pin both ends to commit c61eb7e4
and drop patches 02, 04 and 06. WeKnora now calls the official ui.task_preview
and ui.task_focus methods.
Patch 05 (last-tab lifecycle) is replaced host-side: before tool.tab_close the
gateway lists the Agent Window's tabs and, when the target is the only one,
creates an agent-owned about:blank tab through tool.tab_create so Chrome does
not remove the window and report a user interruption. Patch 01 (bounded CDP
reads and screenshot fence) is rebased onto the new baseline and remains the
only downstream extension change.
The real-extension test follows the upstream borrow semantics: an unowned tab
inside the Agent Window is rejected in place and borrowed after being moved to
a regular window.
* fix(browser): narrow the BrowserSkill read-reliability patch
Keep isAbortError strict and add isCaptureTerminalError for the capture
pipeline, so a renderer read timeout stops fallback reads but tool handlers
report it as cdp_failed instead of cancelled; the gateway no longer pauses the
task on a read timeout.
Drop the CDP-level screenshot fence and the 3 s screenshot deadline: upstream
PR #296 already bounds preview captures and fences them per attachment, and
tool screenshots are serialized by the daemon queue. Stop routing the focus
emulation toggle through the deadline so upstream's raw-command serialization
keeps holding. Remove the duplicate background rendering browser test; the
upstream background-execution and background-screenshot tests cover it.
* docs(browser): link the read-reliability patch to upstream PR #318
* fix(browser): keep BrowserSkill frame discovery consistent after a read timeout
Sync patch 01 with the follow-up pushed to upstream PR #318: a timed-out child
Target.setAutoAttach keeps its session and is retried from getFrameGraph, and
timed-out frame-tree, frame-owner and root auto-attach reads reject the frame
graph instead of letting the coordinator queue a snapshot behind them.
* style(browser): gofmt focus.go
* fix(browser): roll back the last-tab placeholder and surface renderer read timeouts
The placeholder about:blank tab created before tool.tab_close stayed behind
when the extension refused the close (unauthorized or borrowed tab) or the
call was interrupted. After a failed close the gateway lists the Agent Window
again and removes the placeholder while the target tab still exists; when the
extension already closed the target before the reply was lost, the placeholder
is what keeps the window open and stays.
Sync patch 01 with upstream PR #318: renderer reads on a tab are refused while
a timed-out read is still running in Chrome, and observation errors carry
data.reason = renderer_read_timeout. Map that reason to a model hint that
stops repeated reads on the unresponsive tab instead of the generic
inspect-and-retry advice for cdp_failed.
* fix(browser): remove remaining BrowserSkill patches
* fix(browser): add official extension link and setup guidance
* feat(sandbox): resolve workspace paths through WorkspaceLayout
Keep remote sessions on the /workspace contract, while file, shell, and
artifact code follow the session layout instead of hardcoded paths.
* fix(sandbox): fail closed on host layout lookup and restore the shell blacklist
Treat host vs remote as an explicit origin, refuse inspect outside ReadRoots,
and skip collection when the output tree is missing or equals the project root.
* fix(sandbox): close leftover fail-open holes in the layout path
Keep host copy, collection, and prompt text from falling back to /workspace when a lookup fails or an adapter returns unclean roots.
* fix(sandbox): satisfy golangci-lint on the layout helpers
Wrap new copy and test lines to 120 columns, and add the exported comments revive requires.
* fix(sandbox): gofmt the read_file layout copy wrap
* feat(sandbox): add host as an OS-enforced backend type
Host is not a named/pinnable config: each command is a fresh local process, unlike Docker, Cube, and E2B.
* feat(session): persist Lite host workspace directory
Empty means an auto-allocated folder. Standard edition leaves the column unused so the shared session model still loads. SQLite also gains the tenant skill tables Lite shell_exec already expects.
* feat(localsandbox): adapt the host OS sandbox to session interfaces
File tools stay outside Seatbelt and are bounded by PathGuard. A revoked project directory fails closed instead of allocating a replacement workspace.
* feat(sandbox): run unpinned Lite sessions on the host backend
Lite opts into host explicitly; web stays disabled when no named remote config exists. Host workspaces are not destroyed or auto-committed.
* feat(desktop): pick and persist host project directories
The native folder dialog is the authorization path; typed paths are rejected. settings.sandbox.host tells the client when this edition can use it.
* fix(sandbox): fail closed on Lite host layout lookup misses
Unpinned host sessions were described as /workspace when pin read or
sandbox resolve failed. Prompts and attachment staging now share the
same fallback so Lite never pretends the remote tree exists.
* fix(frontend): keep settings.sandbox.host in sync with the backend keys
TestDeploymentCapabilityKeysMatchFrontend requires the lists to match. Host stays fail-closed like docker so a missed probe does not show Lite-only UI.
* fix(lint): satisfy golangci-lint on the host sandbox wiring
Check c.Error, wrap lines over 120 columns, put context first on versionsWorkspace, and document exported adapter methods.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
* feat(localsandbox): add host OS sandbox core and macOS Seatbelt backend
Introduce a standalone localsandbox package so Lite can run agent
commands on the user's machine under Seatbelt, without linking this
into cmd/server or changing remote sandbox behavior.
* fix(localsandbox): stop post-wait Kill, deny credential files, and refuse swapped roots
Seatbelt was SIGKILL-ing the process group after every successful Wait,
credential write denies omitted literal matches for files, PathGuard
followed a replaced workspace root, and the login-shell test never ran -lc.
* fix(localsandbox): tighten app-data deny, env inherit, and post-wait Kill
Keep ask/full from silently widening to auto, stop inheriting host secrets,
and make session workspaces survive midnight so the Seatbelt kernel stays
fail-closed before adapter wiring.
* fix(localsandbox): pass gofmt, errcheck, and revive on the new package
CI now lints from the merge base, so the host sandbox needs formatted tests, checked Close() results, and exported comments.
* fix(localsandbox): refuse broad workspaces and stop leaking host secrets
Keep the host sandbox fail-closed before adapter wiring: home/Users/Volumes
cannot be the workspace, login shells no longer re-import rc secrets, and
Service applies the env allowlist on every Run. Also close the Lstat TOCTOU
and post-wait Kill window.
* fix(localsandbox): check Close() in seatbelt spawn tests
* fix(localsandbox): deny host temp dirs and refuse leftover wide workspaces
file-read* still left /tmp and /var/folders readable, and Build/Relax
accepted /private/var, /System/Volumes/Data, and ~/Library/** as a
workspace, which re-opened the private deny. Also re-open only ~/.local/bin.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(agent): end the turn when the completion cap truncates an answer
A round that came back with finish_reason=length and no tool calls used to
fall through analyzeResponse as non-terminal. With no tool calls to run,
appendToolResults pushed the half-written answer back into the history as a
plain assistant message with nothing telling the model to continue, so the
next round rewrote the answer from the top, hit the same cap, and repeated
until the round budget ran out or the user cancelled.
Three guards, each independently load-bearing:
- analyzeResponse gains a truncation case: deliver the partial text and end
the turn. A cap hit that produced no text at all has nothing to hand over
and defers to the existing bounded empty-content retry. No continuation
nudge is sent — it only grows the prompt with the discarded fragment, and
assistant prefill, the reliable form, is unavailable on most
OpenAI-compatible endpoints.
- A consecutive-length breaker stops the loop when truncation keeps landing
inside tool-call arguments, the one shape that still legitimately loops.
- The stuck-loop detector now counts an identical lack of content. A model
burning its whole budget on reasoning returns empty every round, which used
to land in the else branch and reset the counter, so the guard could never
fire on the case that needed it most.
The answer event and its Done marker carry a truncated flag through the SSE
metadata, and the chat UI shows the same style of notice it already uses for
fallback answers.
Fixes#3446
* fix(agent): treat unterminated reasoning as no answer and keep the truncation marker
Review follow-up on three ways the truncation path could still mislead.
An inline <think> block that the cap cut off never gets its closing tag, so
the pair regex left it whole and the round looked like it had produced an
answer. The truncation case then skipped the empty-content retry it exists
for and emitted the chain of thought as the final answer. StripThinkBlocks
now drops a dangling <think> through end of input, which is what the
streaming splitter already does with the same text, so the two agree on what
counts as an answer.
The truncation marker only rode on the SSE metadata, and a finished
conversation is rebuilt from agent_steps rather than replayed from stream
events. A reloaded, shared or re-opened turn therefore lost the notice and
its half-written answer looked complete. AgentStep carries the flag now — it
is a jsonb column, so no migration — and the frontend marks the rebuilt
answer event and message from it.
The consecutive-length guard ran before the round was recorded, so a final
round of "let me look that up" plus a truncated tool call handed that
preamble over as the final answer, with its tool calls never refused and the
preamble never retracted from the answer area. The guard now records the
round and fails its tool calls the way every other truncated round does, and
a response carrying tool calls never donates its text as the answer.
* fix(security): close the low-severity shared-space findings
Verified and fixed from the shared-space permission review:
- Embed channels kept knowledge_ids, tag_ids, mentioned_items, skill_names
and summary_model_id from anonymous visitors. Mentions are only scoped
for shared agents, so a visitor could query any KB (and pick any model)
of the channel workspace by ID. The payload now drops them; the widget
never sends them.
- PUT /knowledgebase/:kb_id/wiki/issues/:issue_id/status updated any issue
by ID. The update is now scoped to the path's KB (404 otherwise).
- A direct org invite let the inviter attach any user of the target
workspace (or any user by ID) as its representative, whose username and
email the roster then showed to all members. Direct adds attach no
representative, and the roster returns emails only for the caller's own
workspace.
- COS and KS3 dropped the bucket (and COS the region) segment when turning
a path into an object key, while the tenant check scans every segment:
cos://42/<region>/prefix/7/... passed as tenant 42 and read tenant 7's
object from a shared bucket. Both must now match the service's config.
- Shared agent listings returned the full agent to receivers. Prompts and
the creator's user ID are withheld; capabilities and scope stay visible.
- Temporary document uploads let the caller pick the parser engine (and
its credentials) of a shared agent's workspace; only the agent's rules
choose one there now.
- Suggested questions used request / message-snapshot knowledge_ids
without a permission check. Each document is now resolved to its KB and
checked like KB IDs, including a shared agent's scope.
- IM channels accepted any knowledge_base_id (and any agent ID on create).
Both must belong to the channel's workspace, the KB also to a restricted
API key's allow-list.
- KB-restricted API keys could list org-shared KBs outside their allow-list
in the organization views; those listings are now filtered.
- With enable_rbac=false, RequireKBAccess let cross-workspace denials
through. That switch is for roles inside a workspace; the guard now
always denies another workspace's KB, since handlers behind it load by
ID without a tenant filter.
- The org member limit was counted before and enforced apart from the
insert; it is now checked in the insert's transaction with the
organization row locked (SQLite serializes writers instead).
- The UpdateOrganization route comment and API doc said owner-only; the
service and UI allow org admins. Comment and doc corrected.
Documented, not changed: issued /r/ and presigned links stay valid until
they expire after a share is revoked.
* fix(security): address review of the low-severity shared-space fixes
- Embed payloads also drop question_origin. It selects a KB like an
@mention for agents that retrieve only when mentioned, and the widget
never sends it; the selection stays inside the agent's scope, but an
anonymous visitor should not pick among its KBs. suggestion_attribution is
kept (the widget sends it and it is validated).
- The receiver view of a shared agent also withholds the follow-up
suggestion instruction, a prompt like the others. Starter questions are
shown to receivers anyway, and the sharer's name is displayed by design.
- Follow-up suggestion generation moves into the agent's workspace with
types.WithExecutionTenant instead of a bare tenant rewrite, so the
caller is pinned before the document permission check runs. The auth
middleware and the shared run already pinned it; this removes the
dependency on that.
- KS3 CopyFile copies only from the service's own bucket (as COS and S3
do); another bucket's path is a cross-backend copy.
Tests cover the dropped question_origin, the follow-up instruction and the
/shared-agents listing, a follow-up context with the tenant switched, a
repeated wiki issue status update, and the KS3 copy source.
* feat(agent): size agent history by the window and calibrate token estimates
Agent history was capped at history_turns (default 5) because compaction
could not survive a turn. With checkpoints persisted, load history by
tokens instead, and let compaction bound the session.
Budget. The loader drops the oldest turns that do not fit, and what it
drops is lost: neither replayed nor summarized. A budget equal to the
compaction threshold (the earlier attempt, since reverted) let the loader
trim first, so whenever one turn was larger than the system prompt plus
the new question the request never crossed the threshold; nothing was
summarized and the session became a sliding window with no checkpoint.
agent.HistoryTokenBudget is now the whole window: overflow reaches the
first round's compaction, which summarizes it into a new checkpoint.
A turn-by-turn regression test drives the real loader, budget and
compactor and fails as soon as a stored turn after the checkpoint is
missing from the loaded history; restoring the old budget fails it.
Summarizer input. History can now reach the window, so the part being
summarized can outgrow one request. An overflowing request degrades to a
raw archive that is never persisted, so every later turn would fail the
same way. Prepare keeps the newest messages that fit, the prompt states
how many were left out, and file paths still come from all of them.
Calibration. cl100k counts Chinese at about a token a character where
Qwen and DeepSeek spend about 0.6. The estimator now carries a scale
(clamped to 0.5-1.5, text only), learned from consecutive requests in a
turn: they share tools, system prompt and earlier messages, so the
provider's growth between them prices exactly the appended messages,
without mixing in how a provider renders tool schemas. The turn's scale
is stored on its usage (context_token_scale); the next turn's loader
prices history with it and hands it to the engine. Loading, the first
compaction check and compaction itself share one estimator, so they
agree; calibrating only the trigger would let the loader drop turns
again, which the regression test covers.
Also: history is priced as the engine sends it (redacted KB/Wiki results
cost one line); a checkpoint older than the rows read is matched by the
assistant rows' (created_at, id) order; a read ending on a turn's
question keeps that turn; a compaction discarded for freeing under 5%
still persists its checkpoint; a checkpoint write matching no row is an
error; a partial index backs the newest-checkpoint lookup. Agent mode no
longer reads history_turns: the field leaves AgentConfig, the editor
hides it in smart-reasoning mode, and quick-answer keeps its turn cap.
* fix(agent): address review of window-sized history loading
Summarization. The summarizer call had a fixed 60s total timeout. With
history loaded up to the window, a request could carry ~175k tokens and be
cut off while prefilling or writing normally; the degraded result left no
checkpoint, so the next turn reloaded the same history and failed the same
way. Summarization now streams and is cancelled only after no output for the
engine's stall timeout (120s by default, LLMCallTimeout overrides), with the
overall ceiling left to the transport, as for the engine's own rounds and as
in Codex and Pi. A stopped turn is not retried. When the summarizer still
fails, the raw archive is persisted as a checkpoint marked degraded so the
loop is broken, and the next compaction refines it as the previous summary.
The archive is held to the summary budget, newest messages first, so a
degraded compaction always frees room.
Loading. The loader priced turns as sent but returned and held them as
stored, so wiki pages stored in full and sent as one line could pile up in
memory. It now returns the as-sent history and releases each turn's stored
rows once replayed, keeping about a page of rows at a time.
Index. The partial checkpoint index is replaced by a composite
(session_id, created_at DESC, id DESC) index serving the backwards page,
the newest-checkpoint lookup and the recent-messages reads. It is built with
CREATE INDEX CONCURRENTLY so writes to messages are not blocked; the
migration holds that single statement, since a multi-statement query runs as
an implicit transaction. A PostgreSQL integration test, skipped without
WEKNORA_MIGRATION_TEST_POSTGRES_DSN, applies the chain, runs 000106 down and
up through golang-migrate, and checks both queries use the index without a
sort; the SQLite migration test checks the same plans.
Calibration. It no longer waits for a total token count, and a usage with
only a scale is persisted. A turn that measures nothing carries forward the
scale it started from, so the newest turn always has the latest one.
* feat(agent): persist compaction checkpoints across turns
Agent compaction only lived inside the turn that ran it: the summary was
discarded at turn end and LoadAgentHistory rebuilt raw history from the
messages table, so every later turn past the threshold paid for another
summarization of nearly the same history.
A compaction whose history summary ends exactly on a stored turn is now
written onto that turn's assistant message (messages.context_checkpoint).
LoadAgentHistory starts from the newest checkpoint: the summary replaces
that turn and everything before it, and later turns are replayed verbatim.
The compactor already treats a leading summary as the previous summary, so
checkpoints fold forward incrementally.
History messages carry their turn's assistant message ID (chat.Message
TurnID, engine-internal) so compaction can tell whether its cut lands on a
stored-turn boundary. Nothing is persisted when the cut ends inside a stored
turn, covers only the live turn, or the history summary degraded to a raw
archive; a split turn's prefix summary is never persisted because that turn
is replayed whole next time. Writes are column-scoped and best-effort, and a
failed lookup falls back to history without a checkpoint.
Storing the checkpoint on the covered turn keeps the boundary implicit, so
session forks copy it with the turn and deleting the turn drops it. The
HistoryTurns cap still applies, now to the turns after the checkpoint.
* test(skill): wait for the background install in the name-conflict test
InstallSkill queues the install on a goroutine, and the fixture row already
starts at installing, so reading the status right after the call raced the
mock install and failed whenever it landed on ready first. Wait for the run
to finish on the reused row instead, as the other install tests do.
* feat(agent): size agent history by tokens instead of turn count
Agent history was capped at history_turns (default 5) because every turn
used to rebuild raw history and compaction could not persist. With
compaction checkpoints persisted, the cap is no longer what keeps a session
inside the window, and it threw away context that would have fit.
LoadAgentHistory now loads the newest checkpoint summary plus the newest
turns after it that fit a token budget: the run's compaction threshold
(agent.HistoryTokenBudget, the window less the reply reserve). History
that large already triggers the first round's compaction, which persists a
new checkpoint, so compaction rather than a turn count bounds the session.
Loading more would only hand that compaction a summarizer input larger
than the model can read.
Rows are paged backwards by (created_at, id) and the read stops at the
checkpoint or once the budget is full, so a long session is never read
whole. Kept turns stay contiguous, the newest turn is kept even when it
alone exceeds the budget (compaction splits it), and a turn cut by the read
is left out rather than replayed without its original question.
Agent mode no longer reads history_turns: AgentConfig.HistoryTurns is
removed, the editor hides the input in smart-reasoning mode and explains
the automatic behavior, and the smart-reasoning builtins drop the setting.
Quick-answer (KnowledgeQA) keeps its turn cap since it has no compaction.
* Revert "feat(agent): size agent history by tokens instead of turn count"
This reverts commit d05fb6e5f1f161813a88e9127609f7a059f38f12.
Loading history up to the compaction threshold lets the loader drop the
oldest turns before compaction can trigger: whenever a single turn's
replay is larger than the system prompt plus the new question, the
trimmed history stays under the threshold, nothing is summarized, and the
session degrades into a large sliding window that never gets a
checkpoint. A turn-by-turn simulation of LoadAgentHistory with the real
compactor showed zero compactions over 60 turns in those cases.
Keep the persisted checkpoints with the HistoryTurns cap for now; token
budgeted history returns in a separate change with a budget that leaves
the overflow to compaction.
* fix(agent): search bound knowledge bases before answering, and pass suggested-question origins
A user picked the suggested question "为什么在比较图形时需要谨慎处理?"
and the agent answered in round 1 with no tool call, claiming it had
searched the knowledge base and found nothing, then explained misleading
statistical charts. The question had been generated from "Corners Don't
Look Like That: Regarding Screenspace Ambient Occlusion" in a bound base,
where "comparing graphs" means SSAO falloff curves.
Two things let that happen: the grounding prompt said "Stable general
explanations need no lookup", and nothing told the agent where the
question came from - the base in question had no profile, only two
recent titles in the directory, and the click sent the text alone.
Prompt
- With knowledge tools available, search bound bases before answering a
question about a topic, even one that reads like general knowledge;
directory profiles and recent titles cannot prove a base lacks it.
- The general-explanation exemption applies only without bound bases.
- Say you searched or found nothing only when a search tool ran.
Question origin
- Suggested questions from FAQ/document chunks also return knowledge_id.
- Picking one on the home or chat page sends question_origin
{knowledge_base_id, knowledge_id} with that question only.
- The service keeps the origin only when its base is inside the turn's
resolved scope and the document belongs to that base; it never
changes the scope (unlike knowledge_base_ids, which narrows it).
- The engine renders it in runtime_context as <question_origin> with the
base and document handles and a note to search it before answering.
* fix(agent): keep question origins across file/tag scopes and bind them to the send
Review follow-up for the suggested-question origin.
- Scope check: the origin was kept only when its base was in the resolved
KB list, which is empty for file-only and tag-only scopes even though
their bases are searched. It is now checked against SearchTargets after
they are built; the document is kept only when a target for that base
covers it (unfiltered base, or a document scope listing it).
resolveQuestionOriginInfo re-checks SearchTargets, looks up the name of
a base that is not in kbInfos, and the renderer omits an unknown name.
- Frontend: the origin travelled through module-level "pending" state, so
a send that returned early (agent not ready, upload in progress,
steering) left it behind and a second pick overwrote it. It now rides
with the one send: triggerSend(text, options) -> createSession ->
send-msg -> sendMsg / changeFirstQuery.
- RetrieveKBOnlyWhenMentioned: suggestions are still generated from the
agent's bases, but picking one retrieved nothing. The origin now
selects its base, as an @mention would, when the agent may read it.
- API sync: question_origin in docs/api/chat.md and swagger;
QuestionOrigin on the Go SDK's KnowledgeQARequest/AgentQARequest and
knowledge_id on SuggestedQuestion.
Repeating the same query in another mode is usually wasted, but keyword
mode can still recall identifiers that hybrid never surfaced. Stop telling
the model that switching mode cannot help.
The agent searched seven times for "Echo 开放权重模型 降低成本". Hybrid
retrieval ranked the right chunk first (RRF rank 0) every time, but the
rerank model saw only the chunk text, which never names Echo or cost;
it scored 0.0025 and all 18 candidates fell below the 0.15 fallback
floor. The tool reported "No matching chunks", and the model cycled
hybrid -> semantic -> keyword, which cannot help since every mode shares
the same rerank check. Scoring the same chunk with its document title
("Show HN: Echo - ... at 1/3 the cost using open-weight models") gives
0.45, and 0.55 for the query written as a question.
- Rerank passages are title + enriched chunk (rerankPassage); FAQ chunks
keep their own question, and the pattern filter still sees chunk text
only.
- When retrieval found candidates but the rerank model rejected all of
them, the empty statement says so and that switching mode will not
help, instead of "No matching chunks".
- The tool description, query schema and agent system prompt ask for a
complete natural-language question in hybrid/semantic mode rather than
a keyword list, and drop "switch mode" as the retry advice. Every
keyword engine matches any token (OR + BM25), so a question costs the
keyword leg no recall; on the same 20 bases the Echo chunks rank the
same for both query styles.
read_document's in-document query was matched as one literal string, so a
keyword query such as "sunflower f(n,k)" found nothing in a 708-chunk
document even when both words were there. The zero-match result gave the
model no next step, and the UI rendered it as "0 / 708 chunks loaded",
which reads like a failed load.
Paging mode also had no size bound: the registry truncates
ToolResult.Output, but the model sees the result rendered from Data, so a
limit=100 page reached ~68k characters in the model context.
- A plain query is split on whitespace; a chunk matches when it contains
every word, in any order. regex=true still matches one expression.
- A zero-match result carries a <hint> suggesting fewer/different words
or reading a chunk from search_knowledge results.
- A page stops adding chunks at 80% of the output budget (counting
per-chunk rendering overhead); next_offset resumes at the first chunk
not returned, and an oversized first chunk is still returned.
- The UI summary shows in-document search results ("no matches" /
"N matches") and, for offset paging, the chunk range instead of a page
number that no longer holds.
* refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents
The agent's retrieval surface had grown to five overlapping tools whose
behaviour was scattered across long descriptions, interleaved "tips" in
tool output (most of which modelcontext discarded before the model saw
them), and a grep_chunks implementation that ran an unindexed regex scan
over the chunks table ordered by created_at, so large knowledge bases lost
recall and paid a full-table scan on every call.
Tool surface
- search_knowledge replaces knowledge_search and grep_chunks: one query,
mode = hybrid | semantic | keyword, optional knowledge_base_ids, limit.
Keyword mode runs through the retrieval engine's keyword index instead of
the DB regex scan. Modes unavailable for the scope are refused with a
reason. Rerank, MMR and de-duplication are kept; already_seen markers,
retrieval_statistics and the zero-result "CRITICAL" block are gone.
- read_document replaces list_knowledge_chunks, get_document_info and
wiki_read_source_doc: id accepts a dN or cN handle, always returns the
document header, pages by exact offset (fixing the old floor-to-page
bug), finds passages with query (literal by default, regex opt-in) and
reads neighbours around a chunk with context.
- list_documents pages through one knowledge base, honouring @file/@tag
scope.
- wiki_search takes a single query plus regex, knowledge_base_ids, limit;
literal queries are escaped so the model never reasons about JSON
escaping. Legacy queries / knowledge_base_id are still accepted.
- query_knowledge_graph is registered only when a graph-enabled KB is in
scope and its description/tips were trimmed.
Compatibility
- Retired names in stored allowlists, presets and API calls are mapped to
their successors by NormalizeAllowedTools at registration time; no data
migration. Stored histories with old names still decode, render and
redact. Calls to retired names return a replacement hint.
- modelcontext gains an "id" key space that decodes against both the chunk
and document handle tables (decode-only), plus rendering for the
document header, chunk roles, in-document matches and next_offset.
Prompts and surfaces
- The RAG, wiki researcher and hybrid templates now name the tools and the
search -> read -> cite order; tool descriptions are short and contain no
workflow instructions.
- Engine display names, history redaction, grounding prompt, IM rendering,
the built-in MCP server (search_knowledge gains mode/limit, grep_chunks
becomes keyword search, read_document gains query, wiki_search gains
regex), frontend editor/renderers/i18n, presets, website docs and the
changelog are updated accordingly.
Tests cover name mapping, mode validation and refusal, output shape,
read_document paging/context/query/scope, list_documents, wiki_search
escaping and legacy arguments, modelcontext decoding and rendering, and
registration-time aliasing and graph gating.
* fix(agent): address review findings on read_document, list_documents and model rendering
- read_document: fetch the second page whenever the first page did not
reach the end of the document, so an unaligned window near the end no
longer drops the trailing chunks (also fixes context windows around a
chunk near the end); a query passed together with a cN handle now
searches the owning document instead of being ignored; query mode stops
collecting once the output budget is nearly spent and reports truncated;
the document row is authorized in place instead of being fetched twice;
the context read takes the total from the window fetch instead of a
separate count query.
- list_documents: resolve the page's document scope in one batched tag
lookup instead of one call per row.
- modelcontext: render read_document output once and read next_offset
through the existing intValue helper.
- search_knowledge: point title/file-name lookups at list_documents.
- Tests for the boundary window, chunk context near the end, cN + query,
and budget-bound query results.
* fix(agent): close compatibility gaps in the consolidated retrieval tools
- Wiki-only scopes: read_document and list_documents are registered
whenever a wiki KB is in scope, since every KB stores chunks. Their
capability requirement includes wiki but is marked auxiliary, so they do
not widen a RAG agent's derived "all knowledge bases" filter to wiki-only
bases (Go and frontend derivations both honour it).
- search_knowledge resolves the retrieval mode per knowledge base instead
of refusing the call: FAQ and vector-only bases answer keyword requests
semantically, keyword-only bases answer semantic requests by keyword.
Results carry requested_mode and mode_fallbacks, the empty-result text
names the fallback, and the call is refused only when no base in scope
has a chunk index.
- The model rendering uses the mode the search actually ran with (it was
always labelled semantic) and emits <mode_fallback> per degraded base.
- list_documents applies the @file/@tag scope before paging: tags are
pushed into the database filter, pinned documents are loaded by ID, and
total/next_page count only readable documents.
- read_document resolves chunk context by chunk_index through a new
ChunkRepository.ListChunkNeighbors, so parent, summary and image chunks
sharing the index sequence no longer shift the neighbourhood; offset is
documented as a reading-order position, not a chunk index.
- wiki_search (agent and MCP) keeps its regular-expression default and
only matches literally when the text is not a valid expression;
regex=false forces a literal match and regex=true rejects invalid input.
- MCP grep_chunks keeps grep semantics: literal terms from the pattern
fetch candidates from the keyword index (semantic for bases without
one), and every returned chunk is verified against the pattern.
* docs(changelog): align the consolidation entry with regex wiki_search and grep_chunks semantics
* docs(mcp): note that grep_chunks recall is bounded by index candidates
* feat(knowledge): derive knowledge base descriptions from document profiles
Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.
Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.
Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.
Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.
Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.
Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.
* fix(knowledge): keep generated descriptions out of duplicates and refresh clones
DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.
CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.
* fix(knowledge): refresh the loaded clone target and carry document profiles
ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.
cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
* refactor(wiki): consolidate source_refs parsing and resolution
source_refs entries ("uuid" or "uuid|title") were parsed by ten
independent implementations that disagreed on whitespace handling, on
whether a model-supplied title could be trusted, and on how a leading
pipe was treated. The scope-enforced and legacy agent resolvers also
duplicated the lookup and the routing step looked every document up a
second time.
- types.ParseWikiSourceRef / FormatWikiSourceRef / WikiSourceRefMatchesKnowledge
are now the only readers and writer of the format.
- resolveWikiSourceDocuments replaces resolveAuthorizedSourceRefs,
resolveSourceRefs and wikiKnowledgeBasesForSourceRefs: one lookup per
document, titles always rebuilt from server data, dedupe by canonical ID,
unknown documents fail closed in every mode.
- removeSourceRef now matches padded or titled refs of a deleted document.
* refactor(data_analysis): expose the selected document as a fixed table name
The tool required the document ID inside the SQL text and blindly
string-replaced it with the physical DuckDB table name, which forced the
model-context codec to rewrite handles inside quoted SQL and let the
UUID-derived k_<uuid> table name reach the model through stored schema
summaries.
The document is now selected only by knowledge_id. Every query runs on a
dedicated connection where "dataset" is a temporary view over the
physical table (DuckDB temporary objects are connection-local, so
concurrent sessions stay isolated). The physical name remains accepted
for internal callers; any other table fails validation with an error
naming the table the model should use. Schema summaries, the schema
tool, the analyst prompt and the tool schema all name the same table, so
data_analysis.sql no longer carries source handles and its text codec
policy is removed.
* feat(modelcontext): report raw identifiers that survive encoding
Text-level compaction exists because raw UUIDs reach model context through
producers the registry never sees: error strings, slugs, results of
unknown shape. Retiring it requires knowing where those producers are.
Registry.LeakedIdentifiers scans an encoded message list for UUID-shaped
values in every application-composed field (user messages are skipped),
classifies each as registered (codec missed a rewrite) or unregistered
(producer emitted a raw ID), and SummarizeLeaks renders one line per
field. The agent engine and the three RAG pipeline call sites log the
report as a warning after EncodeMessages.
* fix(chat_pipeline): align RAG data analysis with the fixed table contract
The RAG data-analysis plugin still asked the planning model to fill in
knowledge_id and to write SQL from a prompt that exposed the raw
knowledge UUID, then passed the model's JSON straight to Execute. Under
the new data_analysis contract a query that names anything but the
dataset table is rejected, and the plugin logged that rejection at error
level and moved on, so the analysis vanished without any signal. A
"no analysis needed" plan took the same error path.
The pipeline now owns document selection: the model returns only
needs_analysis and sql, the prompt names the dataset table and never
shows the knowledge ID or physical table, and the plugin builds the tool
input itself. A rejected query is fed back to the model once for repair;
a query that still fails is reported through pipelineWarn with the SQL
and the tool error, and "not needed" is logged as such instead of as a
failure.
Covered by unit tests with a scripted model and executor: not-needed,
pipeline-owned knowledge_id, one-shot repair, persistent failure
surfaced, unparseable plan rejected.
Serply returns Google results through a keyed JSON API, which fills the
gap between Google Custom Search (needs an engine ID and a restricted
scope) and the scrapers. The adapter follows brave.go: hardcoded
endpoint, redirects refused so the key never leaves the official host,
4MB response cap, and per-call country and freshness filters. country
maps to Google's gl parameter and pd/pw/pm/py map to tbs qdr:d/w/m/y.
Serply has no date-range filter, so that form returns an error instead
of being silently dropped. num is clamped to 10 because the API serves
one results page per call.
Wiring matches the Bocha provider: type constant and metadata, registry
factory, service validation, frontend union, swagger enum entries (edited
by hand; a full swag regen drags in unrelated drift), and the web search
docs table.
analyzeResponse emitted EventAgentFinalAnswer with Done=true before
returning emptyContent=true to the retry loop, so every empty natural
stop produced a premature terminal answer event while the agent was
still running. Consumers that finalize on Done=true (e.g. the IM
channel, which closes its answer-done channel and then only waits a
bounded time for EventAgentComplete) can lock in a fallback and cancel
the still-running agent when the nudged retry is slower than that
window (#2906).
An empty natural stop now emits nothing — the verdict alone tells the
caller the round is retryable. When retries are exhausted, the loop
emits the selected fallback as the turn's sole terminal answer, so a
turn always ends with exactly one terminal answer event.
Regression tests cover all three shapes: the empty verdict emits no
answer events; empty-then-answer ends with exactly one terminal event
carrying the real answer; all-empty runs end with exactly one terminal
event carrying the fallback (each red on the pre-fix code).