Taxonomy selection asked the embedding provider for every folder in one
request, and for every item in another, by calling BatchEmbed directly.
BatchEmbedWithPool is what reads BATCH_EMBED_SIZE and splits the inputs,
so these two calls sat outside the only control an operator has over
request width: a provider that caps inputs per request rejected them
however low that setting was, and the planner fell back to feeding all
folders (#3390).
Send both sets through the pool, the way every other embedding caller in
the service does. Nothing else changes: the pool reassembles the vectors
in input order, which is what the similarity ranking below depends on.
The regression test carries the real pooling implementation behind a
provider that records the width of every request it receives, so a call
that goes back to the provider directly shows up as one oversized
request rather than as a passing test.
* fix(security): close the low-severity shared-space findings
Verified and fixed from the shared-space permission review:
- Embed channels kept knowledge_ids, tag_ids, mentioned_items, skill_names
and summary_model_id from anonymous visitors. Mentions are only scoped
for shared agents, so a visitor could query any KB (and pick any model)
of the channel workspace by ID. The payload now drops them; the widget
never sends them.
- PUT /knowledgebase/:kb_id/wiki/issues/:issue_id/status updated any issue
by ID. The update is now scoped to the path's KB (404 otherwise).
- A direct org invite let the inviter attach any user of the target
workspace (or any user by ID) as its representative, whose username and
email the roster then showed to all members. Direct adds attach no
representative, and the roster returns emails only for the caller's own
workspace.
- COS and KS3 dropped the bucket (and COS the region) segment when turning
a path into an object key, while the tenant check scans every segment:
cos://42/<region>/prefix/7/... passed as tenant 42 and read tenant 7's
object from a shared bucket. Both must now match the service's config.
- Shared agent listings returned the full agent to receivers. Prompts and
the creator's user ID are withheld; capabilities and scope stay visible.
- Temporary document uploads let the caller pick the parser engine (and
its credentials) of a shared agent's workspace; only the agent's rules
choose one there now.
- Suggested questions used request / message-snapshot knowledge_ids
without a permission check. Each document is now resolved to its KB and
checked like KB IDs, including a shared agent's scope.
- IM channels accepted any knowledge_base_id (and any agent ID on create).
Both must belong to the channel's workspace, the KB also to a restricted
API key's allow-list.
- KB-restricted API keys could list org-shared KBs outside their allow-list
in the organization views; those listings are now filtered.
- With enable_rbac=false, RequireKBAccess let cross-workspace denials
through. That switch is for roles inside a workspace; the guard now
always denies another workspace's KB, since handlers behind it load by
ID without a tenant filter.
- The org member limit was counted before and enforced apart from the
insert; it is now checked in the insert's transaction with the
organization row locked (SQLite serializes writers instead).
- The UpdateOrganization route comment and API doc said owner-only; the
service and UI allow org admins. Comment and doc corrected.
Documented, not changed: issued /r/ and presigned links stay valid until
they expire after a share is revoked.
* fix(security): address review of the low-severity shared-space fixes
- Embed payloads also drop question_origin. It selects a KB like an
@mention for agents that retrieve only when mentioned, and the widget
never sends it; the selection stays inside the agent's scope, but an
anonymous visitor should not pick among its KBs. suggestion_attribution is
kept (the widget sends it and it is validated).
- The receiver view of a shared agent also withholds the follow-up
suggestion instruction, a prompt like the others. Starter questions are
shown to receivers anyway, and the sharer's name is displayed by design.
- Follow-up suggestion generation moves into the agent's workspace with
types.WithExecutionTenant instead of a bare tenant rewrite, so the
caller is pinned before the document permission check runs. The auth
middleware and the shared run already pinned it; this removes the
dependency on that.
- KS3 CopyFile copies only from the service's own bucket (as COS and S3
do); another bucket's path is a cross-backend copy.
Tests cover the dropped question_origin, the follow-up instruction and the
/shared-agents listing, a follow-up context with the tenant switched, a
repeated wiki issue status update, and the KS3 copy source.
* feat(agent): size agent history by the window and calibrate token estimates
Agent history was capped at history_turns (default 5) because compaction
could not survive a turn. With checkpoints persisted, load history by
tokens instead, and let compaction bound the session.
Budget. The loader drops the oldest turns that do not fit, and what it
drops is lost: neither replayed nor summarized. A budget equal to the
compaction threshold (the earlier attempt, since reverted) let the loader
trim first, so whenever one turn was larger than the system prompt plus
the new question the request never crossed the threshold; nothing was
summarized and the session became a sliding window with no checkpoint.
agent.HistoryTokenBudget is now the whole window: overflow reaches the
first round's compaction, which summarizes it into a new checkpoint.
A turn-by-turn regression test drives the real loader, budget and
compactor and fails as soon as a stored turn after the checkpoint is
missing from the loaded history; restoring the old budget fails it.
Summarizer input. History can now reach the window, so the part being
summarized can outgrow one request. An overflowing request degrades to a
raw archive that is never persisted, so every later turn would fail the
same way. Prepare keeps the newest messages that fit, the prompt states
how many were left out, and file paths still come from all of them.
Calibration. cl100k counts Chinese at about a token a character where
Qwen and DeepSeek spend about 0.6. The estimator now carries a scale
(clamped to 0.5-1.5, text only), learned from consecutive requests in a
turn: they share tools, system prompt and earlier messages, so the
provider's growth between them prices exactly the appended messages,
without mixing in how a provider renders tool schemas. The turn's scale
is stored on its usage (context_token_scale); the next turn's loader
prices history with it and hands it to the engine. Loading, the first
compaction check and compaction itself share one estimator, so they
agree; calibrating only the trigger would let the loader drop turns
again, which the regression test covers.
Also: history is priced as the engine sends it (redacted KB/Wiki results
cost one line); a checkpoint older than the rows read is matched by the
assistant rows' (created_at, id) order; a read ending on a turn's
question keeps that turn; a compaction discarded for freeing under 5%
still persists its checkpoint; a checkpoint write matching no row is an
error; a partial index backs the newest-checkpoint lookup. Agent mode no
longer reads history_turns: the field leaves AgentConfig, the editor
hides it in smart-reasoning mode, and quick-answer keeps its turn cap.
* fix(agent): address review of window-sized history loading
Summarization. The summarizer call had a fixed 60s total timeout. With
history loaded up to the window, a request could carry ~175k tokens and be
cut off while prefilling or writing normally; the degraded result left no
checkpoint, so the next turn reloaded the same history and failed the same
way. Summarization now streams and is cancelled only after no output for the
engine's stall timeout (120s by default, LLMCallTimeout overrides), with the
overall ceiling left to the transport, as for the engine's own rounds and as
in Codex and Pi. A stopped turn is not retried. When the summarizer still
fails, the raw archive is persisted as a checkpoint marked degraded so the
loop is broken, and the next compaction refines it as the previous summary.
The archive is held to the summary budget, newest messages first, so a
degraded compaction always frees room.
Loading. The loader priced turns as sent but returned and held them as
stored, so wiki pages stored in full and sent as one line could pile up in
memory. It now returns the as-sent history and releases each turn's stored
rows once replayed, keeping about a page of rows at a time.
Index. The partial checkpoint index is replaced by a composite
(session_id, created_at DESC, id DESC) index serving the backwards page,
the newest-checkpoint lookup and the recent-messages reads. It is built with
CREATE INDEX CONCURRENTLY so writes to messages are not blocked; the
migration holds that single statement, since a multi-statement query runs as
an implicit transaction. A PostgreSQL integration test, skipped without
WEKNORA_MIGRATION_TEST_POSTGRES_DSN, applies the chain, runs 000106 down and
up through golang-migrate, and checks both queries use the index without a
sort; the SQLite migration test checks the same plans.
Calibration. It no longer waits for a total token count, and a usage with
only a scale is persisted. A turn that measures nothing carries forward the
scale it started from, so the newest turn always has the latest one.
* fix(security): close cross-workspace access paths in shared spaces
A review of how shared-space (organization) members reach each other's
resources found six paths that bypass the org-share permission model:
- presigned-preview signed any resource handle or storage path, turning a
known resource:// handle into an anonymous /r/ URL that outlives share
revocation. It now requires the path to belong to the caller's tenant,
like /files.
- POST /initialization/initialize let a shared-KB editor rewrite the owner
workspace's models (base URL, API key) in place. Only the KB's own
workspace may call it, and rewriting a stored model needs the same
authority as PUT /models/:id.
- KB JSON carried the legacy inline credentials (storage_config
secret_id/secret_key, which initialization fills with the MinIO env
keys, and vlm_config api_key) to every share receiver. They are withheld
from JSON; their DB columns are unchanged.
- A chat without agent_source_tenant_id resolved shares before the
caller's own agent, so a member sharing a same-ID built-in agent took
over other workspaces' default agent. Own agents now win, built-in
agents can no longer be shared (existing such shares no longer resolve),
and embed channels drop a visitor-supplied source workspace.
- Any org-admin tenant (even a tenant Viewer inside it) could raise another
workspace's KB share to editor/admin, and the route guard was bypassable
via an unrelated :id. Share management is now: the sharer from the source
tenant (Contributor+), source-tenant Admin+, or an org-admin tenant's
Admin+ that may only remove or lower. share_id must belong to :id.
- Wiki write tools were registered for every wiki KB in the search targets,
which only need read access, so a viewer share (e.g. via the built-in
wiki fixer) could edit or delete another workspace's wiki pages. They now
act only on KBs the caller may edit.
* fix(security): close the KB settings path and owner details for share receivers
Follow-up to the shared-space review:
- PUT /initialization/config shares the route guard that admits share
editors, so an editor could still rebind the owner's KB models, chunking
and storage, and storage backends resolved against the caller's workspace
let it bind the owner's KB to its own backend. Another workspace now needs
an admin share (the only share level the frontend offers settings to) and
may not change the storage binding at all.
- GET /initialization/config showed model base URLs and bucket locations to
any admin of the caller's own workspace, including share receivers. They
now only see whether credentials are configured.
- Wiki write tools on the caller's own KBs had no role floor, so a tenant
Viewer (and the IM, embed and MCP endpoint principals that run as Viewer)
could write through an agent what the HTTP routes deny. Writes now need
Contributor+, or the ingest capability for a scoped API key, following
the RBAC rollout switch.
- resolveAgent logs the share lookup error instead of a nil own-agent error
when a shared selector fails.
* fix(security): close the medium shared-space access findings
Follow-up to the high-risk fixes, from the same review:
- Wiki fixer moved into a shared KB's workspace ran with the caller's own
config there: an unset MCP mode registered all of the owner's MCP
services, and an "all" KB scope reached the owner's other KBs. The scoped
run now uses the built-in config pinned to that one KB, without MCP,
skills, sandbox or web search, and falls back to the KB's own model.
- An unset MCP mode means "all" at runtime but is shown as none in the share
scope; shared runs now treat it as none.
- Quick-answer mode used the request's web_search_enabled alone, so a
receiver could turn on search the agent disables and spend the owner's
provider quota. It now requires the agent setting too, as agent mode does.
- summary_model_id picked any model of the workspace a run executes in. It
is ignored for shared agents and the relocated wiki fixer.
- A workspace that left or was removed from an organization kept sharing its
KBs and agents. Its shares are revoked on departure, and share queries and
permission checks require the source workspace's membership and a live
organization, which also covers rows left behind earlier.
- Sharing an agent exposes its KB scope to all members, so a Contributor
could share colleagues' KBs (or "all") through an agent. The sharer must
now be able to share each KB directly (creator or Admin+; "all" needs
Admin+), also when a shared agent's scope is widened later.
- A scoped API key with manage_channels could mint an MCP endpoint token
wider than itself. Endpoints it creates, updates or rotates must stay
within its KB allow-list and capabilities.
- A shared agent's run indexed the receiver's Q&A into the owner's chat
history KB; indexing now uses the session owner's configuration.
- Sharing an agent with skills keeps injecting the owner's skill
environment values (product decision); the share panel now warns that
members can read them. Preview of originals through read-only shares is
kept as designed and documented.
#3406 (000104_skill_served_version) and #3407
(000104_message_context_checkpoint) each took version 104 and were merged
eleven minutes apart. golang-migrate refuses to load a directory with a
duplicate version ("duplicate migration file"), so every PostgreSQL
deployment of main fails to migrate.
Renumber the later of the two. It is safe for every deployment that could
exist: one that migrated between the two merges applied the skill
migration as 104 and now applies this one as 105, and after the second
merge main could not load its migrations at all, so no deployment applied
this file as 104. The migration is ADD COLUMN IF NOT EXISTS either way.
Add a test that loads both migration directories. Pull request CI runs on
the merge with main, so a second PR taking an already merged number now
fails before it lands.
* fix(ocr): normalize inline HTML tables in VLM/OCR sanitizer
The VLM/OCR path left raw HTML <table> blocks in the body: looksLikeHTML
does not match markdown text with an embedded table, so ocrHTMLToMarkdown
(which also lacked the table plugin) never ran. The chunker then split the
raw markup mid-row, tripping the 7500-char hard limit and invalidating the
heading layer for the whole document.
Reuse docparser.NormalizeHTMLTables (already used by PaddleOCR-VL) instead:
export it and call it unconditionally from sanitizeOCRText, since it is a
safe no-op when no <table> is present.
* fix(ocr): keep merged-cell HTML tables chunkable
- Narrow span detection to values >1 so rowspan=1/colspan=1 tables
still convert to GFM instead of falling back to raw HTML.
- On the merged-cell fallback, split each <tr> onto its own line and
pad the block with blank lines so the chunker can split by "\n"
instead of force-cutting at the absolute max size.
* fix(knowledge): normalize inline HTML tables for all parser engines before chunking
Previously NormalizeHTMLTables was only wired into the paddleocr-vl
converters, so tables emitted by other engines (notably MinerU) reached
the chunker as raw single-line <table> blocks and were force-cut at the
7500 char hard limit.
Add the normalization at the two central pre-chunk boundaries instead:
- knowledge_process.go: after all converter/ASR/image markdown rewrites,
before the chunker step (covers every parser engine)
- knowledge_create.go: before building the splitter config for manual
markdown
Converters keep their existing calls (idempotent no-op).
* fix(chunker): bound Markdown link/image protected patterns to one line
The unbounded [^\]]* / [^)]+ character classes in the Markdown link and
image protectedPatterns also matched newlines, so a stray '[' left behind
by OCR (e.g. a full-width 【公章】 mis-read as half-width) could pair with
a distant '](' and swallow whole paragraphs as one protected atomic span.
buildUnitsWithProtection then treated the region as a single unit and
mergeUnits could not split it, defeating chunking (observed 1 chunk of
1753 runes where 16 chunks were expected).
Exclude newlines and bound the link text (<=200) / destination (<=500),
matching CommonMark's rule that neither may span a blank line.
Regression: protected_spans_test.go — same input now yields multiple
chunks with max 125 runes (chunkSize=128) instead of a single 1753-rune
chunk; well-formed single-line links/images stay protected.
* fix(ocr): keep non-GFM HTML tables chunkable across all ingest paths
Conversion fallbacks for <br>, nested lists, and one-column tables still
left single-line HTML, so MinerU output could hit the same oversized-chunk
bug. Split those rows, accept one-column GFM, skip fenced examples, and
normalize before image resolution so cell images are not escaped.
* style: wrap long lines and avoid shadowing max in table-chunk tests
Pre-push lll/revive failed on the PR diff versus origin/main.
---------
Co-authored-by: neoczr <neoczr@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* feat(agent): persist compaction checkpoints across turns
Agent compaction only lived inside the turn that ran it: the summary was
discarded at turn end and LoadAgentHistory rebuilt raw history from the
messages table, so every later turn past the threshold paid for another
summarization of nearly the same history.
A compaction whose history summary ends exactly on a stored turn is now
written onto that turn's assistant message (messages.context_checkpoint).
LoadAgentHistory starts from the newest checkpoint: the summary replaces
that turn and everything before it, and later turns are replayed verbatim.
The compactor already treats a leading summary as the previous summary, so
checkpoints fold forward incrementally.
History messages carry their turn's assistant message ID (chat.Message
TurnID, engine-internal) so compaction can tell whether its cut lands on a
stored-turn boundary. Nothing is persisted when the cut ends inside a stored
turn, covers only the live turn, or the history summary degraded to a raw
archive; a split turn's prefix summary is never persisted because that turn
is replayed whole next time. Writes are column-scoped and best-effort, and a
failed lookup falls back to history without a checkpoint.
Storing the checkpoint on the covered turn keeps the boundary implicit, so
session forks copy it with the turn and deleting the turn drops it. The
HistoryTurns cap still applies, now to the turns after the checkpoint.
* test(skill): wait for the background install in the name-conflict test
InstallSkill queues the install on a goroutine, and the fixture row already
starts at installing, so reading the status right after the call raced the
mock install and failed whenever it landed on ready first. Wait for the run
to finish on the reused row instead, as the other install tests do.
* feat(agent): size agent history by tokens instead of turn count
Agent history was capped at history_turns (default 5) because every turn
used to rebuild raw history and compaction could not persist. With
compaction checkpoints persisted, the cap is no longer what keeps a session
inside the window, and it threw away context that would have fit.
LoadAgentHistory now loads the newest checkpoint summary plus the newest
turns after it that fit a token budget: the run's compaction threshold
(agent.HistoryTokenBudget, the window less the reply reserve). History
that large already triggers the first round's compaction, which persists a
new checkpoint, so compaction rather than a turn count bounds the session.
Loading more would only hand that compaction a summarizer input larger
than the model can read.
Rows are paged backwards by (created_at, id) and the read stops at the
checkpoint or once the budget is full, so a long session is never read
whole. Kept turns stay contiguous, the newest turn is kept even when it
alone exceeds the budget (compaction splits it), and a turn cut by the read
is left out rather than replayed without its original question.
Agent mode no longer reads history_turns: AgentConfig.HistoryTurns is
removed, the editor hides the input in smart-reasoning mode and explains
the automatic behavior, and the smart-reasoning builtins drop the setting.
Quick-answer (KnowledgeQA) keeps its turn cap since it has no compaction.
* Revert "feat(agent): size agent history by tokens instead of turn count"
This reverts commit d05fb6e5f1f161813a88e9127609f7a059f38f12.
Loading history up to the compaction threshold lets the loader drop the
oldest turns before compaction can trigger: whenever a single turn's
replay is larger than the system prompt plus the new question, the
trimmed history stays under the threshold, nothing is summarized, and the
session degrades into a large sliding window that never gets a
checkpoint. A turn-by-turn simulation of LoadAgentHistory with the real
compactor showed zero compactions over 60 turns in those cases.
Keep the persisted checkpoints with the HistoryTurns cap for now; token
budgeted history returns in a separate change with a budget that leaves
the overflow to compaction.
* fix(skills): keep retries and catalog installs from rewriting the skill definition
Every install path wrote its archive back to the workspace catalog. After a
skill was re-registered as v2, sandboxes still on v1 are pinned to v1's
archive, so retrying one of them (a failed retry, or a ready skill retried
with guidance) reinstalled v1 and also moved the catalog definition back to
v1 and deleted v2's archive.
Installs now carry where their archive came from. Only an upload or a source
pull is a new definition; a retry or a catalog install uses bytes the
workspace already stores and never rewrites the definition. A retry of a
pinned install keeps its pin and replays its own version, and bytes that
neither the definition nor the row holds are refused with a conflict.
The catalog listing also reports each installation's version so the UI can
show what an upgrade moves from and to.
* feat(frontend): upgrade outdated skill installs from the catalog, sandbox drawer and agent editor
An install whose archive differs from the catalog was flagged as outdated,
but nothing could move it to the catalog version: the install drawer hid
ready installs, and the manage drawer only offered a retry, which replays
the sandbox's own archive.
- Skill settings: cards with outdated installs show an upgrade button that
opens the install drawer with every outdated sandbox preselected. The
add wizard and install drawer also let outdated ready installs be picked,
labelled with the version they move from and to.
- Sandbox skill drawer: an outdated skill shows a "new version" row whose
upgrade button installs the catalog onto that one sandbox.
- Agent editor: skills on the chosen sandbox that are outdated show the
version change and an upgrade button, both for admins only, matching the
admin-only catalog install. The install progress entry, which opens a
drawer backed by admin-only endpoints, is now admin-only too.
- A skill that is being upgraded is no longer pruned from the agent's
selected skills while it is briefly not ready.
The outdated check is shared in utils/skillUpgrade.ts.
* feat(skills): keep the previous version serving while an upgrade runs or after it fails
An install rewrote the row to the new version and marked it installing as
soon as it started, and the agent is only offered ready rows. So an upgrade
took the skill away from every sandbox for its whole run, and a failed
upgrade kept it away, even though the image pointer only moves on success
and the previous version was in the image the whole time.
The row now records what it described when it was last ready (served:
version, description, instructions, digest, the archive its files come from,
and the snapshot that installed it). The agent's usable set, member env
acceptance and the @-mention list read TenantSkillEntity.ServedView, which
is that version marked ready while the row is installing or failed. A
successful install clears it and releases its archive; a failure or a stop
keeps it.
Keeping that consistent:
- The served archive counts as a reader when a bundle is released or
replaced by a re-registered catalog, so v1's files stay readable while v2
installs.
- The reaper and a failed removal put the served version back when the image
never moved past it, instead of labelling v1's files with the upgrade's
metadata. A stuck upgrade whose chain cannot be read is left alone, since
v1 keeps serving meanwhile.
- The env declaration is stored after the pointer moves rather than before
the snapshot, so an upgrade that fails late cannot drop the admin value of
a variable the previous version still reads.
Migration 000104 adds tenant_skills.served. The console says "upgrading,
still running 1.1.0" or "upgrade failed, still running 1.1.0", the agent
editor keeps such skills selectable, and the upgrade texts no longer warn
that the skill goes offline.
* fix(skills): offer only the upgrade to installs the catalog has moved past
Review follow-up on the upgrade flow.
- An install that differs from the catalog, ready or failed, is now offered
the upgrade and not the retries. The retry icon and the retry-with-guidance
box both replay the archive the install is pinned to, so next to an
Upgrade button they reinstalled the version the operator came to leave
behind. A failed first install of v1 whose catalog has since moved to v2
used to retry v1 from the drawer while the agent editor's Install went to
v2; every entry now goes to v2, and the agent editor labels it "Upgrade".
The drawer explains that the failed install is replaced by the catalog
version.
- A catalog install or a retry reuses the definition that already holds its
bytes instead of re-running the catalog upsert, which rewrote the row and
bumped its updated_at every time the skill was installed somewhere. The
upsert still runs for a definition that does not exist yet or whose object
was lost.
* feat(audit): show API key name on knowledge-base activity
API-key operations currently look like the first tenant user or
"system". Snapshot the key id/name into activity details so the
feed can tell which key initiated the change, without restoring
key authorization onto async workers.
Fixes#3402
* fix(frontend): show activity API key on a second line
The 132px actor column truncates "user · API Key · name" into one
ellipsis. Keep the person on the first line and the key name below.
---------
Co-authored-by: MercuialC <MercuialC@users.noreply.github.com>
Co-authored-by: root <root@localhost.localdomain>
* fix(agent): search bound knowledge bases before answering, and pass suggested-question origins
A user picked the suggested question "为什么在比较图形时需要谨慎处理?"
and the agent answered in round 1 with no tool call, claiming it had
searched the knowledge base and found nothing, then explained misleading
statistical charts. The question had been generated from "Corners Don't
Look Like That: Regarding Screenspace Ambient Occlusion" in a bound base,
where "comparing graphs" means SSAO falloff curves.
Two things let that happen: the grounding prompt said "Stable general
explanations need no lookup", and nothing told the agent where the
question came from - the base in question had no profile, only two
recent titles in the directory, and the click sent the text alone.
Prompt
- With knowledge tools available, search bound bases before answering a
question about a topic, even one that reads like general knowledge;
directory profiles and recent titles cannot prove a base lacks it.
- The general-explanation exemption applies only without bound bases.
- Say you searched or found nothing only when a search tool ran.
Question origin
- Suggested questions from FAQ/document chunks also return knowledge_id.
- Picking one on the home or chat page sends question_origin
{knowledge_base_id, knowledge_id} with that question only.
- The service keeps the origin only when its base is inside the turn's
resolved scope and the document belongs to that base; it never
changes the scope (unlike knowledge_base_ids, which narrows it).
- The engine renders it in runtime_context as <question_origin> with the
base and document handles and a note to search it before answering.
* fix(agent): keep question origins across file/tag scopes and bind them to the send
Review follow-up for the suggested-question origin.
- Scope check: the origin was kept only when its base was in the resolved
KB list, which is empty for file-only and tag-only scopes even though
their bases are searched. It is now checked against SearchTargets after
they are built; the document is kept only when a target for that base
covers it (unfiltered base, or a document scope listing it).
resolveQuestionOriginInfo re-checks SearchTargets, looks up the name of
a base that is not in kbInfos, and the renderer omits an unknown name.
- Frontend: the origin travelled through module-level "pending" state, so
a send that returned early (agent not ready, upload in progress,
steering) left it behind and a second pick overwrote it. It now rides
with the one send: triggerSend(text, options) -> createSession ->
send-msg -> sendMsg / changeFirstQuery.
- RetrieveKBOnlyWhenMentioned: suggestions are still generated from the
agent's bases, but picking one retrieved nothing. The origin now
selects its base, as an @mention would, when the agent may read it.
- API sync: question_origin in docs/api/chat.md and swagger;
QuestionOrigin on the Go SDK's KnowledgeQARequest/AgentQARequest and
knowledge_id on SuggestedQuestion.
* feat(frontend): show overall progress for knowledge file uploads
Batch uploads ran one file at a time with no feedback until the whole
batch finished, and nothing followed the asynchronous parse afterwards.
The knowledge base list still carried a progress panel whose event
emitter was removed with the old menu upload flow, so it never showed.
- Run uploads through a global queue (stores/uploadQueue.ts, wired to
the API in stores/uploadTasks.ts): three concurrent transfers,
per-file cancel that aborts the request, retry for failed or
cancelled files, and polling of the created knowledge rows until
parsing settles. finalizing counts as searchable.
- Add a floating panel (components/upload-tasks/) mounted in the
platform shell: stage headline with a progress ring, bytes and ETA
while uploading, a stacked bar for searchable / in progress / failed /
duplicate, per-file status with hover actions (cancel, retry, open the
document), an issues filter, grouping by batch, collapse, a confirm
before closing mid-upload, and auto-dismiss after a clean finish.
- Warn before unloading the page while transfers are still running.
- Refresh an open knowledge base list as files land (throttled), but
skip mid-batch reloads while the user has scrolled past the first
page; the batch's final refresh always runs.
- Remove the orphaned list-page panel, the unused useKnowledgeBase
requestMethod, the per-batch toasts and the i18n keys they leave
unused.
* fix(frontend): harden the upload queue's wiring into the app
Review of the upload progress panel found the store misbehaving once
connected to real auth and page events:
- The owner watch used one getter returning a fresh array, so every
setUser() fired it, even for the same person (opening the user menu,
hovering the workspace submenu), and dismissed running uploads. Watch
the user id and tenant id as separate sources.
- Starting a new batch after the previous run finished cleared it
whole, including failed files still waiting for Retry. Only batches
whose every file is searchable are dropped now.
- The knowledge base list page reloaded everything on each mid-batch
refresh (about every 2s), closing open card menus. It now waits for
the batch's final event; the detail page's mid-batch refreshes only
reload files and folders, and tags and wiki probes wait for the end.
- A row missing from one successful status poll was marked deleted for
good. It now takes three consecutive misses, reset when it reappears.
- Closing the panel aborted requests whose body was already sent, which
the server still stores. Those are left to finish and trigger a list
refresh when they land.
- Two identical files in one batch could both pass the server's hash
dedup when uploaded in parallel. Same-size files for one knowledge
base now never run at once, so the second sees the first's row.
Also: detect cancellation from the abort signal instead of the item
state, ignore a cancelled attempt that settles after its retry started,
bind the beforeunload guard to the panel's lifetime, and show the
subfolder next to file names from folder uploads.
The store body moves into uploadTasksCore.ts with the API, events and
identity injected, so the owner watch and enqueue policy are tested
alongside the queue.
Repeating the same query in another mode is usually wasted, but keyword
mode can still recall identifiers that hybrid never surfaced. Stop telling
the model that switching mode cannot help.
The agent searched seven times for "Echo 开放权重模型 降低成本". Hybrid
retrieval ranked the right chunk first (RRF rank 0) every time, but the
rerank model saw only the chunk text, which never names Echo or cost;
it scored 0.0025 and all 18 candidates fell below the 0.15 fallback
floor. The tool reported "No matching chunks", and the model cycled
hybrid -> semantic -> keyword, which cannot help since every mode shares
the same rerank check. Scoring the same chunk with its document title
("Show HN: Echo - ... at 1/3 the cost using open-weight models") gives
0.45, and 0.55 for the query written as a question.
- Rerank passages are title + enriched chunk (rerankPassage); FAQ chunks
keep their own question, and the pattern filter still sees chunk text
only.
- When retrieval found candidates but the rerank model rejected all of
them, the empty statement says so and that switching mode will not
help, instead of "No matching chunks".
- The tool description, query schema and agent system prompt ask for a
complete natural-language question in hybrid/semantic mode rather than
a keyword list, and drop "switch mode" as the retry advice. Every
keyword engine matches any token (OR + BM25), so a question costs the
keyword leg no recall; on the same 20 bases the Echo chunks rank the
same for both query styles.
read_document's in-document query was matched as one literal string, so a
keyword query such as "sunflower f(n,k)" found nothing in a 708-chunk
document even when both words were there. The zero-match result gave the
model no next step, and the UI rendered it as "0 / 708 chunks loaded",
which reads like a failed load.
Paging mode also had no size bound: the registry truncates
ToolResult.Output, but the model sees the result rendered from Data, so a
limit=100 page reached ~68k characters in the model context.
- A plain query is split on whitespace; a chunk matches when it contains
every word, in any order. regex=true still matches one expression.
- A zero-match result carries a <hint> suggesting fewer/different words
or reading a chunk from search_knowledge results.
- A page stops adding chunks at 80% of the output budget (counting
per-chunk rendering overhead); next_offset resumes at the first chunk
not returned, and an oversized first chunk is still returned.
- The UI summary shows in-document search results ("no matches" /
"N matches") and, for offset paging, the chunk range instead of a page
number that no longer holds.
postUpload inherited the axios instance's 30s default, so any upload taking
longer than 30s was aborted client-side no matter what MAX_FILE_SIZE_MB was
set to. On a ~30 Mbps uplink that caps uploads at roughly 100MB even when the
deployment is configured for 200MB — and every layer underneath (reverse proxy,
frontend nginx, Go handler, docreader gRPC) had already accepted the configured
limit. The server only saw the request after the browser had given up, and
logged "failed to upload file to MinIO: context canceled" with a sub-second
latency.
Four of the five postUpload call sites worked around this by passing an explicit
timeout; knowledge-base file upload and chat temporary attachments did not.
Derive the default from the payload so correct behaviour no longer depends on
each caller remembering to opt in — an explicit config.timeout still wins.
The aborted request also surfaced as "network error, please check your
connection": axios timeouts arrive without error.response, so they landed in
the same branch as an unreachable server and sent people chasing a network
problem that did not exist. Report the two separately.
* refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents
The agent's retrieval surface had grown to five overlapping tools whose
behaviour was scattered across long descriptions, interleaved "tips" in
tool output (most of which modelcontext discarded before the model saw
them), and a grep_chunks implementation that ran an unindexed regex scan
over the chunks table ordered by created_at, so large knowledge bases lost
recall and paid a full-table scan on every call.
Tool surface
- search_knowledge replaces knowledge_search and grep_chunks: one query,
mode = hybrid | semantic | keyword, optional knowledge_base_ids, limit.
Keyword mode runs through the retrieval engine's keyword index instead of
the DB regex scan. Modes unavailable for the scope are refused with a
reason. Rerank, MMR and de-duplication are kept; already_seen markers,
retrieval_statistics and the zero-result "CRITICAL" block are gone.
- read_document replaces list_knowledge_chunks, get_document_info and
wiki_read_source_doc: id accepts a dN or cN handle, always returns the
document header, pages by exact offset (fixing the old floor-to-page
bug), finds passages with query (literal by default, regex opt-in) and
reads neighbours around a chunk with context.
- list_documents pages through one knowledge base, honouring @file/@tag
scope.
- wiki_search takes a single query plus regex, knowledge_base_ids, limit;
literal queries are escaped so the model never reasons about JSON
escaping. Legacy queries / knowledge_base_id are still accepted.
- query_knowledge_graph is registered only when a graph-enabled KB is in
scope and its description/tips were trimmed.
Compatibility
- Retired names in stored allowlists, presets and API calls are mapped to
their successors by NormalizeAllowedTools at registration time; no data
migration. Stored histories with old names still decode, render and
redact. Calls to retired names return a replacement hint.
- modelcontext gains an "id" key space that decodes against both the chunk
and document handle tables (decode-only), plus rendering for the
document header, chunk roles, in-document matches and next_offset.
Prompts and surfaces
- The RAG, wiki researcher and hybrid templates now name the tools and the
search -> read -> cite order; tool descriptions are short and contain no
workflow instructions.
- Engine display names, history redaction, grounding prompt, IM rendering,
the built-in MCP server (search_knowledge gains mode/limit, grep_chunks
becomes keyword search, read_document gains query, wiki_search gains
regex), frontend editor/renderers/i18n, presets, website docs and the
changelog are updated accordingly.
Tests cover name mapping, mode validation and refusal, output shape,
read_document paging/context/query/scope, list_documents, wiki_search
escaping and legacy arguments, modelcontext decoding and rendering, and
registration-time aliasing and graph gating.
* fix(agent): address review findings on read_document, list_documents and model rendering
- read_document: fetch the second page whenever the first page did not
reach the end of the document, so an unaligned window near the end no
longer drops the trailing chunks (also fixes context windows around a
chunk near the end); a query passed together with a cN handle now
searches the owning document instead of being ignored; query mode stops
collecting once the output budget is nearly spent and reports truncated;
the document row is authorized in place instead of being fetched twice;
the context read takes the total from the window fetch instead of a
separate count query.
- list_documents: resolve the page's document scope in one batched tag
lookup instead of one call per row.
- modelcontext: render read_document output once and read next_offset
through the existing intValue helper.
- search_knowledge: point title/file-name lookups at list_documents.
- Tests for the boundary window, chunk context near the end, cN + query,
and budget-bound query results.
* fix(agent): close compatibility gaps in the consolidated retrieval tools
- Wiki-only scopes: read_document and list_documents are registered
whenever a wiki KB is in scope, since every KB stores chunks. Their
capability requirement includes wiki but is marked auxiliary, so they do
not widen a RAG agent's derived "all knowledge bases" filter to wiki-only
bases (Go and frontend derivations both honour it).
- search_knowledge resolves the retrieval mode per knowledge base instead
of refusing the call: FAQ and vector-only bases answer keyword requests
semantically, keyword-only bases answer semantic requests by keyword.
Results carry requested_mode and mode_fallbacks, the empty-result text
names the fallback, and the call is refused only when no base in scope
has a chunk index.
- The model rendering uses the mode the search actually ran with (it was
always labelled semantic) and emits <mode_fallback> per degraded base.
- list_documents applies the @file/@tag scope before paging: tags are
pushed into the database filter, pinned documents are loaded by ID, and
total/next_page count only readable documents.
- read_document resolves chunk context by chunk_index through a new
ChunkRepository.ListChunkNeighbors, so parent, summary and image chunks
sharing the index sequence no longer shift the neighbourhood; offset is
documented as a reading-order position, not a chunk index.
- wiki_search (agent and MCP) keeps its regular-expression default and
only matches literally when the text is not a valid expression;
regex=false forces a literal match and regex=true rejects invalid input.
- MCP grep_chunks keeps grep semantics: literal terms from the pattern
fetch candidates from the keyword index (semantic for bases without
one), and every returned chunk is verified against the pattern.
* docs(changelog): align the consolidation entry with regex wiki_search and grep_chunks semantics
* docs(mcp): note that grep_chunks recall is bounded by index candidates
* feat(artifacts): add an artifacts library with its own message_artifacts table
Skill-generated files were only reachable from inside the conversation that
produced them. This adds a sidebar entry and a page that lists them across
every conversation the user can see.
Storage: artifacts move out of the messages.artifacts JSONB column into a new
message_artifacts table (PostgreSQL 000103, SQLite 000023), backfilled on
upgrade. The table is now the only source of truth; the old column is kept and
the down migration writes the table back into it. The message repository
loads artifacts for every message it returns and writes them on create,
update (when non-nil) and fork, so callers still use Message.Artifacts.
mod_time is stored as RFC 3339 text because the collector matches files on
nanosecond mtimes, which PostgreSQL timestamps would truncate.
API: GET /api/v1/artifacts lists the latest version of each file (grouped by
session + source path, with version_count), scoped like the session list,
with keyword / file type filters and paging. Downloads keep using the
per-session endpoint.
Frontend: "产物/Artifacts" sidebar entry (hidden without sandbox support), a
library page with type tabs, search, date groups, infinite loading, preview
drawer, download and jump-to-conversation; strings in all five locales. The
sidebar's top-menu paths are now one shared list.
* fix(artifacts): keep the restore hash guard and count versions by URL
RecordRestoredArtifactMtime again only stamps rows whose content hash matches
the restored content (and ignores empty hashes), and never rewrites the hash,
matching MessageArtifacts.WithRestoredMtime. Without the guard a restore of
v1's content also moved v2's mtime and relabelled v2 with v1's hash.
The library's version_count now counts distinct URLs per (session, source
path). A later answer that references an earlier file stores another row with
the same URL, which COUNT(*) reported as an extra version.
* fix(artifacts): scope the library to web chats and clear the legacy column
The library used the session list's default scope, which counts sessions
without an owner as legacy tenant-level rows. IM sessions are created without
an owner, so every member saw the file names and titles of every IM chat
(downloads then 404 for non-admins). The library now applies the same "web"
bucket predicate as the home sidebar, extracted from QueryPaged so the two
cannot drift: no IM mapping (including cleared ones), no embed marker, no
API-key owner.
Migrations 000103 / SQLite 000023 now clear messages.artifacts after the
backfill. Message loads still SELECT *, so the stale copy was read on every
history load. Only rows that carried artifacts are rewritten; the column stays
for instances on the old binary during a rolling upgrade, and the down
migration still rebuilds it from the table.
* fix(migrations): keep the 000103 backfill alive on malformed artifact JSON
A single file_size that is not an integer ("12.5", "") or a created_at that
does not parse ("garbage", month 13) raised a cast error and failed the whole
migration, blocking startup. Casts now go through pg_temp helpers that return
NULL on bad input, falling back to 0 and the message's created_at; valid rows
are unchanged. SQLite casts never raise; its migration test now covers the
same malformed values.
* docs(artifacts): describe the web-only library scope and the cleared legacy column
response_type=error was being emitted when a tool execution failed,
but that type is reserved for internal agent failures (LLM/pipeline
errors), which clients treat as a crashed run. A failed tool call is
still a tool result: both the chat SSE handler and the skill install
transcript now emit response_type=tool_result with success=false and
the error string in metadata, so clients render it as a failed tool
card and let the agent recover instead of treating the run as dead.
Frontend needs no change: useChatStreamHandler already resolves tool
outcome from the success metadata field, history replay rebuilds from
agent_steps, and the API playground stops failing a whole request on
a recoverable tool error.
Second slice of #3338 (tracked as #3345).
Count alignment: CountKnowledgeByKnowledgeBaseID now excludes rows
mid-deletion, mirroring the document list's existing filter — a stranded
delete no longer shows as '4 documents, 3 listed' on the KB card. Failed
rows stay counted: they are visible, actionable cards.
Sweep C: housekeeping now recovers rows stuck at 'deleting' whose delete
task no longer exists. The delete dead-letter callback only flips rows
that are still 'deleting' when retries exhaust, so a worker death
mid-delete or a lost queue left the row hidden from the list forever.
Same recovery contract as the parse sweep: flip to failed with a clear
error so the row becomes visible and actionable; the delete executor is
idempotent and plans by row ID, so a late-arriving task still finishes
cleanly.
Liveness gate: a new TaskInspector probe (HasQueuedDeleteTasksForKnowledge)
matches knowledge:list_delete batch payloads — the per-parse matcher
cannot see them. A live queued/active delete defers the row (backlog,
not stranded); a probe error defers that row; a nil inspector defers the
whole sweep. Lite mode's noop returns false by design: inline deletes
never queue, so a stale deleting row there genuinely has no task.
Tests: sweep matrix (stranded recovered / backlogged kept / fresh
untouched / probe-error deferred / nil-inspector deferred), count
alignment, and the payload matcher table.
Refs #3345, #3338
* refactor(frontend): batch 1 of style consistency cleanup
- theme.css: alias --td-font-family-mono to --app-font-family-mono so the
29 monospace call sites follow the user's code-font setting
- replace 115 hardcoded rgba(7,192,95,x) brand tints with
color-mix(in srgb, var(--td-brand-color) N%, transparent) so they track
the dark-mode brand color
- strip 358 dead var(--td-*, fallback) fallbacks (tokens are guaranteed by
theme.css); keep the 3 undefined tokens untouched
- menu.vue: drop 9 unreferenced style blocks; remove TencentSans font file
and its global import
- AgentStreamDisplay: drop 6 unused keyframes; move the teleported wiki
drawer styles into components/css/wiki-graph-drawer.less imported by
both chat and WikiBrowser so styling no longer depends on load order
- creatChat: remove four !important fixed-width media queries; chat root no
longer forces font-size: 20px
- list pages: create buttons use native theme="primary" (drops the purple
gradient on AgentList); header-action-btn no longer needs !important
- add styleGuard.test.mjs ratchet test (brand rgba, legacy blue, token
fallbacks, z-index !important, raster icons)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(frontend): unify destructive confirmations on useConfirmDelete
- useConfirmDelete now uses the danger dialog theme with a red confirm
button; it is the single entry point for delete/leave confirmations
- AgentList, KnowledgeBaseList and OrganizationList drop their hand-rolled
del-*-dialog (span-as-button, circle.png) in favour of useConfirmDelete;
circle.png asset removed
- FAQ single-entry delete gains a confirmation (it deleted immediately);
new knowledgeEditor.faq.confirmDelete key in all five locales
- VectorStore/StorageBackend delete dialogs and the generated-question
popconfirm use the danger theme like the rest of the app
- style guard raster-icon baseline lowered to 10
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(frontend): shared modal shell with Esc, overlay close and unsaved guard
Add composables/useModalShell and wire it into the four full-screen
settings-style modals (Settings, AgentEditorModal,
OrganizationSettingsModal, KnowledgeBaseEditorModal):
- Esc closes the modal everywhere (previously only Settings); it defers
to any open TDesign dialog/drawer/popup and to the agent editor's own
placeholder popups
- overlay click closes everywhere (Settings gains it)
- editors snapshot their form after load/save; closing with unsaved
changes now asks "有未保存的更改 / 放弃更改 / 继续编辑" instead of
silently discarding
- new common.unsavedChanges.* keys in all five locales
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(frontend): mark instant-effect sections in the knowledge base editor
Share, data source and activity sections apply changes immediately, but
the editor footer still showed Cancel / Save, so users either re-saved
or assumed Cancel would undo. Those sections now show an "applies
immediately" note with a single Close button; other sections keep the
Cancel / Save footer. New knowledgeEditor.footer.instantEffect key in all
five locales.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(frontend): consistent success wording and confirm labels (zh-CN)
- toasts use the "已X" form everywhere: 25 zh-CN values such as
保存成功 / 删除成功 / 智能体创建成功 become 已保存 / 已删除 / 已创建智能体;
common.copySuccess now reads the same as common.copied
- confirm buttons in dialogs and popconfirms use the bare verb
(删除 / 重置 / 移除 / 应用 ...) instead of 确认X; dialog titles keep 确认删除
- GeneralSettings no longer pops a bare "成功" toast after theme / font /
font-size changes, which are self-evident and applied instantly
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(frontend): extract SettingsModalShell for the four settings-style modals
Settings, AgentEditorModal, OrganizationSettingsModal and
KnowledgeBaseEditorModal each carried a copy of the same overlay / panel /
sidebar-nav / footer markup and ~20 CSS blocks, already drifting
(1000/1080/1100px widths, three shadows, different nav paddings).
components/SettingsModalShell.vue now owns that chrome: one 1080x780
panel, grouped nav with badges, optional loading overlay, footer with a
note slot and an actions slot. Consumers keep only their content and
wire `@close` to their useModalShell.requestClose. Settings keeps its
sub-menu nav through the `nav` slot; OrganizationSettingsModal keeps its
image icon through the `nav-icon` slot.
Net: -1025 lines across the four modals.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(frontend): shared resource-card mixins and EmptyState for list pages
KnowledgeBaseList, AgentList and OrganizationList each re-declared the
same 136px card (border, radius, shadow, hover, header/title/description/
bottom, more-button reveal, feature badges) with small drifts, plus a
copy of the empty-state block using the upload illustration for every
kind of emptiness.
- components/css/resource-card.less: .resource-card(),
.resource-card-grid() and .resource-feature-badge() mixins; the pages
keep only their own variants (type gradients, favourite star,
decoration). More-buttons stay visible on touch devices.
- components/EmptyState.vue: icon / title / description / actions; the
11 empty states now use a matching TDesign icon instead of upload.svg
- badge accent colours move from hard-coded old-blue / material orange /
pink / slate rgba to color-mix over theme tokens (new
--app-accent-purple in theme.css); style guard legacy-blue baseline
drops from 40 to 18
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(frontend): shared settings-section mixins for header, row, info, control
25 settings/system/integration pages each re-declared .section-header,
.setting-row, .setting-info and .setting-control with five different
header margins (18-32px), three description sizes and assorted paddings.
components/css/settings-section.less now provides
.settings-section-header() / .setting-row() / .setting-info() /
.setting-control() / .setting-save-state(); pages keep only their
deliberate deviations (e.g. Ollama's column control, TenantInfo's fixed
label column). PlatformAPIKeys' style block switches to less so it can
use the mixins. Net -570 lines.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(frontend): provider-card mixins, dark-safe vendor badges, one spin keyframe
- components/css/provider-card.less: .provider-card() /
-interactive() / -add() / -badge() / -badge-color() / -body() /
-header() / -title() / -subtitle() / -more() shared by the model,
vector store, web search, storage, parser and sandbox settings cards;
MCP and skill cards reuse the 26px badge variant
- vendor badge colours are now `.provider-card-badge-color(#hex)`: the
foreground is mixed 75/25 with the text colour so it stays readable in
dark mode, the background is the same colour at 12% alpha
- 18 copies of a rotate-360 keyframe (steer-spin, chat-header-spin,
sandbox-terminal-spin, ...) collapse into a global `wk-spin` in
theme.css; 23 files updated, one source test adjusted
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(frontend): use TDesign password toggle for API keys; align chat toolbar buttons
- ModelEditorDialog: the API key field is a plain t-input type="password"
and relies on TDesign's built-in show/hide eye instead of a hand-rolled
suffix toggle, matching every other secret field in the app
- chat answer toolbar buttons are 28px with 6px radius, the same size as
the user-message actions and the input control bar
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(frontend): app design tokens, tokenized literals, global TDesign overrides
theme.css gains an app-level token set derived from the de-facto scale
already used in the codebase, so values do not change:
--app-radius-{xs,sm,md,lg,xl,pill} (4/6/8/10/12/999px)
--app-text-{2xs..4xl} (10..24px)
--app-space-{1..10} (4px grid)
--app-motion-{instant,fast,base,slow} (120/150/200/300ms)
--z-{sticky,dropdown,modal,drawer,popover,toast}
A codemod mapped 675 border-radius, 1904 font-size and 551 transition
duration literals in 191 files onto those tokens (value-preserving; five
source-text tests now accept either form).
assets/theme/tdesign-overrides.less holds global-intent TDesign
overrides; its `.wk-popover` skin replaces six per-file copies of the
blurred popover panel (TenantMembers x2, UserProfile,
OrganizationSettingsModal, systemAdminDialog.less,
share-to-space-panel.overlay.less) without !important.
styleGuard ratchets now also cover radius / font-size / motion literals,
`transition: all`, total `!important` and z-index !important.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(frontend): keep symmetric side insets for the chat column on narrow viewports
.chat had padding 0 0 20px 20px so the scrollbar could sit on the content
edge, but that also left the message list and input box flush against the
right edge whenever the viewport was narrower than 960px + sidebar. The
inset now lives on .msg_list and .input-container (--chat-content-inset,
20px each side, max-width 960px + insets); the scroll container still
spans the full column so the scrollbar stays on the edge. Wide layouts are
unchanged (960px content, centered).
* chore(frontend): strip trailing whitespace flagged by pre-push
* test(frontend): accept tokenized font-size and shell-rendered loading overlay
Two source-text assertions only ran in CI (Node 24 discovers .test.ts,
local Node 20 does not):
- ChatQuestionMinimap: the 13px preview font is now var(--app-text-md)
- modelUsageDetails: the editors' loading overlay is rendered by
SettingsModalShell via :loading instead of an inline v-if
Both regexes now accept either form.
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The "endpoint ready, save the token" dialog stacked the token, the URL and
three client snippets into one tall modal on top of the drawer. The drawer
now has two steps like the embed and IM channel drawers: "Configure" and
"Connect". Creating an endpoint advances to the Connect step in place, where
the one-time token, the endpoint URL and the client snippets (in tabs, one
visible at a time) are read; Finish closes the drawer. Rotating a token and
the card's "Connection info" action open the same step, the latter with the
token placeholder. The separate dialog and the "show client configuration"
button are gone.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(mcp): add built-in MCP server with per-workspace endpoints
WeKnora previously exposed itself to MCP clients only through the Python
package under mcp-server/, which binds one process to one API key and
therefore to one workspace, and mirrors the REST API tool-for-tool.
This adds a native MCP server surface to the Go backend:
- New mcp_endpoints table (PostgreSQL 000100, SQLite 000020). A workspace
can publish any number of endpoints, each with its own bearer token
(SHA-256 stored, rotatable, shown once), knowledge-base allowlist, tool
allowlist, default agent and per-minute rate limit.
- Management API under /api/v1/mcp-endpoints (Viewer reads, Admin writes,
API keys need manage_channels) plus a tool catalog endpoint.
- internal/mcpserver hosts one mcp-go Streamable HTTP server in stateless
mode at /mcp/:endpoint_id. tools/list is filtered per endpoint and
tools/call re-checks the allowlist, applies rate limiting and touches
last_used_at. MCPEndpointAuth resolves the token and injects the
workspace, an mcp_endpoint principal and an API-key style scope derived
from the endpoint, so existing knowledge-base and capability checks
apply unchanged.
- Redesigned tool set (12 tools in four groups): retrieval and reading
reuse the agent's native tools; ask runs the default or a named agent
synchronously and returns a cited answer with a session_id for
follow-ups; wiki tools cover search, read and index; write tools are
off by default. Knowledge bases accept ids or names.
- Frontend: new "MCP Server" integrations tab (tab id mcpserver, to
avoid clashing with the existing MCP-client settings section) with
endpoint cards, a create/edit drawer, a one-time token dialog and
ready-to-paste snippets for Cursor/VS Code/Claude Desktop, Claude Code
and mcp-remote. Locales updated in all five languages.
- Deployment capability integrations.mcpserver wired on both sides.
- Docs: website-docs MCP page now leads with the built-in server; the
Python mcp-server README carries a deprecation notice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): wire the MCP server into DI/router and tighten authorization
The previous commit lost eight staged files to the repo pre-commit hook,
which resolves ROOT to the main checkout and re-adds the main-checkout
copies of tracked files into the worktree index. This commit restores them
(container providers, RouterParams and route registration, context key,
principal, clone table entry, deployment capability key, SQLite parity
test) and addresses the review findings:
- Knowledge-base scope is authorized, not trusted: endpoint allowlists and
every tool call resolve each knowledge base through access.ResolveKB for
the calling principal (ownership or organization share). Foreign IDs are
rejected on create/update and dropped at call time.
- Write tools mint explicit editor grants via access.KBAccess.WithGrant
before calling the knowledge service, so add/update/delete go through the
same RequireKBWrite path as the REST API. add_document also checks the
grant itself.
- ask no longer accepts agent_id. It runs only the endpoint's configured
agent; tenant-owned agents and user-facing builtins qualify, internal
builtins (wiki fixer, skill installer) are refused both on save and at
call time. Web search is forced off for MCP calls.
- Tool annotations match behaviour: ask is not read-only, add_document is
a mutation.
- CORS allows MCP-Protocol-Version / Mcp-Session-Id / Last-Event-ID and
exposes Mcp-Session-Id for browser-based MCP clients.
- Rate limiter runs its local cleanup goroutine; SQLite migration gains the
deleted_at index for parity with PostgreSQL; non-ASCII endpoint names get
unique mcpServers keys derived from the endpoint id.
- Swagger annotations on the management handler.
- Tests: KB isolation (service and server), write grants, endpoint-agent
restriction, tool annotations, and a router test that mounts the public
/mcp/:endpoint_id surface and checks the deployment capability.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(mcp): renumber migrations after rebase onto main
main took 000100 / 000020 for chunks_index_diet, so the mcp_endpoints
migrations move to 000101 (PostgreSQL) and 000021 (SQLite); the SQLite
parity test and docs follow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): run document tools under the knowledge base owner tenant
Shared knowledge bases resolved through access.ResolveKB were only half
usable: documents of an organization-shared knowledge base are stored under
the owner tenant, so the document-level tools that queried by the calling
tenant found nothing, and add_document would have created rows under the
caller's tenant inside someone else's knowledge base.
- knowledgeInScope loads documents tenant-agnostically and then requires
the document to belong to an allowed knowledge base and to that base's
tenant.
- scopedKBContext replaces grantContext: it resolves the grant, switches
the execution tenant to the knowledge base owner (KBAccess.Context) and
swaps TenantInfo to the owner so storage/model settings resolve there.
list_documents, read_document and every write tool run under it.
- Locale text and docs no longer claim clients can pass agent_id.
- Go SDK gains MCP endpoint methods (client/mcp_endpoint.go) with docs, and
the Swagger artifacts are regenerated.
- Tests: shared knowledge base read/write under the owner tenant, spoofed
document tenant rejected, viewer share cannot write.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(mcp): renumber migrations after rebase onto main
main now carries 000101 / 000021 (knowledge_profiles), so the
mcp_endpoints migrations move to 000102 (PostgreSQL) and 000022 (SQLite);
the SQLite parity test expects version 22 and the docs follow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): guard last_used touch without a repository and wrap SDK line
The transport tests build the server without a repository; the background
last_used_at write then dereferenced a nil interface and, on the slower CI
runner, took the test binary down. touchLastUsed now returns early when no
repository is wired, and the tests inject a recording repository and assert
the touch instead. Also wraps a 123-column SDK signature for the client
module's lll check.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* feat(knowledge): derive knowledge base descriptions from document profiles
Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.
Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.
Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.
Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.
Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.
Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.
* fix(knowledge): keep generated descriptions out of duplicates and refresh clones
DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.
CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.
* fix(knowledge): refresh the loaded clone target and carry document profiles
ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.
cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
* refactor(wiki): consolidate source_refs parsing and resolution
source_refs entries ("uuid" or "uuid|title") were parsed by ten
independent implementations that disagreed on whitespace handling, on
whether a model-supplied title could be trusted, and on how a leading
pipe was treated. The scope-enforced and legacy agent resolvers also
duplicated the lookup and the routing step looked every document up a
second time.
- types.ParseWikiSourceRef / FormatWikiSourceRef / WikiSourceRefMatchesKnowledge
are now the only readers and writer of the format.
- resolveWikiSourceDocuments replaces resolveAuthorizedSourceRefs,
resolveSourceRefs and wikiKnowledgeBasesForSourceRefs: one lookup per
document, titles always rebuilt from server data, dedupe by canonical ID,
unknown documents fail closed in every mode.
- removeSourceRef now matches padded or titled refs of a deleted document.
* refactor(data_analysis): expose the selected document as a fixed table name
The tool required the document ID inside the SQL text and blindly
string-replaced it with the physical DuckDB table name, which forced the
model-context codec to rewrite handles inside quoted SQL and let the
UUID-derived k_<uuid> table name reach the model through stored schema
summaries.
The document is now selected only by knowledge_id. Every query runs on a
dedicated connection where "dataset" is a temporary view over the
physical table (DuckDB temporary objects are connection-local, so
concurrent sessions stay isolated). The physical name remains accepted
for internal callers; any other table fails validation with an error
naming the table the model should use. Schema summaries, the schema
tool, the analyst prompt and the tool schema all name the same table, so
data_analysis.sql no longer carries source handles and its text codec
policy is removed.
* feat(modelcontext): report raw identifiers that survive encoding
Text-level compaction exists because raw UUIDs reach model context through
producers the registry never sees: error strings, slugs, results of
unknown shape. Retiring it requires knowing where those producers are.
Registry.LeakedIdentifiers scans an encoded message list for UUID-shaped
values in every application-composed field (user messages are skipped),
classifies each as registered (codec missed a rewrite) or unregistered
(producer emitted a raw ID), and SummarizeLeaks renders one line per
field. The agent engine and the three RAG pipeline call sites log the
report as a warning after EncodeMessages.
* fix(chat_pipeline): align RAG data analysis with the fixed table contract
The RAG data-analysis plugin still asked the planning model to fill in
knowledge_id and to write SQL from a prompt that exposed the raw
knowledge UUID, then passed the model's JSON straight to Execute. Under
the new data_analysis contract a query that names anything but the
dataset table is rejected, and the plugin logged that rejection at error
level and moved on, so the analysis vanished without any signal. A
"no analysis needed" plan took the same error path.
The pipeline now owns document selection: the model returns only
needs_analysis and sql, the prompt names the dataset table and never
shows the knowledge ID or physical table, and the plugin builds the tool
input itself. A rejected query is fed back to the model once for repair;
a query that still fails is reported through pipelineWarn with the SQL
and the tool error, and "not needed" is logged as such instead of as a
failure.
Covered by unit tests with a scripted model and executor: not-needed,
pipeline-owned knowledge_id, one-shot repair, persistent failure
surfaced, unparseable plan rejected.
* perf(chunk): speed up chunk ingestion and batch updates
Chunk writes were dominated by avoidable work in the repository layer:
- UpdateChunks built one CASE WHEN id = ? chain per column, so a batch of
N rows evaluated 5N branches per row (O(N²)). On PostgreSQL it now joins
against a VALUES list keyed by the primary key (O(N), batched by 1000);
other dialects run one small UPDATE per row inside a single transaction.
- CreateChunks copied Content into SourceContent for every new row, doubling
insert volume. SourceContent is only needed once a chunk is edited and the
edit path already backfills it lazily, so it is left empty on create.
Batch size raised from 100 to 500 and the insert (including the SQLite
seq_id pre-assignment) now runs inside one transaction.
- New UpdateChunkFieldsByIDs applies uniform field values with a single
UPDATE ... WHERE id IN per batch. FAQ import uses it to flip status after
indexing instead of shipping every row's content back, and resolves tag
names per batch instead of per entry.
- Drop three chunks indexes that only slow down writes: chunk_type (low
cardinality), content_hash (never used as a filter) and
(tenant_id, knowledge_id) (covered by idx_chunks_knowledge_enabled).
Migrations 000100 (PostgreSQL) and 000020 (SQLite) with down scripts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(chunk): rewrite chunk batch writes and FAQ import status updates
Go changes belonging to the chunk write-path optimization. The previous
commit on this branch only carried the migrations because the repo's
pre-commit hook re-staged files from the main checkout when run inside a
git worktree, silently dropping the staged Go changes.
- UpdateChunks: O(N²) CASE chains replaced by a VALUES join on PostgreSQL
(batched by 1000) and per-row updates in one transaction elsewhere.
- CreateChunks: stop copying Content into SourceContent, batch 500 rows,
run the insert and the SQLite seq_id pre-assignment in one transaction.
- New UpdateChunkFieldsByIDs for uniform, tenant-scoped field updates.
- FAQ import flips status with UpdateChunkFieldsByIDs and resolves tag
names once per batch instead of once per entry.
- Drop the gorm index tag on ContentHash to match migration 000100;
SQLite schema test expects migration version 20.
- SQLite tests for the new and rewritten repository methods.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* feat(session): add fork schema and history queries
为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。
* feat(sandbox): add git, fork snapshots, and bootstrap hook
沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。
* feat(session): checkpoint workspace at the end of each turn
每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。
* feat(session): fork conversations and reap orphan snapshots
支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。
* feat(chat): add session fork entry, prefill, and badges
在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。
* style: add missing newline so session_fork_delete_test.go is gofmt-clean
* fix(session): prune later git history when rolling a fork workspace back
reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。
* style(session): clear golangci-lint findings on the session-fork diff
Pre-push lints the whole PR against main; wrap long lines, check c.Error,
and fix gofumpt/revive/staticcheck so the merge can be pushed.
* fix(chat): show the fork button on hover without shifting the thread
Overlay the action so revealing it no longer expands the user bubble and janks the messages below.
* fix(chat): drop the fork landing banners
Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread.
* fix(agent): dedupe artifacts by content hash, not size
Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only.
* feat(session): allow forking from an assistant answer
User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists.
* fix(session): resolve parent attachments from a forked chat
Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss.
* fix(agent): rewrite sandbox refs after a content-hash skip
Fork restores change mtime on unchanged files, so Collect returns empty
and named citations were left as sandbox: paths. Resolve those names
against session artifacts, notify only real uploads, and stamp restored
mtime only on the matching content hash.
* fix(session): drop a dead fork snapshot and retry checkpoint rewrite
Create from a missing template now abandons bootstrap so the next
Resolve uses a normal sandbox. A failed checkpoint rewrite no longer
marks the bootstrap consumed, which previously made nested forks
SANDBOX_REPLACED.
* fix(session): load forked parent attachments into the prompt
ResolveForPrompt used session-scoped lookup, so copied parent attachment
IDs passed Get but never reached the model. Use the same resolveInSession
path as preview.
* fix(chat): keep the user-message fork button visible
The control was hover-only, which hid it on touch. Always show it, and
drop the unused degrade tooltip.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
processInitializationModels inserted models built by toModel(), which
carries no TenantID, so every model the initialization endpoint had to
create landed in the models table with tenant_id = 0. The read path
filters on (tenant_id = ? OR is_builtin = true), which means no tenant —
not even the one that just configured the KB — could ever resolve those
models; the KB stayed bound to unreadable model IDs and document
processing stalled forever (issue #3333).
Set the tenant from the KB that is already in scope before CreateModel.
The reuse path (updating an already-stored model) keeps its stored
tenant. Tests pin both paths.
Closes#3333
InstallSkill returns after starting a goroutine, so asserting
status==installing immediately is flaky on fast CI. Wait for the
fixture run to finish instead; a skip would never write a new snapshot.
SearchMessagesByKeyword filtered with `messages.content ILIKE ?`, which is
PostgreSQL-only syntax. On SQLite (the Lite build) and MySQL the query fails
with `near "ILIKE": syntax error`, so POST /api/v1/messages/search returned
HTTP 500 in both keyword and the default hybrid modes. The session-list
search (QueryPaged) was already made dialect-aware for the same reason; this
path was missed.
Pick the LIKE expression per dialect — ILIKE on Postgres, LOWER() LIKE
LOWER() elsewhere — and add an explicit `ESCAPE ?` bound to likeEscapeChar so
escapeLikeKeyword's backslash escapes are honored on SQLite, which has no
default LIKE escape character (a no-op on Postgres/MySQL, where `\` is
already the default).
Regression tests in message_search_test.go run against in-memory SQLite:
TestSearchMessagesByKeywordMatchesOnSQLite and
TestSearchMessagesByKeywordTreatsWildcardsLiterally both fail on the old code
with the ILIKE syntax error and pass with the fix.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRrBenkCCr4pacMNwR4NuG
Serply returns Google results through a keyed JSON API, which fills the
gap between Google Custom Search (needs an engine ID and a restricted
scope) and the scrapers. The adapter follows brave.go: hardcoded
endpoint, redirects refused so the key never leaves the official host,
4MB response cap, and per-call country and freshness filters. country
maps to Google's gl parameter and pd/pw/pm/py map to tbs qdr:d/w/m/y.
Serply has no date-range filter, so that form returns an error instead
of being silently dropped. num is clamped to 10 because the API serves
one results page per call.
Wiring matches the Bocha provider: type constant and metadata, registry
factory, service validation, frontend union, swagger enum entries (edited
by hand; a full swag regen drags in unrelated drift), and the web search
docs table.