Commit Graph

625 Commits (014f95f1394a7bfd1c7bbbdcdc19057a80dcaca5)

Author SHA1 Message Date
lyingbug b82fd6c0f9
fix(security): close the low-severity shared-space findings (#3416)
* fix(security): close the low-severity shared-space findings

Verified and fixed from the shared-space permission review:

- Embed channels kept knowledge_ids, tag_ids, mentioned_items, skill_names
  and summary_model_id from anonymous visitors. Mentions are only scoped
  for shared agents, so a visitor could query any KB (and pick any model)
  of the channel workspace by ID. The payload now drops them; the widget
  never sends them.
- PUT /knowledgebase/:kb_id/wiki/issues/:issue_id/status updated any issue
  by ID. The update is now scoped to the path's KB (404 otherwise).
- A direct org invite let the inviter attach any user of the target
  workspace (or any user by ID) as its representative, whose username and
  email the roster then showed to all members. Direct adds attach no
  representative, and the roster returns emails only for the caller's own
  workspace.
- COS and KS3 dropped the bucket (and COS the region) segment when turning
  a path into an object key, while the tenant check scans every segment:
  cos://42/<region>/prefix/7/... passed as tenant 42 and read tenant 7's
  object from a shared bucket. Both must now match the service's config.
- Shared agent listings returned the full agent to receivers. Prompts and
  the creator's user ID are withheld; capabilities and scope stay visible.
- Temporary document uploads let the caller pick the parser engine (and
  its credentials) of a shared agent's workspace; only the agent's rules
  choose one there now.
- Suggested questions used request / message-snapshot knowledge_ids
  without a permission check. Each document is now resolved to its KB and
  checked like KB IDs, including a shared agent's scope.
- IM channels accepted any knowledge_base_id (and any agent ID on create).
  Both must belong to the channel's workspace, the KB also to a restricted
  API key's allow-list.
- KB-restricted API keys could list org-shared KBs outside their allow-list
  in the organization views; those listings are now filtered.
- With enable_rbac=false, RequireKBAccess let cross-workspace denials
  through. That switch is for roles inside a workspace; the guard now
  always denies another workspace's KB, since handlers behind it load by
  ID without a tenant filter.
- The org member limit was counted before and enforced apart from the
  insert; it is now checked in the insert's transaction with the
  organization row locked (SQLite serializes writers instead).
- The UpdateOrganization route comment and API doc said owner-only; the
  service and UI allow org admins. Comment and doc corrected.

Documented, not changed: issued /r/ and presigned links stay valid until
they expire after a share is revoked.

* fix(security): address review of the low-severity shared-space fixes

- Embed payloads also drop question_origin. It selects a KB like an
  @mention for agents that retrieve only when mentioned, and the widget
  never sends it; the selection stays inside the agent's scope, but an
  anonymous visitor should not pick among its KBs. suggestion_attribution is
  kept (the widget sends it and it is validated).
- The receiver view of a shared agent also withholds the follow-up
  suggestion instruction, a prompt like the others. Starter questions are
  shown to receivers anyway, and the sharer's name is displayed by design.
- Follow-up suggestion generation moves into the agent's workspace with
  types.WithExecutionTenant instead of a bare tenant rewrite, so the
  caller is pinned before the document permission check runs. The auth
  middleware and the shared run already pinned it; this removes the
  dependency on that.
- KS3 CopyFile copies only from the service's own bucket (as COS and S3
  do); another bucket's path is a cross-backend copy.

Tests cover the dropped question_origin, the follow-up instruction and the
/shared-agents listing, a follow-up context with the tenant switched, a
repeated wiki issue status update, and the KS3 copy source.
2026-09-18 22:58:46 +08:00
lyingbug 03d9d66a1f
feat(agent): size agent history by the window and calibrate token estimates (#3413)
* feat(agent): size agent history by the window and calibrate token estimates

Agent history was capped at history_turns (default 5) because compaction
could not survive a turn. With checkpoints persisted, load history by
tokens instead, and let compaction bound the session.

Budget. The loader drops the oldest turns that do not fit, and what it
drops is lost: neither replayed nor summarized. A budget equal to the
compaction threshold (the earlier attempt, since reverted) let the loader
trim first, so whenever one turn was larger than the system prompt plus
the new question the request never crossed the threshold; nothing was
summarized and the session became a sliding window with no checkpoint.
agent.HistoryTokenBudget is now the whole window: overflow reaches the
first round's compaction, which summarizes it into a new checkpoint.
A turn-by-turn regression test drives the real loader, budget and
compactor and fails as soon as a stored turn after the checkpoint is
missing from the loaded history; restoring the old budget fails it.

Summarizer input. History can now reach the window, so the part being
summarized can outgrow one request. An overflowing request degrades to a
raw archive that is never persisted, so every later turn would fail the
same way. Prepare keeps the newest messages that fit, the prompt states
how many were left out, and file paths still come from all of them.

Calibration. cl100k counts Chinese at about a token a character where
Qwen and DeepSeek spend about 0.6. The estimator now carries a scale
(clamped to 0.5-1.5, text only), learned from consecutive requests in a
turn: they share tools, system prompt and earlier messages, so the
provider's growth between them prices exactly the appended messages,
without mixing in how a provider renders tool schemas. The turn's scale
is stored on its usage (context_token_scale); the next turn's loader
prices history with it and hands it to the engine. Loading, the first
compaction check and compaction itself share one estimator, so they
agree; calibrating only the trigger would let the loader drop turns
again, which the regression test covers.

Also: history is priced as the engine sends it (redacted KB/Wiki results
cost one line); a checkpoint older than the rows read is matched by the
assistant rows' (created_at, id) order; a read ending on a turn's
question keeps that turn; a compaction discarded for freeing under 5%
still persists its checkpoint; a checkpoint write matching no row is an
error; a partial index backs the newest-checkpoint lookup. Agent mode no
longer reads history_turns: the field leaves AgentConfig, the editor
hides it in smart-reasoning mode, and quick-answer keeps its turn cap.

* fix(agent): address review of window-sized history loading

Summarization. The summarizer call had a fixed 60s total timeout. With
history loaded up to the window, a request could carry ~175k tokens and be
cut off while prefilling or writing normally; the degraded result left no
checkpoint, so the next turn reloaded the same history and failed the same
way. Summarization now streams and is cancelled only after no output for the
engine's stall timeout (120s by default, LLMCallTimeout overrides), with the
overall ceiling left to the transport, as for the engine's own rounds and as
in Codex and Pi. A stopped turn is not retried. When the summarizer still
fails, the raw archive is persisted as a checkpoint marked degraded so the
loop is broken, and the next compaction refines it as the previous summary.
The archive is held to the summary budget, newest messages first, so a
degraded compaction always frees room.

Loading. The loader priced turns as sent but returned and held them as
stored, so wiki pages stored in full and sent as one line could pile up in
memory. It now returns the as-sent history and releases each turn's stored
rows once replayed, keeping about a page of rows at a time.

Index. The partial checkpoint index is replaced by a composite
(session_id, created_at DESC, id DESC) index serving the backwards page,
the newest-checkpoint lookup and the recent-messages reads. It is built with
CREATE INDEX CONCURRENTLY so writes to messages are not blocked; the
migration holds that single statement, since a multi-statement query runs as
an implicit transaction. A PostgreSQL integration test, skipped without
WEKNORA_MIGRATION_TEST_POSTGRES_DSN, applies the chain, runs 000106 down and
up through golang-migrate, and checks both queries use the index without a
sort; the SQLite migration test checks the same plans.

Calibration. It no longer waits for a total token count, and a usage with
only a scale is persisted. A turn that measures nothing carries forward the
scale it started from, so the newest turn always has the latest one.
2026-09-18 22:51:31 +08:00
lyingbug db8e7611be
fix(security): close cross-workspace access paths in shared spaces (#3408)
* fix(security): close cross-workspace access paths in shared spaces

A review of how shared-space (organization) members reach each other's
resources found six paths that bypass the org-share permission model:

- presigned-preview signed any resource handle or storage path, turning a
  known resource:// handle into an anonymous /r/ URL that outlives share
  revocation. It now requires the path to belong to the caller's tenant,
  like /files.
- POST /initialization/initialize let a shared-KB editor rewrite the owner
  workspace's models (base URL, API key) in place. Only the KB's own
  workspace may call it, and rewriting a stored model needs the same
  authority as PUT /models/:id.
- KB JSON carried the legacy inline credentials (storage_config
  secret_id/secret_key, which initialization fills with the MinIO env
  keys, and vlm_config api_key) to every share receiver. They are withheld
  from JSON; their DB columns are unchanged.
- A chat without agent_source_tenant_id resolved shares before the
  caller's own agent, so a member sharing a same-ID built-in agent took
  over other workspaces' default agent. Own agents now win, built-in
  agents can no longer be shared (existing such shares no longer resolve),
  and embed channels drop a visitor-supplied source workspace.
- Any org-admin tenant (even a tenant Viewer inside it) could raise another
  workspace's KB share to editor/admin, and the route guard was bypassable
  via an unrelated :id. Share management is now: the sharer from the source
  tenant (Contributor+), source-tenant Admin+, or an org-admin tenant's
  Admin+ that may only remove or lower. share_id must belong to :id.
- Wiki write tools were registered for every wiki KB in the search targets,
  which only need read access, so a viewer share (e.g. via the built-in
  wiki fixer) could edit or delete another workspace's wiki pages. They now
  act only on KBs the caller may edit.

* fix(security): close the KB settings path and owner details for share receivers

Follow-up to the shared-space review:

- PUT /initialization/config shares the route guard that admits share
  editors, so an editor could still rebind the owner's KB models, chunking
  and storage, and storage backends resolved against the caller's workspace
  let it bind the owner's KB to its own backend. Another workspace now needs
  an admin share (the only share level the frontend offers settings to) and
  may not change the storage binding at all.
- GET /initialization/config showed model base URLs and bucket locations to
  any admin of the caller's own workspace, including share receivers. They
  now only see whether credentials are configured.
- Wiki write tools on the caller's own KBs had no role floor, so a tenant
  Viewer (and the IM, embed and MCP endpoint principals that run as Viewer)
  could write through an agent what the HTTP routes deny. Writes now need
  Contributor+, or the ingest capability for a scoped API key, following
  the RBAC rollout switch.
- resolveAgent logs the share lookup error instead of a nil own-agent error
  when a shared selector fails.

* fix(security): close the medium shared-space access findings

Follow-up to the high-risk fixes, from the same review:

- Wiki fixer moved into a shared KB's workspace ran with the caller's own
  config there: an unset MCP mode registered all of the owner's MCP
  services, and an "all" KB scope reached the owner's other KBs. The scoped
  run now uses the built-in config pinned to that one KB, without MCP,
  skills, sandbox or web search, and falls back to the KB's own model.
- An unset MCP mode means "all" at runtime but is shown as none in the share
  scope; shared runs now treat it as none.
- Quick-answer mode used the request's web_search_enabled alone, so a
  receiver could turn on search the agent disables and spend the owner's
  provider quota. It now requires the agent setting too, as agent mode does.
- summary_model_id picked any model of the workspace a run executes in. It
  is ignored for shared agents and the relocated wiki fixer.
- A workspace that left or was removed from an organization kept sharing its
  KBs and agents. Its shares are revoked on departure, and share queries and
  permission checks require the source workspace's membership and a live
  organization, which also covers rows left behind earlier.
- Sharing an agent exposes its KB scope to all members, so a Contributor
  could share colleagues' KBs (or "all") through an agent. The sharer must
  now be able to share each KB directly (creator or Admin+; "all" needs
  Admin+), also when a shared agent's scope is widened later.
- A scoped API key with manage_channels could mint an MCP endpoint token
  wider than itself. Endpoints it creates, updates or rotates must stay
  within its KB allow-list and capabilities.
- A shared agent's run indexed the receiver's Q&A into the owner's chat
  history KB; indexing now uses the session owner's configuration.
- Sharing an agent with skills keeps injecting the owner's skill
  environment values (product decision); the share panel now warns that
  members can read them. Preview of originals through read-only shares is
  kept as designed and documented.
2026-09-18 22:00:02 +08:00
lyingbug 62f7da5eea
feat(agent): persist compaction checkpoints across turns (#3407)
* feat(agent): persist compaction checkpoints across turns

Agent compaction only lived inside the turn that ran it: the summary was
discarded at turn end and LoadAgentHistory rebuilt raw history from the
messages table, so every later turn past the threshold paid for another
summarization of nearly the same history.

A compaction whose history summary ends exactly on a stored turn is now
written onto that turn's assistant message (messages.context_checkpoint).
LoadAgentHistory starts from the newest checkpoint: the summary replaces
that turn and everything before it, and later turns are replayed verbatim.
The compactor already treats a leading summary as the previous summary, so
checkpoints fold forward incrementally.

History messages carry their turn's assistant message ID (chat.Message
TurnID, engine-internal) so compaction can tell whether its cut lands on a
stored-turn boundary. Nothing is persisted when the cut ends inside a stored
turn, covers only the live turn, or the history summary degraded to a raw
archive; a split turn's prefix summary is never persisted because that turn
is replayed whole next time. Writes are column-scoped and best-effort, and a
failed lookup falls back to history without a checkpoint.

Storing the checkpoint on the covered turn keeps the boundary implicit, so
session forks copy it with the turn and deleting the turn drops it. The
HistoryTurns cap still applies, now to the turns after the checkpoint.

* test(skill): wait for the background install in the name-conflict test

InstallSkill queues the install on a goroutine, and the fixture row already
starts at installing, so reading the status right after the call raced the
mock install and failed whenever it landed on ready first. Wait for the run
to finish on the reused row instead, as the other install tests do.

* feat(agent): size agent history by tokens instead of turn count

Agent history was capped at history_turns (default 5) because every turn
used to rebuild raw history and compaction could not persist. With
compaction checkpoints persisted, the cap is no longer what keeps a session
inside the window, and it threw away context that would have fit.

LoadAgentHistory now loads the newest checkpoint summary plus the newest
turns after it that fit a token budget: the run's compaction threshold
(agent.HistoryTokenBudget, the window less the reply reserve). History
that large already triggers the first round's compaction, which persists a
new checkpoint, so compaction rather than a turn count bounds the session.
Loading more would only hand that compaction a summarizer input larger
than the model can read.

Rows are paged backwards by (created_at, id) and the read stops at the
checkpoint or once the budget is full, so a long session is never read
whole. Kept turns stay contiguous, the newest turn is kept even when it
alone exceeds the budget (compaction splits it), and a turn cut by the read
is left out rather than replayed without its original question.

Agent mode no longer reads history_turns: AgentConfig.HistoryTurns is
removed, the editor hides the input in smart-reasoning mode and explains
the automatic behavior, and the smart-reasoning builtins drop the setting.
Quick-answer (KnowledgeQA) keeps its turn cap since it has no compaction.

* Revert "feat(agent): size agent history by tokens instead of turn count"

This reverts commit d05fb6e5f1f161813a88e9127609f7a059f38f12.

Loading history up to the compaction threshold lets the loader drop the
oldest turns before compaction can trigger: whenever a single turn's
replay is larger than the system prompt plus the new question, the
trimmed history stays under the threshold, nothing is summarized, and the
session degrades into a large sliding window that never gets a
checkpoint. A turn-by-turn simulation of LoadAgentHistory with the real
compactor showed zero compactions over 60 turns in those cases.

Keep the persisted checkpoints with the HistoryTurns cap for now; token
budgeted history returns in a separate change with a budget that leaves
the overflow to compaction.
2026-09-18 20:38:56 +08:00
lyingbug 68e6c03dc4
fix(skills): upgrade outdated sandbox skills without rolling back the catalog or taking them offline (#3406)
* fix(skills): keep retries and catalog installs from rewriting the skill definition

Every install path wrote its archive back to the workspace catalog. After a
skill was re-registered as v2, sandboxes still on v1 are pinned to v1's
archive, so retrying one of them (a failed retry, or a ready skill retried
with guidance) reinstalled v1 and also moved the catalog definition back to
v1 and deleted v2's archive.

Installs now carry where their archive came from. Only an upload or a source
pull is a new definition; a retry or a catalog install uses bytes the
workspace already stores and never rewrites the definition. A retry of a
pinned install keeps its pin and replays its own version, and bytes that
neither the definition nor the row holds are refused with a conflict.

The catalog listing also reports each installation's version so the UI can
show what an upgrade moves from and to.

* feat(frontend): upgrade outdated skill installs from the catalog, sandbox drawer and agent editor

An install whose archive differs from the catalog was flagged as outdated,
but nothing could move it to the catalog version: the install drawer hid
ready installs, and the manage drawer only offered a retry, which replays
the sandbox's own archive.

- Skill settings: cards with outdated installs show an upgrade button that
  opens the install drawer with every outdated sandbox preselected. The
  add wizard and install drawer also let outdated ready installs be picked,
  labelled with the version they move from and to.
- Sandbox skill drawer: an outdated skill shows a "new version" row whose
  upgrade button installs the catalog onto that one sandbox.
- Agent editor: skills on the chosen sandbox that are outdated show the
  version change and an upgrade button, both for admins only, matching the
  admin-only catalog install. The install progress entry, which opens a
  drawer backed by admin-only endpoints, is now admin-only too.
- A skill that is being upgraded is no longer pruned from the agent's
  selected skills while it is briefly not ready.

The outdated check is shared in utils/skillUpgrade.ts.

* feat(skills): keep the previous version serving while an upgrade runs or after it fails

An install rewrote the row to the new version and marked it installing as
soon as it started, and the agent is only offered ready rows. So an upgrade
took the skill away from every sandbox for its whole run, and a failed
upgrade kept it away, even though the image pointer only moves on success
and the previous version was in the image the whole time.

The row now records what it described when it was last ready (served:
version, description, instructions, digest, the archive its files come from,
and the snapshot that installed it). The agent's usable set, member env
acceptance and the @-mention list read TenantSkillEntity.ServedView, which
is that version marked ready while the row is installing or failed. A
successful install clears it and releases its archive; a failure or a stop
keeps it.

Keeping that consistent:
- The served archive counts as a reader when a bundle is released or
  replaced by a re-registered catalog, so v1's files stay readable while v2
  installs.
- The reaper and a failed removal put the served version back when the image
  never moved past it, instead of labelling v1's files with the upgrade's
  metadata. A stuck upgrade whose chain cannot be read is left alone, since
  v1 keeps serving meanwhile.
- The env declaration is stored after the pointer moves rather than before
  the snapshot, so an upgrade that fails late cannot drop the admin value of
  a variable the previous version still reads.

Migration 000104 adds tenant_skills.served. The console says "upgrading,
still running 1.1.0" or "upgrade failed, still running 1.1.0", the agent
editor keeps such skills selectable, and the upgrade texts no longer warn
that the skill goes offline.

* fix(skills): offer only the upgrade to installs the catalog has moved past

Review follow-up on the upgrade flow.

- An install that differs from the catalog, ready or failed, is now offered
  the upgrade and not the retries. The retry icon and the retry-with-guidance
  box both replay the archive the install is pinned to, so next to an
  Upgrade button they reinstalled the version the operator came to leave
  behind. A failed first install of v1 whose catalog has since moved to v2
  used to retry v1 from the drawer while the agent editor's Install went to
  v2; every entry now goes to v2, and the agent editor labels it "Upgrade".
  The drawer explains that the failed install is replaced by the catalog
  version.
- A catalog install or a retry reuses the definition that already holds its
  bytes instead of re-running the catalog upsert, which rewrote the row and
  bumped its updated_at every time the skill was installed somewhere. The
  upsert still runs for a definition that does not exist yet or whose object
  was lost.
2026-09-18 20:27:43 +08:00
MercuialC aaec920a9c
feat(audit): 知识库活动记录对 API Key 操作额外显示 Key 名称 (#3405)
* feat(audit): show API key name on knowledge-base activity

API-key operations currently look like the first tenant user or
"system". Snapshot the key id/name into activity details so the
feed can tell which key initiated the change, without restoring
key authorization onto async workers.

Fixes #3402

* fix(frontend): show activity API key on a second line

The 132px actor column truncates "user · API Key · name" into one
ellipsis. Keep the person on the first line and the key name below.

---------

Co-authored-by: MercuialC <MercuialC@users.noreply.github.com>
Co-authored-by: root <root@localhost.localdomain>
2026-09-18 19:39:34 +08:00
lyingbug 2be7bd4063
fix(agent): search bound knowledge bases before answering, and pass suggested-question origins (#3400)
* fix(agent): search bound knowledge bases before answering, and pass suggested-question origins

A user picked the suggested question "为什么在比较图形时需要谨慎处理?"
and the agent answered in round 1 with no tool call, claiming it had
searched the knowledge base and found nothing, then explained misleading
statistical charts. The question had been generated from "Corners Don't
Look Like That: Regarding Screenspace Ambient Occlusion" in a bound base,
where "comparing graphs" means SSAO falloff curves.

Two things let that happen: the grounding prompt said "Stable general
explanations need no lookup", and nothing told the agent where the
question came from - the base in question had no profile, only two
recent titles in the directory, and the click sent the text alone.

Prompt
- With knowledge tools available, search bound bases before answering a
  question about a topic, even one that reads like general knowledge;
  directory profiles and recent titles cannot prove a base lacks it.
- The general-explanation exemption applies only without bound bases.
- Say you searched or found nothing only when a search tool ran.

Question origin
- Suggested questions from FAQ/document chunks also return knowledge_id.
- Picking one on the home or chat page sends question_origin
  {knowledge_base_id, knowledge_id} with that question only.
- The service keeps the origin only when its base is inside the turn's
  resolved scope and the document belongs to that base; it never
  changes the scope (unlike knowledge_base_ids, which narrows it).
- The engine renders it in runtime_context as <question_origin> with the
  base and document handles and a note to search it before answering.

* fix(agent): keep question origins across file/tag scopes and bind them to the send

Review follow-up for the suggested-question origin.

- Scope check: the origin was kept only when its base was in the resolved
  KB list, which is empty for file-only and tag-only scopes even though
  their bases are searched. It is now checked against SearchTargets after
  they are built; the document is kept only when a target for that base
  covers it (unfiltered base, or a document scope listing it).
  resolveQuestionOriginInfo re-checks SearchTargets, looks up the name of
  a base that is not in kbInfos, and the renderer omits an unknown name.
- Frontend: the origin travelled through module-level "pending" state, so
  a send that returned early (agent not ready, upload in progress,
  steering) left it behind and a second pick overwrote it. It now rides
  with the one send: triggerSend(text, options) -> createSession ->
  send-msg -> sendMsg / changeFirstQuery.
- RetrieveKBOnlyWhenMentioned: suggestions are still generated from the
  agent's bases, but picking one retrieved nothing. The origin now
  selects its base, as an @mention would, when the agent may read it.
- API sync: question_origin in docs/api/chat.md and swagger;
  QuestionOrigin on the Go SDK's KnowledgeQARequest/AgentQARequest and
  knowledge_id on SuggestedQuestion.
2026-09-18 17:00:47 +08:00
lyingbug 92e7c0ab37
refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents (#3381)
* refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents

The agent's retrieval surface had grown to five overlapping tools whose
behaviour was scattered across long descriptions, interleaved "tips" in
tool output (most of which modelcontext discarded before the model saw
them), and a grep_chunks implementation that ran an unindexed regex scan
over the chunks table ordered by created_at, so large knowledge bases lost
recall and paid a full-table scan on every call.

Tool surface
- search_knowledge replaces knowledge_search and grep_chunks: one query,
  mode = hybrid | semantic | keyword, optional knowledge_base_ids, limit.
  Keyword mode runs through the retrieval engine's keyword index instead of
  the DB regex scan. Modes unavailable for the scope are refused with a
  reason. Rerank, MMR and de-duplication are kept; already_seen markers,
  retrieval_statistics and the zero-result "CRITICAL" block are gone.
- read_document replaces list_knowledge_chunks, get_document_info and
  wiki_read_source_doc: id accepts a dN or cN handle, always returns the
  document header, pages by exact offset (fixing the old floor-to-page
  bug), finds passages with query (literal by default, regex opt-in) and
  reads neighbours around a chunk with context.
- list_documents pages through one knowledge base, honouring @file/@tag
  scope.
- wiki_search takes a single query plus regex, knowledge_base_ids, limit;
  literal queries are escaped so the model never reasons about JSON
  escaping. Legacy queries / knowledge_base_id are still accepted.
- query_knowledge_graph is registered only when a graph-enabled KB is in
  scope and its description/tips were trimmed.

Compatibility
- Retired names in stored allowlists, presets and API calls are mapped to
  their successors by NormalizeAllowedTools at registration time; no data
  migration. Stored histories with old names still decode, render and
  redact. Calls to retired names return a replacement hint.
- modelcontext gains an "id" key space that decodes against both the chunk
  and document handle tables (decode-only), plus rendering for the
  document header, chunk roles, in-document matches and next_offset.

Prompts and surfaces
- The RAG, wiki researcher and hybrid templates now name the tools and the
  search -> read -> cite order; tool descriptions are short and contain no
  workflow instructions.
- Engine display names, history redaction, grounding prompt, IM rendering,
  the built-in MCP server (search_knowledge gains mode/limit, grep_chunks
  becomes keyword search, read_document gains query, wiki_search gains
  regex), frontend editor/renderers/i18n, presets, website docs and the
  changelog are updated accordingly.

Tests cover name mapping, mode validation and refusal, output shape,
read_document paging/context/query/scope, list_documents, wiki_search
escaping and legacy arguments, modelcontext decoding and rendering, and
registration-time aliasing and graph gating.

* fix(agent): address review findings on read_document, list_documents and model rendering

- read_document: fetch the second page whenever the first page did not
  reach the end of the document, so an unaligned window near the end no
  longer drops the trailing chunks (also fixes context windows around a
  chunk near the end); a query passed together with a cN handle now
  searches the owning document instead of being ignored; query mode stops
  collecting once the output budget is nearly spent and reports truncated;
  the document row is authorized in place instead of being fetched twice;
  the context read takes the total from the window fetch instead of a
  separate count query.
- list_documents: resolve the page's document scope in one batched tag
  lookup instead of one call per row.
- modelcontext: render read_document output once and read next_offset
  through the existing intValue helper.
- search_knowledge: point title/file-name lookups at list_documents.
- Tests for the boundary window, chunk context near the end, cN + query,
  and budget-bound query results.

* fix(agent): close compatibility gaps in the consolidated retrieval tools

- Wiki-only scopes: read_document and list_documents are registered
  whenever a wiki KB is in scope, since every KB stores chunks. Their
  capability requirement includes wiki but is marked auxiliary, so they do
  not widen a RAG agent's derived "all knowledge bases" filter to wiki-only
  bases (Go and frontend derivations both honour it).
- search_knowledge resolves the retrieval mode per knowledge base instead
  of refusing the call: FAQ and vector-only bases answer keyword requests
  semantically, keyword-only bases answer semantic requests by keyword.
  Results carry requested_mode and mode_fallbacks, the empty-result text
  names the fallback, and the call is refused only when no base in scope
  has a chunk index.
- The model rendering uses the mode the search actually ran with (it was
  always labelled semantic) and emits <mode_fallback> per degraded base.
- list_documents applies the @file/@tag scope before paging: tags are
  pushed into the database filter, pinned documents are loaded by ID, and
  total/next_page count only readable documents.
- read_document resolves chunk context by chunk_index through a new
  ChunkRepository.ListChunkNeighbors, so parent, summary and image chunks
  sharing the index sequence no longer shift the neighbourhood; offset is
  documented as a reading-order position, not a chunk index.
- wiki_search (agent and MCP) keeps its regular-expression default and
  only matches literally when the text is not a valid expression;
  regex=false forces a literal match and regex=true rejects invalid input.
- MCP grep_chunks keeps grep semantics: literal terms from the pattern
  fetch candidates from the keyword index (semantic for bases without
  one), and every returned chunk is verified against the pattern.

* docs(changelog): align the consolidation entry with regex wiki_search and grep_chunks semantics

* docs(mcp): note that grep_chunks recall is bounded by index candidates
2026-09-18 14:12:22 +08:00
lyingbug 6db8fdd601
feat(artifacts): artifacts library page backed by a message_artifacts table (#3384)
* feat(artifacts): add an artifacts library with its own message_artifacts table

Skill-generated files were only reachable from inside the conversation that
produced them. This adds a sidebar entry and a page that lists them across
every conversation the user can see.

Storage: artifacts move out of the messages.artifacts JSONB column into a new
message_artifacts table (PostgreSQL 000103, SQLite 000023), backfilled on
upgrade. The table is now the only source of truth; the old column is kept and
the down migration writes the table back into it. The message repository
loads artifacts for every message it returns and writes them on create,
update (when non-nil) and fork, so callers still use Message.Artifacts.
mod_time is stored as RFC 3339 text because the collector matches files on
nanosecond mtimes, which PostgreSQL timestamps would truncate.

API: GET /api/v1/artifacts lists the latest version of each file (grouped by
session + source path, with version_count), scoped like the session list,
with keyword / file type filters and paging. Downloads keep using the
per-session endpoint.

Frontend: "产物/Artifacts" sidebar entry (hidden without sandbox support), a
library page with type tabs, search, date groups, infinite loading, preview
drawer, download and jump-to-conversation; strings in all five locales. The
sidebar's top-menu paths are now one shared list.

* fix(artifacts): keep the restore hash guard and count versions by URL

RecordRestoredArtifactMtime again only stamps rows whose content hash matches
the restored content (and ignores empty hashes), and never rewrites the hash,
matching MessageArtifacts.WithRestoredMtime. Without the guard a restore of
v1's content also moved v2's mtime and relabelled v2 with v1's hash.

The library's version_count now counts distinct URLs per (session, source
path). A later answer that references an earlier file stores another row with
the same URL, which COUNT(*) reported as an extra version.

* fix(artifacts): scope the library to web chats and clear the legacy column

The library used the session list's default scope, which counts sessions
without an owner as legacy tenant-level rows. IM sessions are created without
an owner, so every member saw the file names and titles of every IM chat
(downloads then 404 for non-admins). The library now applies the same "web"
bucket predicate as the home sidebar, extracted from QueryPaged so the two
cannot drift: no IM mapping (including cleared ones), no embed marker, no
API-key owner.

Migrations 000103 / SQLite 000023 now clear messages.artifacts after the
backfill. Message loads still SELECT *, so the stale copy was read on every
history load. Only rows that carried artifacts are rewritten; the column stays
for instances on the old binary during a rolling upgrade, and the down
migration still rebuilds it from the table.

* fix(migrations): keep the 000103 backfill alive on malformed artifact JSON

A single file_size that is not an integer ("12.5", "") or a created_at that
does not parse ("garbage", month 13) raised a cast error and failed the whole
migration, blocking startup. Casts now go through pg_temp helpers that return
NULL on bad input, falling back to 0 and the message's created_at; valid rows
are unchanged. SQLite casts never raise; its migration test now covers the
same malformed values.

* docs(artifacts): describe the web-only library scope and the cleared legacy column
2026-09-18 12:10:19 +08:00
Fei.Chen c8ebb9ed96
fix(knowledge): recover stranded deleting rows and align the KB count (#3367)
Second slice of #3338 (tracked as #3345).

Count alignment: CountKnowledgeByKnowledgeBaseID now excludes rows
mid-deletion, mirroring the document list's existing filter — a stranded
delete no longer shows as '4 documents, 3 listed' on the KB card. Failed
rows stay counted: they are visible, actionable cards.

Sweep C: housekeeping now recovers rows stuck at 'deleting' whose delete
task no longer exists. The delete dead-letter callback only flips rows
that are still 'deleting' when retries exhaust, so a worker death
mid-delete or a lost queue left the row hidden from the list forever.
Same recovery contract as the parse sweep: flip to failed with a clear
error so the row becomes visible and actionable; the delete executor is
idempotent and plans by row ID, so a late-arriving task still finishes
cleanly.

Liveness gate: a new TaskInspector probe (HasQueuedDeleteTasksForKnowledge)
matches knowledge:list_delete batch payloads — the per-parse matcher
cannot see them. A live queued/active delete defers the row (backlog,
not stranded); a probe error defers that row; a nil inspector defers the
whole sweep. Lite mode's noop returns false by design: inline deletes
never queue, so a stale deleting row there genuinely has no task.

Tests: sweep matrix (stranded recovered / backlogged kept / fresh
untouched / probe-error deferred / nil-inspector deferred), count
alignment, and the payload matcher table.

Refs #3345, #3338
2026-09-18 00:55:44 +08:00
lyingbug 76a19f14c4
feat(mcp): add built-in MCP server with per-workspace endpoints (#3371)
* feat(mcp): add built-in MCP server with per-workspace endpoints

WeKnora previously exposed itself to MCP clients only through the Python
package under mcp-server/, which binds one process to one API key and
therefore to one workspace, and mirrors the REST API tool-for-tool.

This adds a native MCP server surface to the Go backend:

- New mcp_endpoints table (PostgreSQL 000100, SQLite 000020). A workspace
  can publish any number of endpoints, each with its own bearer token
  (SHA-256 stored, rotatable, shown once), knowledge-base allowlist, tool
  allowlist, default agent and per-minute rate limit.
- Management API under /api/v1/mcp-endpoints (Viewer reads, Admin writes,
  API keys need manage_channels) plus a tool catalog endpoint.
- internal/mcpserver hosts one mcp-go Streamable HTTP server in stateless
  mode at /mcp/:endpoint_id. tools/list is filtered per endpoint and
  tools/call re-checks the allowlist, applies rate limiting and touches
  last_used_at. MCPEndpointAuth resolves the token and injects the
  workspace, an mcp_endpoint principal and an API-key style scope derived
  from the endpoint, so existing knowledge-base and capability checks
  apply unchanged.
- Redesigned tool set (12 tools in four groups): retrieval and reading
  reuse the agent's native tools; ask runs the default or a named agent
  synchronously and returns a cited answer with a session_id for
  follow-ups; wiki tools cover search, read and index; write tools are
  off by default. Knowledge bases accept ids or names.
- Frontend: new "MCP Server" integrations tab (tab id mcpserver, to
  avoid clashing with the existing MCP-client settings section) with
  endpoint cards, a create/edit drawer, a one-time token dialog and
  ready-to-paste snippets for Cursor/VS Code/Claude Desktop, Claude Code
  and mcp-remote. Locales updated in all five languages.
- Deployment capability integrations.mcpserver wired on both sides.
- Docs: website-docs MCP page now leads with the built-in server; the
  Python mcp-server README carries a deprecation notice.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(mcp): wire the MCP server into DI/router and tighten authorization

The previous commit lost eight staged files to the repo pre-commit hook,
which resolves ROOT to the main checkout and re-adds the main-checkout
copies of tracked files into the worktree index. This commit restores them
(container providers, RouterParams and route registration, context key,
principal, clone table entry, deployment capability key, SQLite parity
test) and addresses the review findings:

- Knowledge-base scope is authorized, not trusted: endpoint allowlists and
  every tool call resolve each knowledge base through access.ResolveKB for
  the calling principal (ownership or organization share). Foreign IDs are
  rejected on create/update and dropped at call time.
- Write tools mint explicit editor grants via access.KBAccess.WithGrant
  before calling the knowledge service, so add/update/delete go through the
  same RequireKBWrite path as the REST API. add_document also checks the
  grant itself.
- ask no longer accepts agent_id. It runs only the endpoint's configured
  agent; tenant-owned agents and user-facing builtins qualify, internal
  builtins (wiki fixer, skill installer) are refused both on save and at
  call time. Web search is forced off for MCP calls.
- Tool annotations match behaviour: ask is not read-only, add_document is
  a mutation.
- CORS allows MCP-Protocol-Version / Mcp-Session-Id / Last-Event-ID and
  exposes Mcp-Session-Id for browser-based MCP clients.
- Rate limiter runs its local cleanup goroutine; SQLite migration gains the
  deleted_at index for parity with PostgreSQL; non-ASCII endpoint names get
  unique mcpServers keys derived from the endpoint id.
- Swagger annotations on the management handler.
- Tests: KB isolation (service and server), write grants, endpoint-agent
  restriction, tool annotations, and a router test that mounts the public
  /mcp/:endpoint_id surface and checks the deployment capability.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(mcp): renumber migrations after rebase onto main

main took 000100 / 000020 for chunks_index_diet, so the mcp_endpoints
migrations move to 000101 (PostgreSQL) and 000021 (SQLite); the SQLite
parity test and docs follow.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(mcp): run document tools under the knowledge base owner tenant

Shared knowledge bases resolved through access.ResolveKB were only half
usable: documents of an organization-shared knowledge base are stored under
the owner tenant, so the document-level tools that queried by the calling
tenant found nothing, and add_document would have created rows under the
caller's tenant inside someone else's knowledge base.

- knowledgeInScope loads documents tenant-agnostically and then requires
  the document to belong to an allowed knowledge base and to that base's
  tenant.
- scopedKBContext replaces grantContext: it resolves the grant, switches
  the execution tenant to the knowledge base owner (KBAccess.Context) and
  swaps TenantInfo to the owner so storage/model settings resolve there.
  list_documents, read_document and every write tool run under it.
- Locale text and docs no longer claim clients can pass agent_id.
- Go SDK gains MCP endpoint methods (client/mcp_endpoint.go) with docs, and
  the Swagger artifacts are regenerated.
- Tests: shared knowledge base read/write under the owner tenant, spoofed
  document tenant rejected, viewer share cannot write.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(mcp): renumber migrations after rebase onto main

main now carries 000101 / 000021 (knowledge_profiles), so the
mcp_endpoints migrations move to 000102 (PostgreSQL) and 000022 (SQLite);
the SQLite parity test expects version 22 and the docs follow.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(mcp): guard last_used touch without a repository and wrap SDK line

The transport tests build the server without a repository; the background
last_used_at write then dereferenced a nil interface and, on the slower CI
runner, took the test binary down. touchLastUsed now returns early when no
repository is wired, and the tests inject a recording repository and assert
the touch instead. Also wraps a 123-column SDK signature for the client
module's lll check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-17 22:40:05 +08:00
lyingbug 5574ebc011
feat(knowledge): derive knowledge base descriptions from document profiles (#3370)
* feat(knowledge): derive knowledge base descriptions from document profiles

Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.

Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.

Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.

Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.

Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.

Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.

* fix(knowledge): keep generated descriptions out of duplicates and refresh clones

DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.

CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.

* fix(knowledge): refresh the loaded clone target and carry document profiles

ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.

cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
2026-09-17 22:21:55 +08:00
lyingbug 182f67bd86
refactor(modelcontext): consolidate source_refs parsing, fixed data_analysis table, and identifier leak report (#3372)
* refactor(wiki): consolidate source_refs parsing and resolution

source_refs entries ("uuid" or "uuid|title") were parsed by ten
independent implementations that disagreed on whitespace handling, on
whether a model-supplied title could be trusted, and on how a leading
pipe was treated. The scope-enforced and legacy agent resolvers also
duplicated the lookup and the routing step looked every document up a
second time.

- types.ParseWikiSourceRef / FormatWikiSourceRef / WikiSourceRefMatchesKnowledge
  are now the only readers and writer of the format.
- resolveWikiSourceDocuments replaces resolveAuthorizedSourceRefs,
  resolveSourceRefs and wikiKnowledgeBasesForSourceRefs: one lookup per
  document, titles always rebuilt from server data, dedupe by canonical ID,
  unknown documents fail closed in every mode.
- removeSourceRef now matches padded or titled refs of a deleted document.

* refactor(data_analysis): expose the selected document as a fixed table name

The tool required the document ID inside the SQL text and blindly
string-replaced it with the physical DuckDB table name, which forced the
model-context codec to rewrite handles inside quoted SQL and let the
UUID-derived k_<uuid> table name reach the model through stored schema
summaries.

The document is now selected only by knowledge_id. Every query runs on a
dedicated connection where "dataset" is a temporary view over the
physical table (DuckDB temporary objects are connection-local, so
concurrent sessions stay isolated). The physical name remains accepted
for internal callers; any other table fails validation with an error
naming the table the model should use. Schema summaries, the schema
tool, the analyst prompt and the tool schema all name the same table, so
data_analysis.sql no longer carries source handles and its text codec
policy is removed.

* feat(modelcontext): report raw identifiers that survive encoding

Text-level compaction exists because raw UUIDs reach model context through
producers the registry never sees: error strings, slugs, results of
unknown shape. Retiring it requires knowing where those producers are.

Registry.LeakedIdentifiers scans an encoded message list for UUID-shaped
values in every application-composed field (user messages are skipped),
classifies each as registered (codec missed a rewrite) or unregistered
(producer emitted a raw ID), and SummarizeLeaks renders one line per
field. The agent engine and the three RAG pipeline call sites log the
report as a warning after EncodeMessages.

* fix(chat_pipeline): align RAG data analysis with the fixed table contract

The RAG data-analysis plugin still asked the planning model to fill in
knowledge_id and to write SQL from a prompt that exposed the raw
knowledge UUID, then passed the model's JSON straight to Execute. Under
the new data_analysis contract a query that names anything but the
dataset table is rejected, and the plugin logged that rejection at error
level and moved on, so the analysis vanished without any signal. A
"no analysis needed" plan took the same error path.

The pipeline now owns document selection: the model returns only
needs_analysis and sql, the prompt names the dataset table and never
shows the knowledge ID or physical table, and the plugin builds the tool
input itself. A rejected query is fed back to the model once for repair;
a query that still fails is reported through pipelineWarn with the SQL
and the tool error, and "not needed" is logged as such instead of as a
failure.

Covered by unit tests with a scripted model and executor: not-needed,
pipeline-owned knowledge_id, one-shot repair, persistent failure
surfaced, unparseable plan rejected.
2026-09-17 22:08:16 +08:00
lyingbug 68d99d7d21
perf(chunk): speed up chunk ingestion and batch updates (#3369)
* perf(chunk): speed up chunk ingestion and batch updates

Chunk writes were dominated by avoidable work in the repository layer:

- UpdateChunks built one CASE WHEN id = ? chain per column, so a batch of
  N rows evaluated 5N branches per row (O(N²)). On PostgreSQL it now joins
  against a VALUES list keyed by the primary key (O(N), batched by 1000);
  other dialects run one small UPDATE per row inside a single transaction.
- CreateChunks copied Content into SourceContent for every new row, doubling
  insert volume. SourceContent is only needed once a chunk is edited and the
  edit path already backfills it lazily, so it is left empty on create.
  Batch size raised from 100 to 500 and the insert (including the SQLite
  seq_id pre-assignment) now runs inside one transaction.
- New UpdateChunkFieldsByIDs applies uniform field values with a single
  UPDATE ... WHERE id IN per batch. FAQ import uses it to flip status after
  indexing instead of shipping every row's content back, and resolves tag
  names per batch instead of per entry.
- Drop three chunks indexes that only slow down writes: chunk_type (low
  cardinality), content_hash (never used as a filter) and
  (tenant_id, knowledge_id) (covered by idx_chunks_knowledge_enabled).
  Migrations 000100 (PostgreSQL) and 000020 (SQLite) with down scripts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* perf(chunk): rewrite chunk batch writes and FAQ import status updates

Go changes belonging to the chunk write-path optimization. The previous
commit on this branch only carried the migrations because the repo's
pre-commit hook re-staged files from the main checkout when run inside a
git worktree, silently dropping the staged Go changes.

- UpdateChunks: O(N²) CASE chains replaced by a VALUES join on PostgreSQL
  (batched by 1000) and per-row updates in one transaction elsewhere.
- CreateChunks: stop copying Content into SourceContent, batch 500 rows,
  run the insert and the SQLite seq_id pre-assignment in one transaction.
- New UpdateChunkFieldsByIDs for uniform, tenant-scoped field updates.
- FAQ import flips status with UpdateChunkFieldsByIDs and resolves tag
  names once per batch instead of once per entry.
- Drop the gorm index tag on ContentHash to match migration 000100;
  SQLite schema test expects migration version 20.
- SQLite tests for the new and rewritten repository methods.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-17 21:38:15 +08:00
wizardchen 105c177b80 feat(knowledge): make document summary generation optional 2026-09-17 19:39:52 +08:00
lyingbug 5780cb0a71
fix(sandbox): separate file access from artifact publishing (#3356)
* fix(sandbox): separate file access from artifact publishing

* test(sandbox): align service coverage and pre-push checks with CI
2026-09-17 16:18:01 +08:00
nullptr-error 42e6163981
feat(session): fork conversations from a historical user message (#3304)
* feat(session): add fork schema and history queries

为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。

* feat(sandbox): add git, fork snapshots, and bootstrap hook

沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。

* feat(session): checkpoint workspace at the end of each turn

每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。

* feat(session): fork conversations and reap orphan snapshots

支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。

* feat(chat): add session fork entry, prefill, and badges

在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。

* style: add missing newline so session_fork_delete_test.go is gofmt-clean

* fix(session): prune later git history when rolling a fork workspace back

reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。

* style(session): clear golangci-lint findings on the session-fork diff

Pre-push lints the whole PR against main; wrap long lines, check c.Error,
and fix gofumpt/revive/staticcheck so the merge can be pushed.

* fix(chat): show the fork button on hover without shifting the thread

Overlay the action so revealing it no longer expands the user bubble and janks the messages below.

* fix(chat): drop the fork landing banners

Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread.

* fix(agent): dedupe artifacts by content hash, not size

Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only.

* feat(session): allow forking from an assistant answer

User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists.

* fix(session): resolve parent attachments from a forked chat

Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss.

* fix(agent): rewrite sandbox refs after a content-hash skip

Fork restores change mtime on unchanged files, so Collect returns empty
and named citations were left as sandbox: paths. Resolve those names
against session artifacts, notify only real uploads, and stamp restored
mtime only on the matching content hash.

* fix(session): drop a dead fork snapshot and retry checkpoint rewrite

Create from a missing template now abandons bootstrap so the next
Resolve uses a normal sandbox. A failed checkpoint rewrite no longer
marks the bootstrap consumed, which previously made nested forks
SANDBOX_REPLACED.

* fix(session): load forked parent attachments into the prompt

ResolveForPrompt used session-scoped lookup, so copied parent attachment
IDs passed Get but never reached the model. Use the same resolveInSession
path as preview.

* fix(chat): keep the user-message fork button visible

The control was hover-only, which hid it on touch. Always show it, and
drop the unused degrade tooltip.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 14:20:18 +08:00
coderdailyone 0ad08d41be fix(repository): make message keyword search work on SQLite and MySQL
SearchMessagesByKeyword filtered with `messages.content ILIKE ?`, which is
PostgreSQL-only syntax. On SQLite (the Lite build) and MySQL the query fails
with `near "ILIKE": syntax error`, so POST /api/v1/messages/search returned
HTTP 500 in both keyword and the default hybrid modes. The session-list
search (QueryPaged) was already made dialect-aware for the same reason; this
path was missed.

Pick the LIKE expression per dialect — ILIKE on Postgres, LOWER() LIKE
LOWER() elsewhere — and add an explicit `ESCAPE ?` bound to likeEscapeChar so
escapeLikeKeyword's backslash escapes are honored on SQLite, which has no
default LIKE escape character (a no-op on Postgres/MySQL, where `\` is
already the default).

Regression tests in message_search_test.go run against in-memory SQLite:
TestSearchMessagesByKeywordMatchesOnSQLite and
TestSearchMessagesByKeywordTreatsWildcardsLiterally both fail on the old code
with the ILIKE syntax error and pass with the fix.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRrBenkCCr4pacMNwR4NuG
2026-09-17 12:27:52 +08:00
serply e80f5dfbc7 feat(web-search): add Serply provider
Serply returns Google results through a keyed JSON API, which fills the
gap between Google Custom Search (needs an engine ID and a restricted
scope) and the scrapers. The adapter follows brave.go: hardcoded
endpoint, redirects refused so the key never leaves the official host,
4MB response cap, and per-call country and freshness filters. country
maps to Google's gl parameter and pd/pw/pm/py map to tbs qdr:d/w/m/y.
Serply has no date-range filter, so that form returns an error instead
of being silently dropped. num is clamped to 10 because the API serves
one results page per call.

Wiring matches the Bocha provider: type constant and metadata, registry
factory, service validation, frontend union, swagger enum entries (edited
by hand; a full swag regen drags in unrelated drift), and the web search
docs table.
2026-09-17 12:25:09 +08:00
wizardchen 079e96ae64 fix(knowledge): make in-place file replacement race-safe
Queued ProcessDocument tasks still held the old FilePath, and a crash
between the source UPDATE and reparse could leave completed knowledge
pointing at a new file. Dequeue first, flip pending in the same write,
and skip stale worker Saves when the stored path has moved.
2026-09-16 14:54:47 +08:00
ZHE MA 419ff2507c feat(knowledge): add in-place file replacement primitive
Add KnowledgeService.ReplaceKnowledgeFile, which replaces the stored
source file of an existing file knowledge while preserving its knowledge
ID, then re-parses it through ReparseKnowledge. It is groundwork for the
local folder data source (#2873); no existing API, handler or connector
behavior changes.

Object storage is not transactional, so the steps are ordered to keep the
row pointing at an existing file: save the new file, update the source
columns in one UPDATE, reparse, and only then delete the old file. A
failed column update discards the new file; a failed reparse restores the
previous source columns (marked failed, since cleanup may already have
removed the old index) and discards the new file. The row is re-read
before a replacement file is deleted, so a write that committed despite
reporting an error never leaves the knowledge referencing a deleted file.

Identical content under the same name and folder is not re-parsed: changed
metadata is persisted and a DuplicateKnowledgeError is returned, matching
CreateKnowledgeFromFile's duplicate contract. Metadata is merged so
entries such as process overrides survive, and stored overrides are
re-validated against the new file type.
2026-09-16 14:54:47 +08:00
jacentsao 01f0700b35
feat(datasource): add Confluence sync (#3287)
* feat(datasource): add Confluence sync

* fix(confluence): resume full sync and list Server pages without CQL

CQL search silently truncates large spaces and can delete pages that
dropped out of the index. Full-sync checkpoints now keep a deletion
baseline so Asynq retries continue instead of restarting.

* fix(confluence): satisfy package lint for pre-push against main

* fix(confluence): parse Server page lists and keep version expand

The default Server/DC edition unmarshaled /content/page as a nested
envelope, so sync imported zero pages. Pagination next links also drop
expand=version,space, which made later pages versionless and hid edits.

* fix(confluence): classify errors and fail closed on empty listings

Raw Confluence error bodies were copied into sync-log UI, and pagination
next links could leave the configured context path via `..`. Empty
listings against a non-empty baseline now abort instead of deleting.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-16 14:38:16 +08:00
chenrui 9d67a0eb4d
fix: clean up wiki data when deleting knowledge base (#3109)
* fix: clean up wiki data when deleting knowledge base

ProcessKBDelete async task cleaned up vector indices, chunks, files,
graph data and knowledge rows, but left wiki data untouched because
knowledgeBaseService had no wiki repository dependency. Add four
KB-scoped delete methods to WikiPageRepository and invoke them at the
end of ProcessKBDelete: soft-delete wiki_pages/wiki_folders/
wiki_page_issues (they carry DeletedAt), hard-delete wiki_page_revisions
(no DeletedAt column — immutable snapshots).

* fix: scope wiki KB cleanup by tenant and run it on every delete path

Wiki rows do not depend on the vector engine, so ProcessKBDelete now
cleans them in a defer (including SkipRetry). Deletes are keyed by
tenant_id plus knowledge_base_id so a tampered queue payload cannot
wipe another tenant's wiki.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-14 21:02:31 +08:00
VictorMaznyuk b94612a7ec
fix(agent): PUT no longer erases the avatar, and driver errors stay server-side (#3230)
* fix(agent): PUT no longer erases the avatar, and driver errors stay server-side

Two defects in the custom-agent API, both of which surface as a successful or
misleading response.

1. A config-only PUT wiped the avatar.

   UpdateAgentRequest.Avatar was a plain string, so "field omitted" and
   "explicit clear" were indistinguishable, and UpdateAgent assigned it
   unconditionally. Any caller that updated only the config — which is what
   an edit form or an SDK client does when it sends the config it changed —
   silently zeroed the avatar and still received 200.

   Avatar now travels as *string end to end: nil keeps the stored value, a
   pointer to "" is an explicit clear. The service takes it as a separate
   argument because *types.CustomAgent cannot express the difference.

2. Raw repository/driver errors reached the client.

   The fallback branches forwarded err.Error() verbatim, so a postgres
   message — for example "ERROR: value too long for type character
   varying(64) (SQLSTATE 22001)" — was returned in the API response. That
   leaks schema details and tells the caller nothing actionable. Each
   fallback now returns a fixed message naming the operation; the full error
   is still logged immediately above, so nothing is lost for diagnosis.

   The same oversized-avatar case is now rejected earlier, by ValidateAvatar,
   as a 400 that names the varchar(64) limit instead of a 500 produced by the
   database.

Typed sentinels and *errors.AppError keep their existing behaviour — only the
untyped fallbacks changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(agent): complete avatar preservation and regression coverage

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-14 20:50:15 +08:00
nullptr-error 6ccc5cedf1
feat(sandbox): add a graphical desktop to session sandboxes (#3245) 2026-09-14 20:37:37 +08:00
wizardchen 1f82fbc9b5 fix(datasource): preserve distinct GitLab files during deduplication 2026-09-14 19:32:41 +08:00
hujiaxin fc2c12e8ec fix(models): protect auto-tag model references 2026-09-14 19:29:19 +08:00
wizardchen 17f893865d fix(memory): rank memories by the query, not by importance
Long-term memory chose its candidates before it looked at the question.
Recall listed 400 items by `importance DESC, valid_from DESC`, loaded the
vectors belonging to that list, and scored those; extraction did the same
with a 200-item pool. Importance is uncorrelated with relevance, so the
vector stage could only re-order what importance had already picked, and a
memory below the cut was unreachable no matter how well it matched. The
capacity cap goes up to 2000, so everything past the 400th most important
memory was permanently unrecallable, and `memory_item_embeddings.vector`
being a BYTEA blob meant pgvector was never involved.

The store now answers "which of this person's memories is this question
about":

- Migration 000095 (and the SQLite mirror) add `embedding halfvec` plus a
  `(tenant_id, subject_id, model_id, dims)` index. The column is
  conditional on the vector extension, so a deployment whose retrieval
  driver excludes postgres keeps working. No HNSW: an approximate index is
  filtered after the fact, and one subject holds at most a couple of
  thousand rows, so exact ordering inside the scope is both correct and a
  few milliseconds.
- `SearchItemsByVector` ranks in SQL over every vector the subject has,
  with the status, expiry and kind filters joined in so reaching past the
  candidate list cannot reach past the filters. SQLite and pgvector-less
  PostgreSQL fall back to scoring in process, still over the whole scope.
- Recall fuses the lexical ranking with those hits and grows the candidate
  pool with matches the pool did not hold, which is what lets the search
  escape it. The lexical pool is now the capacity cap, so it can no longer
  be a second, invisible limit under the one that decides what stays active.
- Extraction picks the memories it shows the model the same way. This
  matters more than recall: the memory a new statement contradicts is not
  reliably an important one, so an importance-ordered window hid exactly
  the collision that stops a duplicate being written.
- Vectors are written to the searchable column in the upsert's transaction;
  rows predating the column are moved by `SyncVectorColumn` during
  maintenance, which costs no model calls. The embedding backfill goes from
  50 to 200 per run — at 50, a subject at the capacity cap took over a
  month to become fully searchable.
- Traces and logs report `outside_pool`, the number of semantic matches the
  lexical pool did not contain. It was structurally zero before, so it is
  the number to watch when asking whether semantic recall does anything.

Tests cover whole-scope ranking, the lifecycle and kind filters, pool
growth, the pool-size invariant, and extraction seeing a low-importance
memory it would otherwise duplicate. A pgvector integration test
(WEKNORA_MEMORY_TEST_POSTGRES_DSN) exercises the migration, the halfvec
write, the SQL ranking and the column sync.
2026-09-13 11:43:17 +08:00
lyingbug fd7fa6a7d4
fix(memory): preserve extraction progress and atomic memory updates (#3224)
* fix(memory): preserve extraction progress and atomic memory updates

* fix(memory): bound extraction failures and persist indexed session progress
2026-09-13 00:04:16 +08:00
wizardchen a799f23b26 feat(sandbox): stream live shell and install command output
Publish a bounded stdout/stderr tail for pending shell_exec and skill installs without putting partial logs into the model context.
2026-09-12 10:16:39 +08:00
lyingbug 3910cfb117
fix(browser): complete task cancellation, screenshots, and result display (#3206) 2026-09-12 09:32:26 +08:00
lyingbug 7a98a8e3de
feat(browser): polish connection settings, open tasks in a new window, and streamline prompts (#3193)
* feat(browser): enhance BrowserIcon component and update localization strings

- Updated BrowserIcon component to accept dynamic width and height props.
- Revised localization strings in English, Japanese, Korean, Russian, and Chinese for improved clarity and user guidance regarding browser connection and usage instructions.
- Enhanced BrowserConnectionSettings view with better structure and user feedback elements.

* Default BrowserSkill tasks to a separate window

* refactor(prompts): streamline agent system prompts and context templates

- Updated agent system prompts to enhance clarity and specificity in user interactions.
- Revised context templates to better separate metadata and user requests, improving overall structure.
- Adjusted fallback and intent prompts to ensure consistent language handling and response requirements.
- Introduced new localization strings for browser search instructions across multiple languages.
- Added tests for agent prompt serialization and hydration to ensure template integrity and functionality.

* chore: wrap prompt strings to satisfy golangci-lint

Keep the settings, separate-window, and prompt commits pushable together
without rewriting their history.

* fix(frontend): match agent editor save test to serialized payload

The prompt-inheritance save path still closes after update; the test
should assert that payload, not the unsanitized form object.

* feat(browser): implement browser connection management and related tests

- Introduced a new `browserConnection` store to manage the state of browser connections, including online status and device information.
- Updated the `Input-field` component to handle browser connection states, enabling/disabling features based on the connection status.
- Added tests for the `browserConnection` store to ensure correct behavior under various scenarios.
- Enhanced the `Input-field` and `BrowserConnectionSettings` components to provide user feedback based on the browser connection state.
- Adjusted related components and views to integrate the new browser connection logic, improving overall user experience.

* fix(frontend): update model-save assertion for serialized agent payload

The agent editor still persists the selected model ID via a spread of
formData; the source test should match that payload, not the raw form.
2026-09-11 21:28:58 +08:00
lyingbug 7e53c31671
feat(browser): connect local Chrome with task controls and display modes (#3180)
* feat(browser): integrate BrowserSkill local task tabs and account pairing

* feat(browser): refine task routing, lifecycle, and display modes

* fix(browser): reclaim temporary task tabs and groups

* test(database): cover browser authorization migration schema
2026-09-11 17:59:27 +08:00
langcaiye 5db13a131e
feat(faq): filter entries by enabled status (#3157)
* docs: design FAQ enabled-status filter

* feat(faq): filter entries by enabled status

* test(faq): cover disabled list filtering

* docs(faq): publish enabled filter contract
2026-09-10 14:42:59 +08:00
wizardchen 5cf093706e fix(chat): preserve steering continuity and composer focus 2026-09-09 20:15:23 +08:00
nullptr-error fbaff900a0
feat(chat): append messages to a running agent turn (#3123)
* feat(stream): persist live-run markers and per-run steer queues

Mid-run sends need a cross-replica pointer to the generating
assistant and a control-plane list that never hits the SSE stream.

* feat(agent): inject queued user messages at ReAct round boundaries

The engine drains the steer sink after each round, persists accepted
rows, and delays Done until after a loop-end inject so the turn can continue.

* feat(session): expose /steer and hand leftover messages to a follow-up turn

Owner-scoped steer routes queue or inject into the live run, single-flight
AgentQA via the live-run marker, and history replay keeps multiple user rows
inside one turn.

* feat(frontend): queue mid-run messages while an agent turn streams

Stop and Send stay visible together, the overlay follows server delivery,
and history splits the transcript around injected user bubbles.

* fix(frontend): type pending and client_id on SteerQueueItem

vue-tsc failed because the overlay sets those fields while POST /steer is in flight.

* fix(session): harden steer persist, handoff, and new_run

Keep SetLiveRun ahead of SSE and roll back orphan turns, make inject
persist idempotent, append-then-delete on rebind, cap overrun rounds,
and stop aborting a live stream when /steer returns new_run.

* fix(session): satisfy pre-push lint on steer paths

Wrap long lines, check discarded Error/Emit results, and keep
context as the first argument so the fork push hook can pass.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-09 16:43:33 +08:00
wizardchen 73462de222 feat(mcp)!: streamline setup and use short routing handles
BREAKING CHANGE: Updates that explicitly include usage_instructions must supply nonblank text of at most 16000 characters. Omitting the field preserves existing instructions; legacy descriptions remain readable.
2026-09-09 16:11:52 +08:00
lyingbug 85b76d1a30
feat(mcp)!: persist tool catalogs and add guided configuration (#3118)
* feat(mcp)!: persist tool catalogs and add guided configuration

* fix(mcp): write caller-owned catalogs and require a real tool sync

OAuth viewers can now persist their own directory, and the setup wizard no longer treats an unsynced service as complete. Missing catalogs auto-fetch on the tools step; already-used MCP functions are restored across engine turns.

* fix(mcp): skip redundant list_servers and tighten catalog UI

When the advertised server directory already fits in discover_mcp_tools, models should list_tools or describe directly. Chat and settings now show server names, collapsed docs, and initialize debug metadata for empty optional fields.

* fix(mcp): surface catalog summaries and polish tool detail UI

List responses now include a saved catalog count without live MCP calls, so settings cards and @ mentions can show sync state. Tool details open in a popup instead of expanding the row.

* fix(mcp): drop incomplete Anthropic calls and gate shared catalog refresh

Unclosed streamed tool_use was emitted as {} and executed. list_tools refresh=true now re-lists upstream instead of returning the saved snapshot, and only Admin (or MCP-capable API keys) can overwrite tenant-wide static-auth catalogs.

* fix(mcp): wrap client refresh path to satisfy lll

The metadata refresh URL made doRequest exceed the 120-character limit in the client module CI lint.
2026-09-09 13:34:56 +08:00
nullptr-error 277d6ee851
feat(sandbox): add an interactive terminal and files tab beside chat (#3110)
* feat(sandbox): stream interactive PTYs from Cube and E2B

Give session sandboxes a provider-neutral terminal capability so a later
WebSocket layer can attach without knowing Cube vs E2B. Long-lived
streams use a timeout-free client; idle terminals keep the sandbox alive
by refreshing TTL, and a workspace can set how long to wait before
disconnect.

* feat(session): bridge browser terminals to the session sandbox

Mint a short-lived ticket over a normal authenticated POST so the browser
WebSocket never puts the access JWT in the query string. After upgrade,
recheck the bound token, membership, and session ownership so logout or
revocation tears the PTY down. Provisioning stays opt-in: a GET
handshake only creates a sandbox when the client sends provision=1.

* feat(chat): add a sandbox terminal panel beside the conversation

Open a side panel with an xterm attached to the session sandbox. Starting
the terminal is an explicit click so merely opening the panel does not
create or resume a microVM; reconnects stay lookup-only until the user
confirms create-and-start. Persist the remote shell PID in
sessionStorage so a refresh reattaches instead of spawning an orphan.

* feat(sandbox): let workspaces set terminal idle disconnect

Expose the idle-disconnect duration in the sandbox config editor so an
operator can close unused terminals and let the provider TTL pause the
sandbox, instead of leaving PTYs open for the built-in 15 minutes.

* feat(chat): show generated files in the sandbox side panel

Move the per-message artifact list into the first tab of the right-hand panel so generated files stay in view with the terminal. Keep the old drawer only for embedded chat.

* fix(sandbox-terminal): keep tickets out of logs and live PTYs across refresh

Gin was copying the handshake JWT into response_body logs, silent access-token
rotation dropped open terminals as AUTH_REVOKED, and first-use provisioning
missed shared agents. Local echo now skips password prompts.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-08 17:43:15 +08:00
lyingbug 3d81b977f1
refactor(resource-access): unify scoped authorization and transfer recovery (#3097)
* refactor(resource-access): unify scoped authorization and transfer recovery

* fix(resource-access): close reparse and transfer review gaps

* fix: enforce explicit file bindings and recover move cleanup

* fix: align mutation admission and cleanup scope checks
2026-09-08 15:36:33 +08:00
wizardchen e18cf138a3 fix: remove GPL converter and bundle third-party license sources 2026-09-08 15:19:10 +08:00
wizardchen 694f0413dd refactor(agent): discover and invoke MCP tools on demand 2026-09-08 12:01:36 +08:00
lyingbug 647848f395
fix(agent): improve web search and persistent page reading (#3086)
* fix(agent): improve web search and persistent page reading

* fix(agent): make web fetch snapshots waitable and retryable

Keep in-flight page fetches independent of a caller's short deadline so
content=true and continuations cannot cancel a shared download or miss a
snapshot that is still loading. Mark snapshot_expired as retryable, fetch
only the first three search hits in parallel, and stop forcing Brave's
country to US.
2026-09-07 23:09:27 +08:00
Fei.Chen 15727591a7
feat(web-search): add Bocha AI Search provider (#3036)
* feat(web-search): add Bocha AI Search provider

Add a Bocha (bochaai.com) web search provider following the Metaso
adapter pattern: hardcoded official endpoint (SSRF-safe), Bearer auth,
4MB response cap, error-detail extraction, and RFC3339 date parsing
(datePublished preferred, dateLastCrawled fallback).

Tenant-facing config fields:
- freshness: noLimit/oneDay/oneWeek/oneMonth/oneYear (default noLimit)
- summary: request long text summaries, preferred as snippets (default on)

Wiring: registry factory in container, provider type enum + /types
metadata (dynamic form fields), frontend provider union type, and
swagger docs enum entries.

* fix(web-search): match Bocha's real wire format for codes and error messages

Live endpoint probing showed errors return {"message": .., "code": "401"}
(string code, message field) instead of the documented {msg, code:int}.
Parse the code tolerantly (string or number) and extract the detail from
message/msg so auth and rate-limit failures surface their reason.

* fix(web-search): allow Bocha in provider CRUD validation

In-service integration testing caught a fourth wiring point the adapter,
registry, /types metadata and frontend covered but CRUD did not: the
service-layer whitelist (isValidProviderType) and parameter validation
switches rejected provider type 'bocha', so providers could be tested
via POST /web-search-providers/test but never created or updated.

Add Bocha to both switches (delegating to ValidateBochaParameters like
Metaso/Zhipu) with service-level test coverage.

* fix(web-search): correct Bocha legacy timestamp timezone

---------

Co-authored-by: vicfei <vicfei@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 20:06:26 +08:00
小妍儿 ✨ 061df8efdc
feat(datasource): persist source created time; Feishu uses content edit time for source_updated_at (#3083)
* feat(datasource): persist source document creation time and use content edit time for Feishu

Follow-up to 160c806b (Refs #3016), which started persisting the source
last-modified time as metadata.source_updated_at. Two gaps remained:

1. Creation time was never carried. FetchedItem gains CreatedAt and
   ingestItem persists it as metadata.source_created_at (RFC3339, UTC)
   with the same absent-means-unknown rule as source_updated_at.

2. The Feishu wiki connector filled FetchedItem.UpdatedAt from
   NodeEditTime, which moves when a node is renamed or dragged in the
   tree without any content change. The connector's own cursor already
   prefers ObjEditTime for exactly this reason; use the same rule for the
   persisted value (contentEditTime / contentCreateTime, node_* as
   fallback), so source_updated_at reflects document edits.

Feishu Drive populates CreatedAt from created_time. GitLab stops
fabricating UpdatedAt = time.Now(): a fetch timestamp is not a document
timestamp, and an absent value is the documented "unknown" (the commit
time is not fetched today; leaving it unset is honest until it is).

Tests: ingestItem covers both keys present/absent; the wiki fetch test
asserts obj_* times win and node_* are the fallback.

* style(gitlab): break up the FetchedItem literal (lll)

---------

Co-authored-by: xiaoyaner0201 <>
2026-09-07 19:39:01 +08:00
GUO Zhaogong 6f0ed94a47
feat(i18n): add Japanese (ja-JP) locale support (#3080)
* feat(i18n): add Japanese (ja-JP) locale support

Add a complete Japanese UI locale for the admin frontend and the web
embed widget, and register it everywhere the supported-locale list is
enumerated.

- frontend/src/i18n/locales/ja-JP.ts: full bundle (5,600+ keys) aligned
  with en-US/zh-CN key structure; verified by the locale key audit
- frontend/src/i18n/embed.ts: ja-JP block for the embed widget, plus
  `ja*` normalisation in normalizeEmbedLocale
- Register ja-JP in i18n index, SUPPORTED_LOCALES, localeKeyAudit
  bundles/order, TDesign locale map (ja_JP), login language picker,
  general settings dropdown and embed default-locale options
- Add `language.jaJP` key to every locale so the settings dropdown can
  use $t like its siblings
- Accept ja-JP as an embed channel default_locale on the backend and as
  DEFAULT_LOCALE in the frontend docker entrypoint
- Extend locale-enumerating tests and update accepted-value lists in
  .env.example, public/config.js, widget script and website-docs

* feat(i18n): add ja-JP names and descriptions for prompt templates

Add a ja-JP block to the i18n map of all 32 prompt templates
(system_prompt, context_template, rewrite, fallback, intent_prompts,
agent_system_prompt) so the template picker shows Japanese labels when
the UI locale is ja-JP instead of falling back to the Chinese names.
Prompt content is unchanged.

* feat(i18n): add ja-JP strings for the CLI integration page

main gained the CLI setup page (#3081) with 18 new integrations.cli.* /
integrations.tabs.cli keys; add their Japanese translations so the
locale key audit stays green after rebasing onto main.

* style(i18n): unify ja-JP spacing and long-vowel conventions

- Remove spaces between Latin tokens and adjacent Japanese text
  (e.g. "MCP サービス" -> "MCPサービス"), matching the majority style
  already used in the bundle
- Drop the trailing long-vowel mark on common technical katakana terms
  (サーバ, ユーザ, フォルダ, プロバイダ, パラメータ, ヘッダ, クラスタ, ...)
  so the whole locale follows one convention

* feat(i18n): add ja-JP labels for agent type presets

config/agent_type_presets.yaml carried default / zh-CN / zh-TW labels
only, so the agent editor showed English preset names in the Japanese
UI. Add a ja-JP label and description to all five presets.
2026-09-07 19:20:41 +08:00
coderdailyone 9e8673816e fix(wiki): keep folder names that look like page-type labels when placing pages
A folder named 概念/实体/Concepts was stripped from category_path as model noise, so a page moved into it kept depth 0 and a root-level wiki_path and rendered at the root of the directory tree. Folder paths are authoritative and now map to category_path verbatim; the model-label cleaner still applies to pages without a folder.

Fixes #3053

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DFSMrLZZ3oq1dKmJFAofz6
2026-09-07 17:23:19 +08:00
Max Freedom Pollard 801005f736 fix(knowledge): keep long non-ASCII folder names valid UTF-8
NormalizeKnowledgeFolderPath capped a directory name with
segment[:MaxKnowledgeFolderSegmentLength] (internal/types/knowledge_folder.go:51).
That is a raw byte index, so a name longer than 128 bytes was cut wherever
byte 128 landed. For a Chinese folder name that is inside the 43rd character,
and the leading bytes of the half-cut character stay in the string.

The result is an invalid UTF-8 folder_path. json.Marshal replaces the broken
bytes with U+FFFD, so the path the client reads back is not the path stored in
the row, and RenameKnowledgeFolder (internal/application/service/knowledge.go:611)
normalizes that returned value and matches nothing.

The cut now backs off to the nearest rune start. The cap stays a byte budget,
so a 60-character Chinese name keeps its first 42 characters (126 bytes)
instead of 128 bytes ending in half a character. ASCII names are unchanged.
2026-09-07 11:50:32 +08:00
ninesun 1e36e071ec
feat(mcp): add per-tool enable toggle (#2989)
* feat(mcp): add per-tool enable toggle

* style: format migration test

* fix(mcp): keep enable and approval updates from overwriting each other

Partial policy writes now touch only the supplied columns, and a missing
tenant fails closed so a disabled tool cannot slip through.

---------

Co-authored-by: myysophia <myysophia@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-05 17:19:20 +08:00
lyingbug d5cb842580
feat(agent): unify sandbox tools and drop host preloaded skills (#3033)
* feat(agent): unify sandbox tools and drop host preloaded skills

Read workspace and skill files through read_file, and run installed or staged skills through shell_exec(skill_name). Remove the unused skills/preloaded tree and keep generated-file previews stable across re-renders.

* fix(frontend): do not assume classList in markdown sanitize hooks

Node unit tests stub anchors without a DOM classList. Detect download cards from class/download attributes instead.
2026-09-05 16:45:59 +08:00