* fix(security): close the low-severity shared-space findings
Verified and fixed from the shared-space permission review:
- Embed channels kept knowledge_ids, tag_ids, mentioned_items, skill_names
and summary_model_id from anonymous visitors. Mentions are only scoped
for shared agents, so a visitor could query any KB (and pick any model)
of the channel workspace by ID. The payload now drops them; the widget
never sends them.
- PUT /knowledgebase/:kb_id/wiki/issues/:issue_id/status updated any issue
by ID. The update is now scoped to the path's KB (404 otherwise).
- A direct org invite let the inviter attach any user of the target
workspace (or any user by ID) as its representative, whose username and
email the roster then showed to all members. Direct adds attach no
representative, and the roster returns emails only for the caller's own
workspace.
- COS and KS3 dropped the bucket (and COS the region) segment when turning
a path into an object key, while the tenant check scans every segment:
cos://42/<region>/prefix/7/... passed as tenant 42 and read tenant 7's
object from a shared bucket. Both must now match the service's config.
- Shared agent listings returned the full agent to receivers. Prompts and
the creator's user ID are withheld; capabilities and scope stay visible.
- Temporary document uploads let the caller pick the parser engine (and
its credentials) of a shared agent's workspace; only the agent's rules
choose one there now.
- Suggested questions used request / message-snapshot knowledge_ids
without a permission check. Each document is now resolved to its KB and
checked like KB IDs, including a shared agent's scope.
- IM channels accepted any knowledge_base_id (and any agent ID on create).
Both must belong to the channel's workspace, the KB also to a restricted
API key's allow-list.
- KB-restricted API keys could list org-shared KBs outside their allow-list
in the organization views; those listings are now filtered.
- With enable_rbac=false, RequireKBAccess let cross-workspace denials
through. That switch is for roles inside a workspace; the guard now
always denies another workspace's KB, since handlers behind it load by
ID without a tenant filter.
- The org member limit was counted before and enforced apart from the
insert; it is now checked in the insert's transaction with the
organization row locked (SQLite serializes writers instead).
- The UpdateOrganization route comment and API doc said owner-only; the
service and UI allow org admins. Comment and doc corrected.
Documented, not changed: issued /r/ and presigned links stay valid until
they expire after a share is revoked.
* fix(security): address review of the low-severity shared-space fixes
- Embed payloads also drop question_origin. It selects a KB like an
@mention for agents that retrieve only when mentioned, and the widget
never sends it; the selection stays inside the agent's scope, but an
anonymous visitor should not pick among its KBs. suggestion_attribution is
kept (the widget sends it and it is validated).
- The receiver view of a shared agent also withholds the follow-up
suggestion instruction, a prompt like the others. Starter questions are
shown to receivers anyway, and the sharer's name is displayed by design.
- Follow-up suggestion generation moves into the agent's workspace with
types.WithExecutionTenant instead of a bare tenant rewrite, so the
caller is pinned before the document permission check runs. The auth
middleware and the shared run already pinned it; this removes the
dependency on that.
- KS3 CopyFile copies only from the service's own bucket (as COS and S3
do); another bucket's path is a cross-backend copy.
Tests cover the dropped question_origin, the follow-up instruction and the
/shared-agents listing, a follow-up context with the tenant switched, a
repeated wiki issue status update, and the KS3 copy source.
* feat(agent): size agent history by the window and calibrate token estimates
Agent history was capped at history_turns (default 5) because compaction
could not survive a turn. With checkpoints persisted, load history by
tokens instead, and let compaction bound the session.
Budget. The loader drops the oldest turns that do not fit, and what it
drops is lost: neither replayed nor summarized. A budget equal to the
compaction threshold (the earlier attempt, since reverted) let the loader
trim first, so whenever one turn was larger than the system prompt plus
the new question the request never crossed the threshold; nothing was
summarized and the session became a sliding window with no checkpoint.
agent.HistoryTokenBudget is now the whole window: overflow reaches the
first round's compaction, which summarizes it into a new checkpoint.
A turn-by-turn regression test drives the real loader, budget and
compactor and fails as soon as a stored turn after the checkpoint is
missing from the loaded history; restoring the old budget fails it.
Summarizer input. History can now reach the window, so the part being
summarized can outgrow one request. An overflowing request degrades to a
raw archive that is never persisted, so every later turn would fail the
same way. Prepare keeps the newest messages that fit, the prompt states
how many were left out, and file paths still come from all of them.
Calibration. cl100k counts Chinese at about a token a character where
Qwen and DeepSeek spend about 0.6. The estimator now carries a scale
(clamped to 0.5-1.5, text only), learned from consecutive requests in a
turn: they share tools, system prompt and earlier messages, so the
provider's growth between them prices exactly the appended messages,
without mixing in how a provider renders tool schemas. The turn's scale
is stored on its usage (context_token_scale); the next turn's loader
prices history with it and hands it to the engine. Loading, the first
compaction check and compaction itself share one estimator, so they
agree; calibrating only the trigger would let the loader drop turns
again, which the regression test covers.
Also: history is priced as the engine sends it (redacted KB/Wiki results
cost one line); a checkpoint older than the rows read is matched by the
assistant rows' (created_at, id) order; a read ending on a turn's
question keeps that turn; a compaction discarded for freeing under 5%
still persists its checkpoint; a checkpoint write matching no row is an
error; a partial index backs the newest-checkpoint lookup. Agent mode no
longer reads history_turns: the field leaves AgentConfig, the editor
hides it in smart-reasoning mode, and quick-answer keeps its turn cap.
* fix(agent): address review of window-sized history loading
Summarization. The summarizer call had a fixed 60s total timeout. With
history loaded up to the window, a request could carry ~175k tokens and be
cut off while prefilling or writing normally; the degraded result left no
checkpoint, so the next turn reloaded the same history and failed the same
way. Summarization now streams and is cancelled only after no output for the
engine's stall timeout (120s by default, LLMCallTimeout overrides), with the
overall ceiling left to the transport, as for the engine's own rounds and as
in Codex and Pi. A stopped turn is not retried. When the summarizer still
fails, the raw archive is persisted as a checkpoint marked degraded so the
loop is broken, and the next compaction refines it as the previous summary.
The archive is held to the summary budget, newest messages first, so a
degraded compaction always frees room.
Loading. The loader priced turns as sent but returned and held them as
stored, so wiki pages stored in full and sent as one line could pile up in
memory. It now returns the as-sent history and releases each turn's stored
rows once replayed, keeping about a page of rows at a time.
Index. The partial checkpoint index is replaced by a composite
(session_id, created_at DESC, id DESC) index serving the backwards page,
the newest-checkpoint lookup and the recent-messages reads. It is built with
CREATE INDEX CONCURRENTLY so writes to messages are not blocked; the
migration holds that single statement, since a multi-statement query runs as
an implicit transaction. A PostgreSQL integration test, skipped without
WEKNORA_MIGRATION_TEST_POSTGRES_DSN, applies the chain, runs 000106 down and
up through golang-migrate, and checks both queries use the index without a
sort; the SQLite migration test checks the same plans.
Calibration. It no longer waits for a total token count, and a usage with
only a scale is persisted. A turn that measures nothing carries forward the
scale it started from, so the newest turn always has the latest one.
* fix(security): close cross-workspace access paths in shared spaces
A review of how shared-space (organization) members reach each other's
resources found six paths that bypass the org-share permission model:
- presigned-preview signed any resource handle or storage path, turning a
known resource:// handle into an anonymous /r/ URL that outlives share
revocation. It now requires the path to belong to the caller's tenant,
like /files.
- POST /initialization/initialize let a shared-KB editor rewrite the owner
workspace's models (base URL, API key) in place. Only the KB's own
workspace may call it, and rewriting a stored model needs the same
authority as PUT /models/:id.
- KB JSON carried the legacy inline credentials (storage_config
secret_id/secret_key, which initialization fills with the MinIO env
keys, and vlm_config api_key) to every share receiver. They are withheld
from JSON; their DB columns are unchanged.
- A chat without agent_source_tenant_id resolved shares before the
caller's own agent, so a member sharing a same-ID built-in agent took
over other workspaces' default agent. Own agents now win, built-in
agents can no longer be shared (existing such shares no longer resolve),
and embed channels drop a visitor-supplied source workspace.
- Any org-admin tenant (even a tenant Viewer inside it) could raise another
workspace's KB share to editor/admin, and the route guard was bypassable
via an unrelated :id. Share management is now: the sharer from the source
tenant (Contributor+), source-tenant Admin+, or an org-admin tenant's
Admin+ that may only remove or lower. share_id must belong to :id.
- Wiki write tools were registered for every wiki KB in the search targets,
which only need read access, so a viewer share (e.g. via the built-in
wiki fixer) could edit or delete another workspace's wiki pages. They now
act only on KBs the caller may edit.
* fix(security): close the KB settings path and owner details for share receivers
Follow-up to the shared-space review:
- PUT /initialization/config shares the route guard that admits share
editors, so an editor could still rebind the owner's KB models, chunking
and storage, and storage backends resolved against the caller's workspace
let it bind the owner's KB to its own backend. Another workspace now needs
an admin share (the only share level the frontend offers settings to) and
may not change the storage binding at all.
- GET /initialization/config showed model base URLs and bucket locations to
any admin of the caller's own workspace, including share receivers. They
now only see whether credentials are configured.
- Wiki write tools on the caller's own KBs had no role floor, so a tenant
Viewer (and the IM, embed and MCP endpoint principals that run as Viewer)
could write through an agent what the HTTP routes deny. Writes now need
Contributor+, or the ingest capability for a scoped API key, following
the RBAC rollout switch.
- resolveAgent logs the share lookup error instead of a nil own-agent error
when a shared selector fails.
* fix(security): close the medium shared-space access findings
Follow-up to the high-risk fixes, from the same review:
- Wiki fixer moved into a shared KB's workspace ran with the caller's own
config there: an unset MCP mode registered all of the owner's MCP
services, and an "all" KB scope reached the owner's other KBs. The scoped
run now uses the built-in config pinned to that one KB, without MCP,
skills, sandbox or web search, and falls back to the KB's own model.
- An unset MCP mode means "all" at runtime but is shown as none in the share
scope; shared runs now treat it as none.
- Quick-answer mode used the request's web_search_enabled alone, so a
receiver could turn on search the agent disables and spend the owner's
provider quota. It now requires the agent setting too, as agent mode does.
- summary_model_id picked any model of the workspace a run executes in. It
is ignored for shared agents and the relocated wiki fixer.
- A workspace that left or was removed from an organization kept sharing its
KBs and agents. Its shares are revoked on departure, and share queries and
permission checks require the source workspace's membership and a live
organization, which also covers rows left behind earlier.
- Sharing an agent exposes its KB scope to all members, so a Contributor
could share colleagues' KBs (or "all") through an agent. The sharer must
now be able to share each KB directly (creator or Admin+; "all" needs
Admin+), also when a shared agent's scope is widened later.
- A scoped API key with manage_channels could mint an MCP endpoint token
wider than itself. Endpoints it creates, updates or rotates must stay
within its KB allow-list and capabilities.
- A shared agent's run indexed the receiver's Q&A into the owner's chat
history KB; indexing now uses the session owner's configuration.
- Sharing an agent with skills keeps injecting the owner's skill
environment values (product decision); the share panel now warns that
members can read them. Preview of originals through read-only shares is
kept as designed and documented.
* feat(agent): persist compaction checkpoints across turns
Agent compaction only lived inside the turn that ran it: the summary was
discarded at turn end and LoadAgentHistory rebuilt raw history from the
messages table, so every later turn past the threshold paid for another
summarization of nearly the same history.
A compaction whose history summary ends exactly on a stored turn is now
written onto that turn's assistant message (messages.context_checkpoint).
LoadAgentHistory starts from the newest checkpoint: the summary replaces
that turn and everything before it, and later turns are replayed verbatim.
The compactor already treats a leading summary as the previous summary, so
checkpoints fold forward incrementally.
History messages carry their turn's assistant message ID (chat.Message
TurnID, engine-internal) so compaction can tell whether its cut lands on a
stored-turn boundary. Nothing is persisted when the cut ends inside a stored
turn, covers only the live turn, or the history summary degraded to a raw
archive; a split turn's prefix summary is never persisted because that turn
is replayed whole next time. Writes are column-scoped and best-effort, and a
failed lookup falls back to history without a checkpoint.
Storing the checkpoint on the covered turn keeps the boundary implicit, so
session forks copy it with the turn and deleting the turn drops it. The
HistoryTurns cap still applies, now to the turns after the checkpoint.
* test(skill): wait for the background install in the name-conflict test
InstallSkill queues the install on a goroutine, and the fixture row already
starts at installing, so reading the status right after the call raced the
mock install and failed whenever it landed on ready first. Wait for the run
to finish on the reused row instead, as the other install tests do.
* feat(agent): size agent history by tokens instead of turn count
Agent history was capped at history_turns (default 5) because every turn
used to rebuild raw history and compaction could not persist. With
compaction checkpoints persisted, the cap is no longer what keeps a session
inside the window, and it threw away context that would have fit.
LoadAgentHistory now loads the newest checkpoint summary plus the newest
turns after it that fit a token budget: the run's compaction threshold
(agent.HistoryTokenBudget, the window less the reply reserve). History
that large already triggers the first round's compaction, which persists a
new checkpoint, so compaction rather than a turn count bounds the session.
Loading more would only hand that compaction a summarizer input larger
than the model can read.
Rows are paged backwards by (created_at, id) and the read stops at the
checkpoint or once the budget is full, so a long session is never read
whole. Kept turns stay contiguous, the newest turn is kept even when it
alone exceeds the budget (compaction splits it), and a turn cut by the read
is left out rather than replayed without its original question.
Agent mode no longer reads history_turns: AgentConfig.HistoryTurns is
removed, the editor hides the input in smart-reasoning mode and explains
the automatic behavior, and the smart-reasoning builtins drop the setting.
Quick-answer (KnowledgeQA) keeps its turn cap since it has no compaction.
* Revert "feat(agent): size agent history by tokens instead of turn count"
This reverts commit d05fb6e5f1f161813a88e9127609f7a059f38f12.
Loading history up to the compaction threshold lets the loader drop the
oldest turns before compaction can trigger: whenever a single turn's
replay is larger than the system prompt plus the new question, the
trimmed history stays under the threshold, nothing is summarized, and the
session degrades into a large sliding window that never gets a
checkpoint. A turn-by-turn simulation of LoadAgentHistory with the real
compactor showed zero compactions over 60 turns in those cases.
Keep the persisted checkpoints with the HistoryTurns cap for now; token
budgeted history returns in a separate change with a budget that leaves
the overflow to compaction.
* feat(audit): show API key name on knowledge-base activity
API-key operations currently look like the first tenant user or
"system". Snapshot the key id/name into activity details so the
feed can tell which key initiated the change, without restoring
key authorization onto async workers.
Fixes#3402
* fix(frontend): show activity API key on a second line
The 132px actor column truncates "user · API Key · name" into one
ellipsis. Keep the person on the first line and the key name below.
---------
Co-authored-by: MercuialC <MercuialC@users.noreply.github.com>
Co-authored-by: root <root@localhost.localdomain>
* refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents
The agent's retrieval surface had grown to five overlapping tools whose
behaviour was scattered across long descriptions, interleaved "tips" in
tool output (most of which modelcontext discarded before the model saw
them), and a grep_chunks implementation that ran an unindexed regex scan
over the chunks table ordered by created_at, so large knowledge bases lost
recall and paid a full-table scan on every call.
Tool surface
- search_knowledge replaces knowledge_search and grep_chunks: one query,
mode = hybrid | semantic | keyword, optional knowledge_base_ids, limit.
Keyword mode runs through the retrieval engine's keyword index instead of
the DB regex scan. Modes unavailable for the scope are refused with a
reason. Rerank, MMR and de-duplication are kept; already_seen markers,
retrieval_statistics and the zero-result "CRITICAL" block are gone.
- read_document replaces list_knowledge_chunks, get_document_info and
wiki_read_source_doc: id accepts a dN or cN handle, always returns the
document header, pages by exact offset (fixing the old floor-to-page
bug), finds passages with query (literal by default, regex opt-in) and
reads neighbours around a chunk with context.
- list_documents pages through one knowledge base, honouring @file/@tag
scope.
- wiki_search takes a single query plus regex, knowledge_base_ids, limit;
literal queries are escaped so the model never reasons about JSON
escaping. Legacy queries / knowledge_base_id are still accepted.
- query_knowledge_graph is registered only when a graph-enabled KB is in
scope and its description/tips were trimmed.
Compatibility
- Retired names in stored allowlists, presets and API calls are mapped to
their successors by NormalizeAllowedTools at registration time; no data
migration. Stored histories with old names still decode, render and
redact. Calls to retired names return a replacement hint.
- modelcontext gains an "id" key space that decodes against both the chunk
and document handle tables (decode-only), plus rendering for the
document header, chunk roles, in-document matches and next_offset.
Prompts and surfaces
- The RAG, wiki researcher and hybrid templates now name the tools and the
search -> read -> cite order; tool descriptions are short and contain no
workflow instructions.
- Engine display names, history redaction, grounding prompt, IM rendering,
the built-in MCP server (search_knowledge gains mode/limit, grep_chunks
becomes keyword search, read_document gains query, wiki_search gains
regex), frontend editor/renderers/i18n, presets, website docs and the
changelog are updated accordingly.
Tests cover name mapping, mode validation and refusal, output shape,
read_document paging/context/query/scope, list_documents, wiki_search
escaping and legacy arguments, modelcontext decoding and rendering, and
registration-time aliasing and graph gating.
* fix(agent): address review findings on read_document, list_documents and model rendering
- read_document: fetch the second page whenever the first page did not
reach the end of the document, so an unaligned window near the end no
longer drops the trailing chunks (also fixes context windows around a
chunk near the end); a query passed together with a cN handle now
searches the owning document instead of being ignored; query mode stops
collecting once the output budget is nearly spent and reports truncated;
the document row is authorized in place instead of being fetched twice;
the context read takes the total from the window fetch instead of a
separate count query.
- list_documents: resolve the page's document scope in one batched tag
lookup instead of one call per row.
- modelcontext: render read_document output once and read next_offset
through the existing intValue helper.
- search_knowledge: point title/file-name lookups at list_documents.
- Tests for the boundary window, chunk context near the end, cN + query,
and budget-bound query results.
* fix(agent): close compatibility gaps in the consolidated retrieval tools
- Wiki-only scopes: read_document and list_documents are registered
whenever a wiki KB is in scope, since every KB stores chunks. Their
capability requirement includes wiki but is marked auxiliary, so they do
not widen a RAG agent's derived "all knowledge bases" filter to wiki-only
bases (Go and frontend derivations both honour it).
- search_knowledge resolves the retrieval mode per knowledge base instead
of refusing the call: FAQ and vector-only bases answer keyword requests
semantically, keyword-only bases answer semantic requests by keyword.
Results carry requested_mode and mode_fallbacks, the empty-result text
names the fallback, and the call is refused only when no base in scope
has a chunk index.
- The model rendering uses the mode the search actually ran with (it was
always labelled semantic) and emits <mode_fallback> per degraded base.
- list_documents applies the @file/@tag scope before paging: tags are
pushed into the database filter, pinned documents are loaded by ID, and
total/next_page count only readable documents.
- read_document resolves chunk context by chunk_index through a new
ChunkRepository.ListChunkNeighbors, so parent, summary and image chunks
sharing the index sequence no longer shift the neighbourhood; offset is
documented as a reading-order position, not a chunk index.
- wiki_search (agent and MCP) keeps its regular-expression default and
only matches literally when the text is not a valid expression;
regex=false forces a literal match and regex=true rejects invalid input.
- MCP grep_chunks keeps grep semantics: literal terms from the pattern
fetch candidates from the keyword index (semantic for bases without
one), and every returned chunk is verified against the pattern.
* docs(changelog): align the consolidation entry with regex wiki_search and grep_chunks semantics
* docs(mcp): note that grep_chunks recall is bounded by index candidates
* feat(mcp): add built-in MCP server with per-workspace endpoints
WeKnora previously exposed itself to MCP clients only through the Python
package under mcp-server/, which binds one process to one API key and
therefore to one workspace, and mirrors the REST API tool-for-tool.
This adds a native MCP server surface to the Go backend:
- New mcp_endpoints table (PostgreSQL 000100, SQLite 000020). A workspace
can publish any number of endpoints, each with its own bearer token
(SHA-256 stored, rotatable, shown once), knowledge-base allowlist, tool
allowlist, default agent and per-minute rate limit.
- Management API under /api/v1/mcp-endpoints (Viewer reads, Admin writes,
API keys need manage_channels) plus a tool catalog endpoint.
- internal/mcpserver hosts one mcp-go Streamable HTTP server in stateless
mode at /mcp/:endpoint_id. tools/list is filtered per endpoint and
tools/call re-checks the allowlist, applies rate limiting and touches
last_used_at. MCPEndpointAuth resolves the token and injects the
workspace, an mcp_endpoint principal and an API-key style scope derived
from the endpoint, so existing knowledge-base and capability checks
apply unchanged.
- Redesigned tool set (12 tools in four groups): retrieval and reading
reuse the agent's native tools; ask runs the default or a named agent
synchronously and returns a cited answer with a session_id for
follow-ups; wiki tools cover search, read and index; write tools are
off by default. Knowledge bases accept ids or names.
- Frontend: new "MCP Server" integrations tab (tab id mcpserver, to
avoid clashing with the existing MCP-client settings section) with
endpoint cards, a create/edit drawer, a one-time token dialog and
ready-to-paste snippets for Cursor/VS Code/Claude Desktop, Claude Code
and mcp-remote. Locales updated in all five languages.
- Deployment capability integrations.mcpserver wired on both sides.
- Docs: website-docs MCP page now leads with the built-in server; the
Python mcp-server README carries a deprecation notice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): wire the MCP server into DI/router and tighten authorization
The previous commit lost eight staged files to the repo pre-commit hook,
which resolves ROOT to the main checkout and re-adds the main-checkout
copies of tracked files into the worktree index. This commit restores them
(container providers, RouterParams and route registration, context key,
principal, clone table entry, deployment capability key, SQLite parity
test) and addresses the review findings:
- Knowledge-base scope is authorized, not trusted: endpoint allowlists and
every tool call resolve each knowledge base through access.ResolveKB for
the calling principal (ownership or organization share). Foreign IDs are
rejected on create/update and dropped at call time.
- Write tools mint explicit editor grants via access.KBAccess.WithGrant
before calling the knowledge service, so add/update/delete go through the
same RequireKBWrite path as the REST API. add_document also checks the
grant itself.
- ask no longer accepts agent_id. It runs only the endpoint's configured
agent; tenant-owned agents and user-facing builtins qualify, internal
builtins (wiki fixer, skill installer) are refused both on save and at
call time. Web search is forced off for MCP calls.
- Tool annotations match behaviour: ask is not read-only, add_document is
a mutation.
- CORS allows MCP-Protocol-Version / Mcp-Session-Id / Last-Event-ID and
exposes Mcp-Session-Id for browser-based MCP clients.
- Rate limiter runs its local cleanup goroutine; SQLite migration gains the
deleted_at index for parity with PostgreSQL; non-ASCII endpoint names get
unique mcpServers keys derived from the endpoint id.
- Swagger annotations on the management handler.
- Tests: KB isolation (service and server), write grants, endpoint-agent
restriction, tool annotations, and a router test that mounts the public
/mcp/:endpoint_id surface and checks the deployment capability.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(mcp): renumber migrations after rebase onto main
main took 000100 / 000020 for chunks_index_diet, so the mcp_endpoints
migrations move to 000101 (PostgreSQL) and 000021 (SQLite); the SQLite
parity test and docs follow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): run document tools under the knowledge base owner tenant
Shared knowledge bases resolved through access.ResolveKB were only half
usable: documents of an organization-shared knowledge base are stored under
the owner tenant, so the document-level tools that queried by the calling
tenant found nothing, and add_document would have created rows under the
caller's tenant inside someone else's knowledge base.
- knowledgeInScope loads documents tenant-agnostically and then requires
the document to belong to an allowed knowledge base and to that base's
tenant.
- scopedKBContext replaces grantContext: it resolves the grant, switches
the execution tenant to the knowledge base owner (KBAccess.Context) and
swaps TenantInfo to the owner so storage/model settings resolve there.
list_documents, read_document and every write tool run under it.
- Locale text and docs no longer claim clients can pass agent_id.
- Go SDK gains MCP endpoint methods (client/mcp_endpoint.go) with docs, and
the Swagger artifacts are regenerated.
- Tests: shared knowledge base read/write under the owner tenant, spoofed
document tenant rejected, viewer share cannot write.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore(mcp): renumber migrations after rebase onto main
main now carries 000101 / 000021 (knowledge_profiles), so the
mcp_endpoints migrations move to 000102 (PostgreSQL) and 000022 (SQLite);
the SQLite parity test expects version 22 and the docs follow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(mcp): guard last_used touch without a repository and wrap SDK line
The transport tests build the server without a repository; the background
last_used_at write then dereferenced a nil interface and, on the slower CI
runner, took the test binary down. touchLastUsed now returns early when no
repository is wired, and the tests inject a recording repository and assert
the touch instead. Also wraps a 123-column SDK signature for the client
module's lll check.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* feat(knowledge): derive knowledge base descriptions from document profiles
Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.
Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.
Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.
Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.
Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.
Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.
* fix(knowledge): keep generated descriptions out of duplicates and refresh clones
DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.
CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.
* fix(knowledge): refresh the loaded clone target and carry document profiles
ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.
cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
* refactor(wiki): consolidate source_refs parsing and resolution
source_refs entries ("uuid" or "uuid|title") were parsed by ten
independent implementations that disagreed on whitespace handling, on
whether a model-supplied title could be trusted, and on how a leading
pipe was treated. The scope-enforced and legacy agent resolvers also
duplicated the lookup and the routing step looked every document up a
second time.
- types.ParseWikiSourceRef / FormatWikiSourceRef / WikiSourceRefMatchesKnowledge
are now the only readers and writer of the format.
- resolveWikiSourceDocuments replaces resolveAuthorizedSourceRefs,
resolveSourceRefs and wikiKnowledgeBasesForSourceRefs: one lookup per
document, titles always rebuilt from server data, dedupe by canonical ID,
unknown documents fail closed in every mode.
- removeSourceRef now matches padded or titled refs of a deleted document.
* refactor(data_analysis): expose the selected document as a fixed table name
The tool required the document ID inside the SQL text and blindly
string-replaced it with the physical DuckDB table name, which forced the
model-context codec to rewrite handles inside quoted SQL and let the
UUID-derived k_<uuid> table name reach the model through stored schema
summaries.
The document is now selected only by knowledge_id. Every query runs on a
dedicated connection where "dataset" is a temporary view over the
physical table (DuckDB temporary objects are connection-local, so
concurrent sessions stay isolated). The physical name remains accepted
for internal callers; any other table fails validation with an error
naming the table the model should use. Schema summaries, the schema
tool, the analyst prompt and the tool schema all name the same table, so
data_analysis.sql no longer carries source handles and its text codec
policy is removed.
* feat(modelcontext): report raw identifiers that survive encoding
Text-level compaction exists because raw UUIDs reach model context through
producers the registry never sees: error strings, slugs, results of
unknown shape. Retiring it requires knowing where those producers are.
Registry.LeakedIdentifiers scans an encoded message list for UUID-shaped
values in every application-composed field (user messages are skipped),
classifies each as registered (codec missed a rewrite) or unregistered
(producer emitted a raw ID), and SummarizeLeaks renders one line per
field. The agent engine and the three RAG pipeline call sites log the
report as a warning after EncodeMessages.
* fix(chat_pipeline): align RAG data analysis with the fixed table contract
The RAG data-analysis plugin still asked the planning model to fill in
knowledge_id and to write SQL from a prompt that exposed the raw
knowledge UUID, then passed the model's JSON straight to Execute. Under
the new data_analysis contract a query that names anything but the
dataset table is rejected, and the plugin logged that rejection at error
level and moved on, so the analysis vanished without any signal. A
"no analysis needed" plan took the same error path.
The pipeline now owns document selection: the model returns only
needs_analysis and sql, the prompt names the dataset table and never
shows the knowledge ID or physical table, and the plugin builds the tool
input itself. A rejected query is fed back to the model once for repair;
a query that still fails is reported through pipelineWarn with the SQL
and the tool error, and "not needed" is logged as such instead of as a
failure.
Covered by unit tests with a scripted model and executor: not-needed,
pipeline-owned knowledge_id, one-shot repair, persistent failure
surfaced, unparseable plan rejected.
Serply returns Google results through a keyed JSON API, which fills the
gap between Google Custom Search (needs an engine ID and a restricted
scope) and the scrapers. The adapter follows brave.go: hardcoded
endpoint, redirects refused so the key never leaves the official host,
4MB response cap, and per-call country and freshness filters. country
maps to Google's gl parameter and pd/pw/pm/py map to tbs qdr:d/w/m/y.
Serply has no date-range filter, so that form returns an error instead
of being silently dropped. num is clamped to 10 because the API serves
one results page per call.
Wiring matches the Bocha provider: type constant and metadata, registry
factory, service validation, frontend union, swagger enum entries (edited
by hand; a full swag regen drags in unrelated drift), and the web search
docs table.
* feat(website): add product homepage and unified docs deployment
* fix(website): build the unified site in a multi-stage Docker image
* fix(website): reveal and highlight the active documentation sidebar item
* fix(milvus): support multilingual BM25 retrieval
* fix(milvus): preserve source metric and tighten collection matching
Read the source embedding index metric during BM25 migration, match
collections by {prefix}_{dimension}, and route Japanese/Korean text to
the ICU fallback.
* fix(milvus): satisfy lint on multilingual BM25 changes
---------
Co-authored-by: Jonesxq <239089032+Jonesxq@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(agent): improve web search and persistent page reading
* fix(agent): make web fetch snapshots waitable and retryable
Keep in-flight page fetches independent of a caller's short deadline so
content=true and continuations cannot cancel a shared download or miss a
snapshot that is still loading. Mark snapshot_expired as retryable, fetch
only the first three search hits in parallel, and stop forcing Brave's
country to US.
* feat(i18n): add Japanese (ja-JP) locale support
Add a complete Japanese UI locale for the admin frontend and the web
embed widget, and register it everywhere the supported-locale list is
enumerated.
- frontend/src/i18n/locales/ja-JP.ts: full bundle (5,600+ keys) aligned
with en-US/zh-CN key structure; verified by the locale key audit
- frontend/src/i18n/embed.ts: ja-JP block for the embed widget, plus
`ja*` normalisation in normalizeEmbedLocale
- Register ja-JP in i18n index, SUPPORTED_LOCALES, localeKeyAudit
bundles/order, TDesign locale map (ja_JP), login language picker,
general settings dropdown and embed default-locale options
- Add `language.jaJP` key to every locale so the settings dropdown can
use $t like its siblings
- Accept ja-JP as an embed channel default_locale on the backend and as
DEFAULT_LOCALE in the frontend docker entrypoint
- Extend locale-enumerating tests and update accepted-value lists in
.env.example, public/config.js, widget script and website-docs
* feat(i18n): add ja-JP names and descriptions for prompt templates
Add a ja-JP block to the i18n map of all 32 prompt templates
(system_prompt, context_template, rewrite, fallback, intent_prompts,
agent_system_prompt) so the template picker shows Japanese labels when
the UI locale is ja-JP instead of falling back to the Chinese names.
Prompt content is unchanged.
* feat(i18n): add ja-JP strings for the CLI integration page
main gained the CLI setup page (#3081) with 18 new integrations.cli.* /
integrations.tabs.cli keys; add their Japanese translations so the
locale key audit stays green after rebasing onto main.
* style(i18n): unify ja-JP spacing and long-vowel conventions
- Remove spaces between Latin tokens and adjacent Japanese text
(e.g. "MCP サービス" -> "MCPサービス"), matching the majority style
already used in the bundle
- Drop the trailing long-vowel mark on common technical katakana terms
(サーバ, ユーザ, フォルダ, プロバイダ, パラメータ, ヘッダ, クラスタ, ...)
so the whole locale follows one convention
* feat(i18n): add ja-JP labels for agent type presets
config/agent_type_presets.yaml carried default / zh-CN / zh-TW labels
only, so the agent editor showed English preset names in the Japanese
UI. Add a ja-JP label and description to all five presets.
* feat(agent): unify sandbox tools and drop host preloaded skills
Read workspace and skill files through read_file, and run installed or staged skills through shell_exec(skill_name). Remove the unused skills/preloaded tree and keep generated-file previews stable across re-renders.
* fix(frontend): do not assume classList in markdown sanitize hooks
Node unit tests stub anchors without a DOM classList. Detect download cards from class/download attributes instead.
Refuse truncated tool calls, compact-and-retry once on overflow, and let
sandbox files be appended, paged, and batch-edited instead of rewritten.
Models can set context_window (default 200K) so compaction sizes itself
from the real window; agent max_iterations of -1 means unlimited.
* fix(models): show usage details when deletion is blocked
* fix(models): open the matching agent section and cap usage lists
Agent "open configuration" now lands on the tab that owns the binding
(VLM/ASR vs follow-ups), and delete failures keep the server message when
structured details cannot be parsed. Listed KB/agent rows are capped at 50
while totals still drive the deletion guard and dialog headings.
---------
Co-authored-by: Leon <leon_hfut@foxmail.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(auth): record last-active-tenant preference on SwitchTenant
* docs(auth): note SwitchTenant updating last-active-tenant preference
* fix(auth): persist last-active tenant before SwitchTenant tokens
RefreshToken re-resolves from this preference because refresh JWTs omit
tenant_id. Fail the switch if the write fails so a 200 never returns a
token pair whose later refresh would land in a different workspace.
* fix(cli): align HybridSearch interfaces with SDK variadic options
Client.HybridSearch gained ResourceURLOptions; CLI ChunksService and
MCP knowledgeService still used the old signature, so *Client no longer
satisfied those interfaces.
* style(cli): wrap HybridSearch signatures under the 120-column lint
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* feat(provider): add LiteLLM as an OpenAI-compatible provider
* docs: use revive-compliant comment for ProviderLiteLLM const
* fix(provider): stop defaulting LiteLLM to localhost under SSRF
The catalog URL localhost:4000 is blocked by default SSRF and does not
match DetectProvider, so UI/CLI could neither save nor auto-detect it.
Use a placeholder host, keep thinking aligned with generic, and sync
ModelSource across types, Swagger, and docs.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
Local was a host-subprocess fallback that skill install, snapshots, and
session sandboxes already excluded. Drop it so Cube, E2B, and Docker are
the only backends; leftover local configs can only be deleted.
* fix(hybrid-search): normalize MatchCount once instead of at the truncation site
An omitted match_count deserializes to Go's zero value, which reached the
truncation step in HybridSearch and sliced the deduplicated chunk list to
[:0] — a successful retrieval returned an empty result set. A negative
value panicked on that same slice bound and surfaced as a 500.
Resolve the value once at the top of HybridSearch via normalizedMatchCount
so all three consumers agree, rather than patching the truncation site
alone: the over-retrieval floor, the FAQ iterative-retrieval trigger
(which never fired on this path because len(chunks) < 0 is false), and the
truncation itself.
Bound the FAQ iterative path while here, since normalization is what makes
it reachable with a large MatchCount: its TopK seeds from MatchCount*3 and
doubles five times, previously without limit. It now stops at the pool cap
instead of re-issuing an identical query.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(retrieval): share the default retrieval depth as one constant
The value 50 was written three times — the over-retrieval floor in
HybridSearch, the MatchCount fallback added alongside it, and
RetrievalConfig.GetEffectiveEmbeddingTopK — with nothing recording that the
three are meant to agree. Anyone tuning one would silently desync the
others.
Promote it to types.DefaultRetrievalTopK. No behavior change.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: drakezhang <drakezhang@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(sandbox): unify remote sandbox on E2B protocol with self-hosted support
- Add e2b.proxy_url gateway config and generic gateway transport layer
- Add envd compat layer for Basic auth and multipart /files uploads
- Support Dockerfile.sandbox runtime and e2b-backend build targets in CI
- Rename sandbox image non-root user to align with E2B template convention
- Add sandbox-protocol.md and extend cluster deployment / local repro docs
Enables connecting to any E2B-compatible control plane (E2B Cloud,
CubeSandbox, Agent-Sandbox, etc.) without per-backend client code.
* style(sandbox): gofmt tenant_resolver struct alignment
Import and update WeKnora documentation with VitePress: getting started,
architecture, features, API reference, clients, and development guides.
Includes landing page with inline SVG illustrations, theme components,
doc check scripts, and product screenshots.
This commit introduces a complete set of documentation for WeKnora, organized into six main sections: Getting Started, Architecture, Features, API Reference, Clients, and Development Guide. Key highlights include:
- Detailed installation instructions and quick start guides.
- In-depth architectural overview and backend design documentation.
- Comprehensive feature descriptions, including multi-tenant support, document parsing, and agent capabilities.
- API reference covering approximately 360 endpoints with usage examples.
- Client documentation for various interfaces including web, CLI, and SDK.
This documentation aims to facilitate user onboarding and provide a thorough understanding of the WeKnora system.