Commit Graph

112 Commits (cf502ba46a629598db393d747be05da3df682c18)

Author SHA1 Message Date
lyingbug cf502ba46a
feat(plugin): event delivery and webhooks for plugins (P2 batch 6) (#3728)
* feat(plugin): event delivery and webhooks for plugins
2026-09-26 14:47:13 +08:00
lyingbug 04adc13272
fix(retrieval): fix recall correctness, score scales, cost and agent tools (#3687)
* fix(retrieval): keep index writes and shared-KB lookups correct

Index writes
- Image OCR/caption chunks were indexed without IsEnabled, so every engine
  but Postgres stored them disabled and retrieval never returned them.
  The SQLite engine re-enables the affected rows at startup.
- FAQ edits, similar-question additions and import merges re-index under
  the old SourceID and relied on the engine overwriting it. Postgres and
  SQLite kept the stale row (ON CONFLICT DO NOTHING); ES v8 and Qdrant
  mint a new document ID and kept both. FAQ re-indexing now deletes the
  old rows first, and Postgres/SQLite BatchSave upserts by
  (source_id, source_type).
- Postgres: drop the GORM default tag on is_enabled, which replaced false
  with true on insert and indexed disabled chunks as enabled.
- SQLite: BatchSave no longer attaches vectors and FTS rows to the wrong
  rows when part of a batch already exists, and vecTables is guarded
  against concurrent map writes.
- CopyIndices: Postgres filters to the copied knowledge, pages by id and
  keeps is_enabled; SQLite copies every row of a chunk (generated
  questions included) with stable source IDs and enabled state.

Shared knowledge bases
- Chat merge: FAQ answer population and neighbor expansion look chunks up
  without the caller-tenant filter, like parent resolution (#3342).
- Neighbor expansion never cuts the retrieval hit; the length budget goes
  to the text adjacent to it.
- FAQ negative questions are applied whenever an FAQ KB is in scope and
  to chunks of shared FAQ KBs.
- Search results get image_info for shared-KB chunks.
- search_knowledge reads FAQ metadata from the search result instead of
  one tenant-scoped query per hit, so shared FAQs render their answers.

* fix(retrieval): put every retrieval score on one scale

Scores from different retrieval paths were ranked together while living on
different scales, so ordering depended on which path produced a hit.

Fusion
- RRF ranked a chunk by its position in the concatenation of every
  vector (or keyword) list, appended in goroutine completion order. The
  second list's best hit ranked behind every hit of the first, and the
  order changed between identical requests. Each list is now ranked on
  its own by score and a chunk takes its best rank.
- RRF is divided by its maximum, so hybrid scores land in [0, 1] like
  vector-only (cosine) and keyword-only (rescaled BM25) results. Raw RRF
  topped out near 0.016 and lost to any single-retriever hit it was ranked
  against across searches (document targets, model groups, FAQ calls);
  rerank's base-score term and MMR's relevance term had nothing to use.
- Keyword-only BM25 is rescaled per list; engines' BM25 is not comparable.
- FAQ iterative retrieval fuses each round like the first pass instead of
  sorting raw BM25 and cosine together, starts at twice the depth already
  searched, triggers when any single vector list is full (the old summed
  count never matched with more than one list) and replaces results per
  round.

Rerank and chat pipeline
- The FAQ boost is no longer capped at 1; capping made strong FAQs tie at
  1.0 and fall to the tie-breaker. The composite score before boosts is
  kept as composite_score.
- The FAQ direct-answer threshold is compared against that pre-boost
  score, and the FAQ that cleared it is the one marked match="exact"
  rather than the first FAQ.
- Graph hits no longer carry a fabricated retrieval score of 1.0, which
  ranked them above every scored hit when rerank did not run; the
  composite score uses the model score in its place.

Engines
- OpenSearch and Weaviate return (1 + cos) / 2; their drivers now report
  cosine similarity and map thresholds onto the engine scale, so a
  threshold means the same on every engine.
- Milvus L2 returned squared distance, ranked farthest-first downstream,
  and took the threshold as a radius; scores and radius are converted.
- The multi-store normalizer no longer maps Milvus through (s + 1) / 2,
  which put it on a different scale from every other engine and from its
  own single-store results.

Docs updated for fusion, normalization, FAQ boost and iterative recall.

* perf(retrieval): bound rerank, agent and pipeline cost; stop failing on slow or missing models

Rerank
- Rerankers report the longest document a request can carry
  (max_document_chars / max_request_chars minus the query) and the rerank
  stage fits its own passages to it, trimming appended captions, OCR and
  generated questions first. One chunk with a large screenshot's OCR used
  to exceed the vendor limit and fail the whole batch, so every candidate
  lost its score and the threshold stopped applying.
- Rerank HTTP requests default to a 60 s timeout; chat and agent contexts
  carry no deadline, so a stalled self-hosted rerank server held the turn
  open.
- OCR and caption chunks no longer repeat their own text from image_info in
  the passage.
- The chat pipeline and the agent tool rerank at most 200 candidates, best
  retrieval score first.
- A rerank model that cannot be loaded degrades to retrieval order instead
  of failing every chat turn, as the search APIs already do; without rerank
  results the merge stage keeps only RerankTopK hits instead of enriching
  every one.

Retrieval APIs
- knowledge-search match_count and rerank.top_k are capped at 200
  (unbounded values multiplied into tens of thousands of billed rerank
  passages); normalizedMatchCount clamps to the pool size so the
  over-retrieval arithmetic cannot overflow.
- hybrid-search with rerank recalls at least rerank.top_k deep.

Agent
- search_knowledge fits its rendered output to four fifths of the tool
  output budget, dropping the lowest-ranked rows and telling the model how
  many it omitted; the model's view is rebuilt from Data, which registry
  truncation never reached.
- Agent searches skip neighbor/parent/related enrichment rows, which were
  never retrieved yet tripled the rerank input.
- ParallelToolCalls is enabled. The model was always told it may call tools
  in parallel while the engine ran them one by one; only read-only tools
  overlap, anything else stays a barrier.

Chat pipeline
- Query expansion searches keywords only (its variants are local rewrites
  whose embeddings repeat the main search) and is skipped when keyword
  matching is off.
- Entity extraction, a second LLM call, is skipped for intents that need no
  KB retrieval and uses the rewritten query.
- Partial-overlap dedup tokenizes each text once instead of per pair, and
  only compares token overlap between texts within 3x of each other's size.

Storage
- Migration 000112 adds btree indexes on embeddings(knowledge_id) and
  (chunk_id); document deletes, chunk re-index and enable/tag updates
  scanned the whole table.
- Shared-KB knowledge rows are fetched in one query instead of one per ID.
- Drop GORM Debug() from the ParadeDB keyword query and the knowledge batch
  lookup, which logged SQL and user queries on every search.

Docs updated for the API limits, rerank timeout and passage fitting, agent
parallel tools and output budget, and pipeline changes.

* fix(retrieval): make agent tools and pipeline stages report what they did

Agent tools
- query_knowledge_graph now queries the graph store: entity names from the
  query are matched in each KB's graph, relations are returned (and shown to
  the model), and the entities' source chunks lead the results, with text
  search as a supplement. It used to run text search only, so a graph-only
  KB always answered "no relevant graph information".
- search_knowledge reports failures instead of "no matching chunks": when
  every search fails the tool fails with the reason; partial failures are
  listed. A failed query embedding falls back to keyword search for hybrid
  KBs, as the chat pipeline does.
- Keyword mode uses rerank to order hits, not to reject them (identifiers
  score low), and the empty-result hint no longer sends keyword mode back to
  keyword mode.
- Dedup keeps siblings under one parent, no longer merges every image chunk
  of a document through chunk_index 0, and keeps the better-scored copy.
- Search rows omit a <match> excerpt that only repeats the content; snippet
  terms are segmented for Chinese queries and skip stopwords, and FAQ
  matched questions are chosen by match strength.
- read_document query mode returns next_offset when truncated and resumes
  from offset.
- wiki_search over-fetches under document/tag scopes before filtering and
  says when every match was out of scope.
- Prompts point to read_document(id=cN, context=k) for surrounding text.
- The agent editor hides the retrieval-strategy section in agent mode,
  where search_knowledge uses the global retrieval settings.

Chat pipeline
- Intent labels are normalized; an unknown label retrieves instead of
  silently skipping the knowledge base, and web_search without web search
  enabled searches the knowledge bases.
- Image turns get a 2048-token rewrite budget, and a reply cut off by the
  cap is salvaged field by field instead of being discarded.
- History references are matched by how much of the query they cover;
  Jaccard against chunk-sized text never reached its threshold.
- Entity search resolves chunks of org-shared KBs within the turn's KB
  scope, skips chunks whose document is gone (it used to panic) and caps
  graph chunks at 30.
- Empty history is not fetched twice; fetched pages are cut on a rune
  boundary.

Retrieval
- A multi-KB search whose primary KB has no vectors embeds the query with a
  vector KB's model instead of failing on an empty model ID.
- Rerank passages include the chunk's heading breadcrumb, which is part of
  what was embedded.
- SQLite FTS searches the content column only and turns a lone Han
  character into a prefix query; TencentVectorDB ef follows TopK.

Docs updated for the agent tools and pipeline stages.

* fix(retrieval): address review findings on the retrieval fixes

- Restore NeedsRetrieval for web_search without web search enabled: the
  intent prompt "web_search" is how the user learns web search is off, and
  running KB retrieval instead bypassed it.
- query_knowledge_graph keeps the case of entity terms; Neo4j matches node
  names with case-sensitive CONTAINS, so lowercased terms never matched
  English entities.
- An empty search_knowledge or query_knowledge_graph result states partial
  failures in its text, which is all the model sees for an empty result.
- SQLite BatchSave replaces a source in one transaction, so a failed insert
  keeps the old rows instead of leaving the source unindexed.
- Reuse stringSliceValue and an exported searchutil.ContainsChinese instead
  of new copies; drop an unused named return and the FAQ obsolete-source
  delete that the full per-chunk delete now covers.

* fix(retrieval): address second review of the retrieval fixes

- query_knowledge_graph: under a document/tag scope, keep only relations
  whose entities come from an in-scope evidence chunk (the graph namespace
  is the whole KB); drop chunks whose document no longer exists; report a
  graph or text search failure even when the other source returned hits.
- search_knowledge: record the hybrid -> keyword fallback taken when the
  query embedding fails in mode_fallbacks, so an empty result is not read
  as "no semantic match".
- query understanding salvage: take a cut-off value only for the image
  description; a truncated rewrite_query or intent is dropped instead of
  replacing the user's query.
- Graph hits (no retrieval score) are kept past the no-rerank RerankTopK
  cut and the rerank candidate cap instead of always falling below them.
- FAQ boost is capped at 1 again so scores stay on the [0, 1] scale; FAQs
  tied at the cap are ordered by their pre-boost score.
- Negative-question filtering loads only candidates of FAQ knowledge bases.
- read_document query continuation starts one chunk before offset so the
  first match keeps its preceding context at a page boundary.
- graphSearchTerms and searchQueryTokens share one queryTerms splitter.
2026-09-25 10:48:18 +08:00
lyingbug 9fc08c2079
feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675)
* feat(retrieval): unify rerank across search APIs and expose diagnostics

The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.

- internal/reranking: one rerank stage (passage = title + cleaned body +
  captions/OCR + generated questions, threshold with degradation and
  top-1 fallback, composite score, MMR) used by the chat pipeline, the
  agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
  switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
  rerank model; an unknown requested model is a 400. Load or call
  failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
  an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.

Refs #2976

* docs(retrieval): lead with which search API to use and when

knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
2026-09-24 20:41:49 +08:00
wizardchen 02734d7499 feat(chat): add session reasoning controls and polish model configuration 2026-09-23 17:12:19 +08:00
徐少 2c6ae0c262
fix(session): reject invalid QA queries before session lookup (#3477)
* fix(session): reject invalid QA queries before session lookup

* fix(session): narrow shared QA input validation
2026-09-23 12:01:33 +08:00
hahahahahayesyeseys 78d7d862e0 fix(chat): surface truncated quick answers 2026-09-22 19:44:38 +08:00
lyingbug 33c0333ec9
fix(chat): restore shared workspace images after stream completion (#3523)
* fix(chat): restore shared workspace images after stream completion

* test(chat): wrap shared image completion cases for lint

* fix(chat): wait for persisted completion and preserve streamed replies
2026-09-22 00:16:58 +08:00
nullptr-error d4b9fd0c72
feat(session): in-place rewind with workspace checkpoint reset (#3469)
* refactor(session): extract shared workspace git reset helper

Fork bootstrap and in-place rewind both need the same /workspace reset
script. Move it out of fork_bootstrapper so rewind can reuse it without
copying ExecShellCommand plumbing.

* feat(message): add DeleteMessagesFrom for rewind truncation

Rewind needs a composite created_at+id cursor so same-millisecond
messages after the fork point are not left behind. Delete inside a
transaction and return the rows so callers can log the count.

* feat(session): add in-place rewind service

Truncate the current session at a user or assistant message and git-reset
the live sandbox when a reachable checkpoint exists. Reset failure is
fail-closed so messages are never deleted ahead of the workspace.

* feat(session): expose POST /sessions/:id/rewind

Wire SessionRewindService through the session handler and container so
the client can rewind in place. Busy sources map to 409; git reset
failures stay 500.

* feat(chat): add rewind API client and i18n copy

Expose rewindSession with the same 180s timeout as fork, and add
chat.rewind strings in all five locales so the confirm dialog is not
hardcoded Chinese.

* feat(chat): add rewind button with confirm dialog

Offer in-place rewind next to fork on user and assistant messages.
Confirm before truncating the thread, then reset locally and prefill
the original question when the cut point is a user message.

* fix(chat): confirm rewind with a compact popover

Replace the full-page confirm dialog with a small popconfirm above the
rewind button, and shorten the swagger description to satisfy line length.

* fix(chat): skip markdown images with an empty destination

Empty ![]() was rendered as <img src="">, which shows as a broken image in the reply.

* fix(chat): clip sandbox panel slide-in and rename open tooltip

Keep the enter animation inside the viewport so it cannot flash a document
scrollbar and jump the page. Rename the header toggle from sandbox terminal
to expand sidebar.

* fix(session): 回滚时完成截断、互斥发送并重拉历史

客户端中断不再让 git reset 成功后留下未删的消息;reset/删消息前复查 busy,并锁住当前会话输入。成功后重拉历史,避免切会话预填串台和 steer 分段截错。

* fix(session): 工作区 git 外置,并让未开箱 fork 的回滚对齐 SHA

checkpoint、回滚和 fork 共用 --git-dir=/var/lib/weknora/workspace.git,避免 rm -rf /workspace 丢掉历史。
未开箱分叉会话回滚时改写或清除 ForkBootstrap,避免第一次开箱仍 reset 到旧切点。

* fix(session): 回滚加会话锁,并堵住发送中回滚与脏 SHA

同一会话并发回滚互斥(进程内 + Redis),冲突返回 409。发送/流式中禁用回滚按钮;历史重拉成功后再改界面。未开箱 fork 的 checkpoint SHA 必须是 40 位 hex。工作区 git 只认外置 GIT_DIR,不再 mv/rm 根上的 .git。

* fix(session): 未开箱 fork 回滚成功时不要提示没有沙箱

ForkBootstrap 已对齐切点后返回 workspace_reset,前端走「已回滚」;清掉引导时改报 NO_CHECKPOINT。纯聊天仍用 NO_SANDBOX。

* fix(chat): stub composerLocked in steer interaction tests

handleSteerMsg now bails when rewind locks the composer. The VM harness
did not provide that ref, so CI threw ReferenceError on every steer case.

* fix(session): allow git-layout mkdir in pinned checkpoint test

SkipWorkspacePrep still omits the input/output bootstrap exec. The
checkpoint script now mkdirs WORK_TREE and GIT_DIR, so the old
"command must not contain mkdir" assertion was stale.

* fix(session): block send while rewind holds the session lock

BeginTurn now fails if the rewind key is held, so AgentQA cannot start
during git reset. Knowledge-chat and executeQA check the same lock
before persisting messages. composerLocked is still page-local.

* fix(session): close send/rewind races and fail closed without a checkpoint

Hold the turn lease before persist so rewind cannot truncate an in-flight
send. TryLockRewind fails while a turn is live; emptying the conversation
resets the workspace through the same git path as a SHA rewind.

* fix(session): wrap rewind reset call to satisfy line length lint

* fix(session): fail closed on a replaced sandbox and close leftover rewind holes

Do not truncate history when the live workspace cannot follow the checkpoint, retry deletes after a successful git reset, and keep send and rewind exclusive even without a sandbox manager.

* fix(session): address rewind review findings

Correctness:

- Adopt a pre-move /workspace/.git into SessionGitDir on first use. Sandboxes
  provisioned before the git dir moved out of the work tree still hold their
  checkpoints there, so every SHA recorded on those turns stopped resolving:
  rewind 500'd instead of truncating and fork's rollback failed on a bad
  object. Adoption is guarded on the new dir being absent and on the work-tree
  repo carrying the checkpointer's committer identity, so an agent's own
  `git init` in /workspace is left alone.
- Only fail a send closed when BeginSessionTurn reports a rewind lock. Every
  other lease error degrades to running without a lease, as before rewind
  existed; a Redis blip on the lease script was rejecting the user's message.
- Release an abandoned fork snapshot after the truncate lands, not before.
  Clearing the bootstrap column is a DB write a retry can redo; deleting the
  snapshot is not, and a delete that exhausted its retries left the fork with
  no workspace to boot from under a full transcript.
- Re-acquire the rewind lock when a renew call errors, not just when it
  reports the lease lost. Sitting the tick out let a short Redis outage expire
  the 2m key mid-reset and hand the session to a second rewind.
- Keep the history cursor when a post-rewind reload fails. Clearing it sent
  the next scroll-up back to the newest page, which the kept prefix already
  holds, instead of to older messages.

Efficiency:

- Add ListAssistantCheckpointsUpTo and use it to answer "which checkpoint does
  kept history still reach". Rewind was reading every message body and artifact
  row in the session to find one SHA.
- Take the send-side turn lease once per request. The handler already holds it
  before persisting; QARequest.TurnLeaseHeld lets the QA services skip their
  own, so IM/MCP still take one and HTTP send stops doubling every rewind check
  and lease round trip.
- Batch the suggestion delete behind DeleteByMessageIDs instead of one query
  per deleted message while the rewind lock is held.
- Resolve the rewind affordance for the whole transcript in one pass rather
  than two list scans per rendered row on every streamed token.

Cleanup:

- Drop the SANDBOX_REPLACED skip reason, which the service never produces (it
  fail-closes with a 409), along with its copy in all five locales.
- Drop GetMessagesBySession from rewindMessageStore; rewind never calls it.
- Drop the unused chat.rewind.confirmTitle copy from all five locales.
- Restore the chatHeader.toggleSandboxPanel wording this branch had changed
  unrelated to rewind.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-20 21:01:00 +08:00
lyingbug db8e7611be
fix(security): close cross-workspace access paths in shared spaces (#3408)
* fix(security): close cross-workspace access paths in shared spaces

A review of how shared-space (organization) members reach each other's
resources found six paths that bypass the org-share permission model:

- presigned-preview signed any resource handle or storage path, turning a
  known resource:// handle into an anonymous /r/ URL that outlives share
  revocation. It now requires the path to belong to the caller's tenant,
  like /files.
- POST /initialization/initialize let a shared-KB editor rewrite the owner
  workspace's models (base URL, API key) in place. Only the KB's own
  workspace may call it, and rewriting a stored model needs the same
  authority as PUT /models/:id.
- KB JSON carried the legacy inline credentials (storage_config
  secret_id/secret_key, which initialization fills with the MinIO env
  keys, and vlm_config api_key) to every share receiver. They are withheld
  from JSON; their DB columns are unchanged.
- A chat without agent_source_tenant_id resolved shares before the
  caller's own agent, so a member sharing a same-ID built-in agent took
  over other workspaces' default agent. Own agents now win, built-in
  agents can no longer be shared (existing such shares no longer resolve),
  and embed channels drop a visitor-supplied source workspace.
- Any org-admin tenant (even a tenant Viewer inside it) could raise another
  workspace's KB share to editor/admin, and the route guard was bypassable
  via an unrelated :id. Share management is now: the sharer from the source
  tenant (Contributor+), source-tenant Admin+, or an org-admin tenant's
  Admin+ that may only remove or lower. share_id must belong to :id.
- Wiki write tools were registered for every wiki KB in the search targets,
  which only need read access, so a viewer share (e.g. via the built-in
  wiki fixer) could edit or delete another workspace's wiki pages. They now
  act only on KBs the caller may edit.

* fix(security): close the KB settings path and owner details for share receivers

Follow-up to the shared-space review:

- PUT /initialization/config shares the route guard that admits share
  editors, so an editor could still rebind the owner's KB models, chunking
  and storage, and storage backends resolved against the caller's workspace
  let it bind the owner's KB to its own backend. Another workspace now needs
  an admin share (the only share level the frontend offers settings to) and
  may not change the storage binding at all.
- GET /initialization/config showed model base URLs and bucket locations to
  any admin of the caller's own workspace, including share receivers. They
  now only see whether credentials are configured.
- Wiki write tools on the caller's own KBs had no role floor, so a tenant
  Viewer (and the IM, embed and MCP endpoint principals that run as Viewer)
  could write through an agent what the HTTP routes deny. Writes now need
  Contributor+, or the ingest capability for a scoped API key, following
  the RBAC rollout switch.
- resolveAgent logs the share lookup error instead of a nil own-agent error
  when a shared selector fails.

* fix(security): close the medium shared-space access findings

Follow-up to the high-risk fixes, from the same review:

- Wiki fixer moved into a shared KB's workspace ran with the caller's own
  config there: an unset MCP mode registered all of the owner's MCP
  services, and an "all" KB scope reached the owner's other KBs. The scoped
  run now uses the built-in config pinned to that one KB, without MCP,
  skills, sandbox or web search, and falls back to the KB's own model.
- An unset MCP mode means "all" at runtime but is shown as none in the share
  scope; shared runs now treat it as none.
- Quick-answer mode used the request's web_search_enabled alone, so a
  receiver could turn on search the agent disables and spend the owner's
  provider quota. It now requires the agent setting too, as agent mode does.
- summary_model_id picked any model of the workspace a run executes in. It
  is ignored for shared agents and the relocated wiki fixer.
- A workspace that left or was removed from an organization kept sharing its
  KBs and agents. Its shares are revoked on departure, and share queries and
  permission checks require the source workspace's membership and a live
  organization, which also covers rows left behind earlier.
- Sharing an agent exposes its KB scope to all members, so a Contributor
  could share colleagues' KBs (or "all") through an agent. The sharer must
  now be able to share each KB directly (creator or Admin+; "all" needs
  Admin+), also when a shared agent's scope is widened later.
- A scoped API key with manage_channels could mint an MCP endpoint token
  wider than itself. Endpoints it creates, updates or rotates must stay
  within its KB allow-list and capabilities.
- A shared agent's run indexed the receiver's Q&A into the owner's chat
  history KB; indexing now uses the session owner's configuration.
- Sharing an agent with skills keeps injecting the owner's skill
  environment values (product decision); the share panel now warns that
  members can read them. Preview of originals through read-only shares is
  kept as designed and documented.
2026-09-18 22:00:02 +08:00
lyingbug 2be7bd4063
fix(agent): search bound knowledge bases before answering, and pass suggested-question origins (#3400)
* fix(agent): search bound knowledge bases before answering, and pass suggested-question origins

A user picked the suggested question "为什么在比较图形时需要谨慎处理?"
and the agent answered in round 1 with no tool call, claiming it had
searched the knowledge base and found nothing, then explained misleading
statistical charts. The question had been generated from "Corners Don't
Look Like That: Regarding Screenspace Ambient Occlusion" in a bound base,
where "comparing graphs" means SSAO falloff curves.

Two things let that happen: the grounding prompt said "Stable general
explanations need no lookup", and nothing told the agent where the
question came from - the base in question had no profile, only two
recent titles in the directory, and the click sent the text alone.

Prompt
- With knowledge tools available, search bound bases before answering a
  question about a topic, even one that reads like general knowledge;
  directory profiles and recent titles cannot prove a base lacks it.
- The general-explanation exemption applies only without bound bases.
- Say you searched or found nothing only when a search tool ran.

Question origin
- Suggested questions from FAQ/document chunks also return knowledge_id.
- Picking one on the home or chat page sends question_origin
  {knowledge_base_id, knowledge_id} with that question only.
- The service keeps the origin only when its base is inside the turn's
  resolved scope and the document belongs to that base; it never
  changes the scope (unlike knowledge_base_ids, which narrows it).
- The engine renders it in runtime_context as <question_origin> with the
  base and document handles and a note to search it before answering.

* fix(agent): keep question origins across file/tag scopes and bind them to the send

Review follow-up for the suggested-question origin.

- Scope check: the origin was kept only when its base was in the resolved
  KB list, which is empty for file-only and tag-only scopes even though
  their bases are searched. It is now checked against SearchTargets after
  they are built; the document is kept only when a target for that base
  covers it (unfiltered base, or a document scope listing it).
  resolveQuestionOriginInfo re-checks SearchTargets, looks up the name of
  a base that is not in kbInfos, and the renderer omits an unknown name.
- Frontend: the origin travelled through module-level "pending" state, so
  a send that returned early (agent not ready, upload in progress,
  steering) left it behind and a second pick overwrote it. It now rides
  with the one send: triggerSend(text, options) -> createSession ->
  send-msg -> sendMsg / changeFirstQuery.
- RetrieveKBOnlyWhenMentioned: suggestions are still generated from the
  agent's bases, but picking one retrieved nothing. The origin now
  selects its base, as an @mention would, when the agent may read it.
- API sync: question_origin in docs/api/chat.md and swagger;
  QuestionOrigin on the Go SDK's KnowledgeQARequest/AgentQARequest and
  knowledge_id on SuggestedQuestion.
2026-09-18 17:00:47 +08:00
nullptr-error 42e6163981
feat(session): fork conversations from a historical user message (#3304)
* feat(session): add fork schema and history queries

为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。

* feat(sandbox): add git, fork snapshots, and bootstrap hook

沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。

* feat(session): checkpoint workspace at the end of each turn

每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。

* feat(session): fork conversations and reap orphan snapshots

支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。

* feat(chat): add session fork entry, prefill, and badges

在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。

* style: add missing newline so session_fork_delete_test.go is gofmt-clean

* fix(session): prune later git history when rolling a fork workspace back

reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。

* style(session): clear golangci-lint findings on the session-fork diff

Pre-push lints the whole PR against main; wrap long lines, check c.Error,
and fix gofumpt/revive/staticcheck so the merge can be pushed.

* fix(chat): show the fork button on hover without shifting the thread

Overlay the action so revealing it no longer expands the user bubble and janks the messages below.

* fix(chat): drop the fork landing banners

Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread.

* fix(agent): dedupe artifacts by content hash, not size

Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only.

* feat(session): allow forking from an assistant answer

User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists.

* fix(session): resolve parent attachments from a forked chat

Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss.

* fix(agent): rewrite sandbox refs after a content-hash skip

Fork restores change mtime on unchanged files, so Collect returns empty
and named citations were left as sandbox: paths. Resolve those names
against session artifacts, notify only real uploads, and stamp restored
mtime only on the matching content hash.

* fix(session): drop a dead fork snapshot and retry checkpoint rewrite

Create from a missing template now abandons bootstrap so the next
Resolve uses a normal sandbox. A failed checkpoint rewrite no longer
marks the bootstrap consumed, which previously made nested forks
SANDBOX_REPLACED.

* fix(session): load forked parent attachments into the prompt

ResolveForPrompt used session-scoped lookup, so copied parent attachment
IDs passed Get but never reached the model. Use the same resolveInSession
path as preview.

* fix(chat): keep the user-message fork button visible

The control was hover-only, which hid it on touch. Always show it, and
drop the unused degrade tooltip.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 14:20:18 +08:00
lyingbug 7e53c31671
feat(browser): connect local Chrome with task controls and display modes (#3180)
* feat(browser): integrate BrowserSkill local task tabs and account pairing

* feat(browser): refine task routing, lifecycle, and display modes

* fix(browser): reclaim temporary task tabs and groups

* test(database): cover browser authorization migration schema
2026-09-11 17:59:27 +08:00
nullptr-error fbaff900a0
feat(chat): append messages to a running agent turn (#3123)
* feat(stream): persist live-run markers and per-run steer queues

Mid-run sends need a cross-replica pointer to the generating
assistant and a control-plane list that never hits the SSE stream.

* feat(agent): inject queued user messages at ReAct round boundaries

The engine drains the steer sink after each round, persists accepted
rows, and delays Done until after a loop-end inject so the turn can continue.

* feat(session): expose /steer and hand leftover messages to a follow-up turn

Owner-scoped steer routes queue or inject into the live run, single-flight
AgentQA via the live-run marker, and history replay keeps multiple user rows
inside one turn.

* feat(frontend): queue mid-run messages while an agent turn streams

Stop and Send stay visible together, the overlay follows server delivery,
and history splits the transcript around injected user bubbles.

* fix(frontend): type pending and client_id on SteerQueueItem

vue-tsc failed because the overlay sets those fields while POST /steer is in flight.

* fix(session): harden steer persist, handoff, and new_run

Keep SetLiveRun ahead of SSE and roll back orphan turns, make inject
persist idempotent, append-then-delete on rebind, cap overrun rounds,
and stop aborting a live stream when /steer returns new_run.

* fix(session): satisfy pre-push lint on steer paths

Wrap long lines, check discarded Error/Emit results, and keep
context as the first argument so the fork push hook can pass.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-09 16:43:33 +08:00
lyingbug 3d81b977f1
refactor(resource-access): unify scoped authorization and transfer recovery (#3097)
* refactor(resource-access): unify scoped authorization and transfer recovery

* fix(resource-access): close reparse and transfer review gaps

* fix: enforce explicit file bindings and recover move cleanup

* fix: align mutation admission and cleanup scope checks
2026-09-08 15:36:33 +08:00
wizardchen 7fbbaffb9f fix(observability): nest follow-up completions under the chat trace
POST /suggestions often wins the race to generate follow-ups and has no
Langfuse parent, so chat.completion auto-creates an orphan root. Stamp
the originating W3C traceparent on the assistant message and resume it
before the extra LLM call.
2026-08-31 19:42:32 +08:00
wizardchen e683008188 fix(chat): align live timestamps with persisted created_at
Echo the stored user/assistant times on the first stream event so the
UI stops showing a client-clock fallback that would jump after reload.
2026-08-24 17:04:48 +08:00
wizardchen 9ed6266264 fix(chat): 持久化快速问答的检索步骤,刷新后不再丢失时间轴
快速问答的检索步骤只存在于 SSE 事件流里,刷新后前端只能靠 knowledge_references
反推时间轴,于是「检索了但没命中」的那一轮回到历史时步骤全部消失,只剩一条
孤立的记忆行。

改为在快速问答模式下订阅 pipeline 的工具事件,把 query_understand /
knowledge_search / attachment_parsing / image_analysis 落到 agent_steps。只在收到
tool_result 时写入,避免被中断的步骤刷新后显示成已完成。

这些调用来自 pipeline 而非模型,ID 加 ragpipe- 前缀并在 Agent 历史回放里过滤掉,
否则会把模型没发起过、甚至没被提供的工具调用塞进请求协议。
2026-08-13 21:34:21 +08:00
T3L000 4e53aa9d77 修复:同步 knowledge-search 的 Swagger 路由 2026-08-13 18:08:58 +08:00
wizardchen 10cc694fed feat(memory): introduce long-term memory feature with i18n updates
- Added a new configuration option for agents to enable or disable long-term memory.
- Implemented UI elements in the Agent Editor for managing long-term memory settings.
- Updated internationalization files for English, Korean, Russian, and Chinese to include new labels and descriptions related to long-term memory.
- Enhanced backend logic to support memory retrieval and management based on user and agent preferences.
- Introduced a new tool for searching long-term memory, ensuring it respects the agent's memory settings.
- Added tests to verify the functionality of the new memory features and their integration with existing systems.
2026-08-13 18:00:02 +08:00
wizardchen ffcb10f953 feat(memory): add cross-session long-term memory
Persist extracted and explicit memories across sessions with topic
resolution, semantic recall, document affinity, consolidation, workspace
settings, and inbox/interests UI.
2026-08-13 18:00:02 +08:00
nullptr-error 5df788dc77
feat(sandbox) (#2301)
* cubesandbox support

* feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system

- Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read
  - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation
  - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination
  - sandbox_read: read sandbox file content with offset-based chunked reading
- Add Artifacts system:
  - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages
  - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend
  - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI
  - DB migration 000066_message_artifacts: add artifacts column to messages table
- Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access
- Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules
- Extend Agent Observe: include agent steps in Langfuse observability tracing
- Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries
- Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales
- Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc.

* feat(agent):add remote sandbox client abstract  & client errors

* feat(sandbox): implement Redis binding store with distributed lifecycle lock;

* feat(sandbox): add remote session lifecycle with ownership context

Implement remoteSessionLifecycle to coordinate persistent remote sandboxes
across multi-instance deployments. Supports Resolve (lazy provision,
recovery, race-safe create) and Destroy with compare-delete semantics.
Add ownership context isolation to decouple remote resource cleanup from
request cancellation.

* feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient

Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient
implementation. Error normalization, timeout mapping, and metadata
support are fully integrated. SDK details remain private to the
adapter.

* refactor(sandbox): decouple SessionBoundManager from Cube

Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add
RemoteSandbox as provider-neutral Sandbox interface implementation.
Rewrite SessionBoundManager to depend on RemoteSandboxClient +
SessionSandboxBindingStore + SessionExistenceChecker, with all
lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed
lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry
with time.Time ModTime. Cleanup delegated to provider idle-timeout.

* refactor(agent): enhance session capability handling in sandbox manager

- Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution.
- Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends.
- Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types.
- Removed Cube-specific checks, promoting a more provider-neutral approach to session management.

* refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding

- Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation
  covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove)
- Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager)
- Dependency: matiasinsaurralde/go-e2b v0.1.0

- Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go,
  cube_defaults.go) and replace with direct *cubesandbox.Client usage
- Parse CubeProxy URL directly in adapter (parseProxyURL), removing the
  old hand-rolled envd transport layer
- Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra
- Target Cube deployment v0.5.1

- Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess,
  AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters
- Both adapters default to permissive (internet+public on) for backwards compat

- Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by
  WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE
- Add repositorySessionExistenceChecker — checks session existence via DB
  (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding)
- newSandboxManager() now accepts redisClient + sessionRepo; deprecated
  hardcoded NewMemorySessionSandboxBindingStore() in production path

- buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume
  (Cube and E2B). Sessions survive brief idle periods; providers still
  handle TTL-based reaping.

- buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized
  by RemoteProvider (was hardcoded to Cube Config fields)
- NewSessionBoundManager validates provider and delegates defaults to
  per-provider apply*Defaults helpers

- Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go,
  cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines)
- Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal,
  IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding
- Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds
- Dead ExecuteResult method: GetOutput
- Unused test: session_artifacts_integration_test.go
- Preloaded skill: sandbox-test-python

- Extract shared httpErrorKind() from per-adapter normalize functions
- Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore
  advertisement (E2B returns true post-reconnect, Cube always true)
- Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future"
- Bump session sandbox TTL defaults: 604800s → 300s (.env.example)
- Update docs/agent-skills.md: full multi-instance deployment guide

Total: 29 files, +1188 / -3721

* feat(agent): bind collected skill artifacts to owning message in resource registry

When ArtifactCollector persists a skill-generated file it now registers a
resource:// binding (owner_type=message, owner_id=assistantMessageID,
relation=artifact) in addition to the messages.artifacts JSONB record. This
mirrors how knowledge uploads and chat attachments bind their resources, so
artifacts can be enumerated and garbage-collected through the resource
registry instead of only via the message JSONB column.

- ArtifactCollector: inject ResourceCatalog, add messageID to Collect/
  maybePersist, and add best-effort bindArtifactResource (binding failures are
  logged, never drop the artifact; only resource:// refs are bound).
- agent_stream_handler: pass assistantMessageID into Collect.
- Renumber message_artifacts migration 000066 -> 000077 to resolve the
  version collision with existing migrations (upstream now uses 000075/000076).
- Tests: update collector signatures and add coverage for the binding path
  and for graceful degradation when Bind fails.

Note: binding applies to newly generated artifacts only; existing rows are
not backfilled.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(sandbox): harden remote sandbox security and artifact collection

Disable Cube local fallback so skill scripts never run on the host when
the remote provider is unavailable. Require Redis for cube/e2b binding
unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev.

Collect artifacts with WithoutCancel so user stop does not drop files,
reject shell_exec work_dir outside /workspace, and refuse session-scoped
Execute when a local fallback would be engaged.

* fix(ci): gofmt sandbox files and resolve artifact drawer type error

Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer
props in AgentStreamDisplay so session-id is always a string.

* fix(sandbox): adapt to upstream main after rebase

Renumber the message_artifacts migration 000077 -> 000079: upstream now
uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata.

Declare empty model-handle policies for shell_exec / list_sandbox_files /
read_sandbox_file. Upstream added a coverage test requiring every
UI-exposed tool to declare one; sandbox output is arbitrary skill-authored
content, so it gets the same treatment as execute_skill_script.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant.

* feat(sandbox): add sandbox config entity, cordon lease and session pin field

* feat(sandbox): add sandbox config repository with tenant scoping and cordon lease

* feat(sandbox): stamp sandbox config ID onto sandbox metadata

* feat(sandbox): scope sandbox listing, release and reaping to a single config

* feat(sandbox): resolve managers per config and refuse cordoned or missing configs

* feat(sandbox): implement session pinning and config identity change detection

- Add SessionSandboxPinner with read/pin/clear to bind session to config
- Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config
- Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL)
- Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts
- Fix artifact collection for default backend and destroy for legacy empty-pin sessions

* feat(sandbox): add multi-tenant sandbox config management and HTTP APIs

Service layer:
- Add sandbox config business service supporting CRUD and inventory listing
- For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions
- For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side,
  and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting
- Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config
- Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation

HTTP interfaces and routing:
- Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory)
- Mask sensitive credentials (API key, etc.) in all read responses
- Return 409 with structured inventory details on rejection scenarios
- sandbox-check probing supports specifying an existing config_id directly
- Remove old handler file, consolidate validation logic into service layer
- DI wiring and route registration for all six endpoints

* feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config

- Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults
- Validate config existence on create/update; return 400 if not found
- Inject sandboxConfigService into CustomAgentHandler
- Add unit tests covering field serialization and validation logic

* feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy

Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default.

* feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch

- Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios.
- Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned.
- Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace.

* feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities

- Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked
- Add named backend type validation, only cube/e2b can be stored as workspace named config
- Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default
- Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents
- Adapt artifact collector and other callers to fill missing resolve signature parameter

* fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube

Session teardown & deletion ordering
- Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins.
- Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions.

## Config update edge cases
- When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409.
- UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results.

Artifact collection for Cube skills
- Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`.

HTTP pool sharing for named Cube configs
- Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request.
- Add `CubeTransportPool` and `cubeSplitTransport`:
  - Control plane traffic reuses the same `SharedTransport` as E2B.
  - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header.
- Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes.

* feat(sandbox): sandbox execution DefaultUser change from root to user

* fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults

- Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars
  instead of DefaultConfig() hardcoded values, allowing empty fields to inherit
  from .env rather than Go constants
- Centralize deployment baseline parsing in sandbox/defaults.go with
  DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv
- Delegate container startup to shared helpers to avoid drift
- Propagate tenant env_vars through ResolveEffectiveConfig into
  buildSessionCreateRequest for both Cube and E2B session sandboxes

* fix(multi-sandbox-config-per-tenant): fix some bugs frontend.

* fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant

Three defects found while reviewing the remote sandbox work:

- The Cube connectivity probe and the deployment-default Cube/E2B managers
  built clients without a guarded transport, so an admin-supplied api_url or
  proxy_url could reach link-local addresses via DNS rebinding even though
  save-time validation passed. All three now dial through the same outbound
  guard the per-tenant resolver enforces.

- Attachment staging was gated on the process-wide manager, so a workspace
  pointing an agent at a named Cube/E2B config while the deployment default
  stayed docker/local would run its tools remotely against an empty
  /workspace/input. The gate now resolves the manager the session actually
  runs on.

- Session to sandbox bindings were keyed by the tenant in context, which a
  shared agent deliberately swaps for the agent owner's so models, KBs and
  named configs resolve in the sharing workspace. Session deletion only ever
  knows the session's own tenant, so teardown looked up a key nobody wrote
  and abandoned a paused MicroVM that keeps billing. The binding tenant is
  now carried separately and survives CloneContext.

* chore(sandbox): renumber migrations above upstream and de-collide test stub

Upstream main took 000079 (knowledge_folder_path) and 000080
(knowledge_base_auto_tag_config) while this branch was open, so the three
sandbox migrations move to 000081-000083. golang-migrate requires unique,
ordered versions, and the RAISE NOTICE labels move with the file names.

Main also added a handler test declaring stubSessionService in the same
package; the artifact download test's configurable variant is renamed rather
than merged, since the two stubs answer GetSession differently.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 21:28:20 +08:00
Xi(Darcy) Chen 090612da54
fix: pptx/ppt attachment parsing fails — missing parser engine + markitdown extras (#2621) 2026-08-09 16:23:36 +08:00
ljl e8882fa403 fix(docs): correct swagger @Router path annotations for agent-chat and knowledge-chat 2026-08-06 14:31:20 +08:00
wizardchen 39f4220c13 fix(wiki): ensure prompt language is always resolved
Wiki generation sometimes rendered "Write in ." because async workers
read empty locale payloads without falling back to context or defaults.
Unify language resolution helpers and apply them at enqueue and consume
time so every wiki and related async prompt gets a non-empty language.
2026-08-05 16:14:50 +08:00
wizardchen eb6ac0110c fix(api): close the resource_urls=public gaps found in review
Follow-up on resource_urls=public. Review turned up one authorization gap plus a
set of correctness problems in the streaming path.

Authorization:

  - Embed channels were documented as excluded, but nothing enforced it. The
    delegating handlers hand the visitor's own gin context to the shared
    handlers, so ?resource_urls=public — and a deployment-wide
    RESOURCE_URL_MODE=public — reached anonymous traffic and produced
    shareable, credential-free URLs that bypass the channel-scoped
    /embed/:channel_id/files proxy. ensureEmbedSession now pins the request
    context to handle mode, which covers every delegated endpoint including
    future ones. The downgrade is silent so a client that forwards the
    parameter keeps working.

  - A knowledge-base-restricted API key is refused with 403. Such a key is
    already denied /files because a raw storage path cannot be bound to its
    allow-list, so handing it anonymous file URLs would reopen that hole from
    the other side. The default handle mode stays available to it.

Streaming:

  - Rewriter.ref holds its lock across resolution. Resolver implementations keep
    an unsynchronised per-provider cache, so the previous unlocked window was a
    real data race for any caller sharing a StreamRewriter between goroutines —
    which its own doc comment invited. Serialising also collapses a concurrent
    duplicate into one signature.

  - Release the holdback buffer on every path that ends a stream while the
    client is still connected: a user-requested stop, an error event, and
    giving up on the event store. Only completion flushed before, so the tail
    generated just before a stop was silently dropped.

  - A released tail carries the metadata of the event it was cut from instead of
    a bare event_id.

  - The maxHeldBytes safety valve aligns to a UTF-8 boundary rather than cutting
    a character in half, and an unclosed "![…](" only holds back a
    whitespace-free destination under 2048 bytes, so prose that merely mentions
    Markdown image syntax no longer stalls the stream behind the buffer.

Also resolves the mode before running a knowledge search instead of after, so an
invalid value no longer costs a full retrieval, and warns once per distinct bad
RESOURCE_URL_MODE value instead of once per process.

Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
2026-08-05 14:42:51 +08:00
lyingbug ef1f0b36c4 docs(api): document resource_urls and RESOURCE_URL_MODE
Adds a cross-cutting "文件与图片引用" section to docs/api/README.md covering the
parameter, the deployment default, the endpoints it applies to, and the two
caveats that matter in practice: it needs APP_EXTERNAL_URL (or a publicly
reachable storage backend) to produce a link at all, and the links it produces
are time-limited but anonymously readable.

Also annotates the affected endpoints for Swagger and refreshes the generated
files for those parameters only, leaving unrelated drift in the committed
swagger output alone.

Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
2026-08-05 14:42:51 +08:00
lyingbug 877a6ebf9d feat(api): add resource_urls=public to return directly loadable file URLs
API responses reference stored files as opaque resource:// handles, so an
integrating app had to make a second authenticated call to /files for every
image before it could render anything.

Add an opt-in that resolves those references server-side into time-limited
HTTP(S) URLs, using the same mechanism the IM channels already rely on:

  - per request: ?resource_urls=public (default: handle, unchanged)
  - per deployment: RESOURCE_URL_MODE=public

Applied to the chat SSE endpoints (knowledge-chat, agent-chat,
continue-stream), message history load, and knowledge-search. Streamed answers
buffer a trailing incomplete reference so a handle split across two deltas is
still rewritten. References that cannot become an HTTP URL (for example local
storage with no APP_EXTERNAL_URL) stay handles, so clients keep the /files
fallback. Embed channels are deliberately excluded: their visitors are
anonymous.

SSE payloads share SearchResult pointers and metadata maps with the stream
replay buffer and the message being persisted, so those are rewritten as copies.

Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
2026-08-05 14:42:51 +08:00
wizardchen cc846d8e29 fix: harden shared agent source selector validation
Reject invalid agent_source_tenant_id instead of silently falling back, fix a resolveAgent nil panic when a shared selector fails, and add regression tests for wiki write-tool filtering and agent resolution.
2026-07-30 12:28:23 +08:00
wizardchen ab9f705843 fix: harden shared agent source permissions 2026-07-30 12:28:23 +08:00
wizardchen 29bec6c613 feat(session): implement strict owner scope for API-key sessions
Add GetOwnedSession method to enforce strict ownership checks for session mutations, ensuring tenant admins can read but not modify API-key sessions. Update relevant handlers to utilize this method for operations that alter session data, maintaining security and integrity of API-key sessions.
2026-07-20 19:31:02 +08:00
wizardchen 380e371c2e refactor: remove Neo4j-based conversation memory feature
Drop the memory pipeline, API fields, settings UI, and embed toggles so chat no longer depends on Neo4j graph storage for episodic recall.
2026-07-18 23:41:48 +08:00
wizardchen eedd7d4641 feat(agent): enhance suggested questions API with tag scopes
Update the SuggestedQuestionsRequest to include TagScopes for better filtering of knowledge items based on their parent knowledge bases. Modify related API documentation and frontend interfaces to accommodate the new structure. Ensure backend services handle the new tag scope logic correctly, including tests for the updated functionality.
2026-07-17 10:19:43 +08:00
wizardchen 85d10d99e3 feat(attachments): implement normalization and limits for temporary attachment IDs
Add functionality to normalize temporary attachment IDs, ensuring deduplication and enforcing a maximum limit of 5 attachments per message. Update related handlers and tests to support this new behavior, enhancing the robustness of attachment management in chat sessions.
2026-07-16 17:41:27 +08:00
wizardchen 9ae17be43c fix(chat): persist attachment content and fix fallback multi-turn context
Persist resolved temporary-attachment content onto the stored user message
so later turns can replay it in history. Close the retrieval progress spinner
on early pipeline exits, and reshape model-fallback messages with a proper
system role plus trailing user question.
2026-07-16 17:41:27 +08:00
wizardchen ceb4d1a310 feat(chat): add session-scoped temporary attachments with async parsing
Let users upload documents in chat without ingesting them into a knowledge base: files are parsed in the background, referenced by ID on send, and expire after a configurable TTL.
2026-07-16 17:41:27 +08:00
wizardchen 087fcc9fe7 feat(agent): add question suggestions with after-answer follow-ups
Enable agents to configure starter prompts and async follow-up questions after completed answers, with durable caching, analytics events, and chat/embed UI integration.
2026-07-14 11:17:25 +08:00
wizardchen d9bb7aa8f2 fix(search): harden tag scope resolution after #1907
Default missing KB metadata to document tag relation lookup, propagate tag resolution failures instead of returning empty results, and reject ambiguous bare tag_ids when multiple knowledge bases are in scope.
2026-07-08 10:55:42 +08:00
wizardchen 9a76a24ce6 refactor(api-keys): use role field and route-level API key guards
Replace scopes JSON with a viewer/contributor/admin role column, move
unsafe-method authorization from path-prefix middleware to declarative
APIKey* guards on routes, and close remaining KB-scope gaps in handlers.
2026-07-07 12:00:13 +08:00
wizardchen 505bc7ddec feat: add scoped tenant api keys 2026-07-07 12:00:13 +08:00
langcaiye 1708d6895a fix(search): apply tag filters to knowledge search 2026-07-06 23:30:33 +08:00
wizardchen b86efe3f92 fix(agent): harden @mention scope and simplify must_use hints
Tighten per-turn @MCP/@Skill/@tag handling after review: enforce agent
MCP/Skill whitelist, ignore mentions when selection mode is none, tag-scope
grep_chunks, and simplified English must_use lines without tool dumps.
Skip persisting scope envelopes to history rendered_content.
2026-06-29 18:09:44 +08:00
wizardchen 049f85139d feat(chat): add @Skill/@MCP mentions with scoped agent runtime
Wire per-turn @mentions for skills, MCP services, tags, and files through
QA handlers into agent config, runtime_context/must_use user blocks, and
tool registration. Fix tag-only scopes dropping KB tools, refresh suggested
questions on mention changes, and strengthen MCP must_use so @selected
services are invoked before local KB search.
2026-06-29 18:09:44 +08:00
wizardchen be26563f86 fix(agent): remove unused RunnableByViewer gate blocking viewer chat
The runnable_by_viewer field was never populated for built-in agents,
so TenantRoleViewer users always hit 403 on agent-chat. Drop the field
and RBAC check so agent-chat matches knowledge-chat access for viewers.
2026-06-23 18:21:23 +08:00
wizardchen 063809896a refactor(chat): improve handling of <think> tags in chat messages
- Updated logic to manage <think> and </think> tags in chat message content, ensuring accurate representation of thinking states.
- Introduced a new function to handle the closing of thinking events, enhancing clarity and reducing redundancy in event emissions.
- Adjusted event handling for reasoning content to support historical message reconstruction, improving user experience during chat interactions.
2026-06-05 14:56:48 +08:00
wizardchen 362bc7f6e5 feat(session): enhance stop watcher with timeout for improved resource management
- Introduced a constant `stopWatcherMaxDuration` to limit the lifetime of the stop watcher, preventing potential resource leaks in pathological stream scenarios.
- Updated the `startStopWatcher` function to use a context with a timeout, ensuring the watcher self-terminates after the specified duration.
- Refactored the handling of stop events to improve clarity and maintainability, ensuring the watcher exits appropriately on terminal stream events or user requests.
- Removed the `stopWatchCancel` from the `sseStreamContext` as the watcher now manages its own lifecycle more effectively.
2026-06-04 15:48:04 +08:00
wizardchen b6337cd970 fix(session): handle user-requested stop events gracefully in QA execution
- Updated the QA execution logic to differentiate between user-requested cancellations and actual errors, preventing unnecessary error events from being emitted when a stop is requested.
- Enhanced logging to provide clear feedback when a QA session is cancelled by the user, improving the user experience and reducing confusion.
- Added context cancellation checks to ensure proper handling of stopped generation scenarios, maintaining the integrity of the session flow.
2026-06-04 15:48:04 +08:00
wizardchen 9cce0c8e5e feat(system): consolidate system admin and settings into one Settings panel
Replace the standalone /platform/system/* routes with a single
"系统设置" section inside the canonical Settings modal. The previous
SystemLayout.vue / SystemAdmins.vue surfaces are removed and their
functionality (system admin roster, global settings) is hosted directly
in SystemSettings.vue under the standard `.section-header` /
`.settings-group` skeleton. Legacy URLs redirect to the modal section
so external bookmarks don't 404.

Backend:
- SystemSettingService.Reset + repo.Delete: drop the DB override for a
  key so the 3-tier resolver falls back to ENV / built-in default.
  Idempotent (resetting a never-persisted key returns nil); emits an
  audit row only on real deletions, invalidates the local cache, and
  publishes to peers via the existing pubsub channel.
- TenantService.BulkSetStorageQuota: overwrite every tenant's
  storage_quota in one statement. Powers POST /system/admin/tenants/
  apply-default-storage-quota; bypasses the per-tenant whitelist on
  PUT /tenants/:id which intentionally forbids storage_quota edits.
- AuditAction{SystemAdminPromoted,SystemAdminRevoked} constants and
  emitAdminAudit() in SystemHandler — promote / revoke now leave an
  audit trail with TenantID=0 and {target_email,target_username,
  idempotent|changed} in details.
- SystemSetting.LastModifiedByName: derived per-request display label
  (username, email fallback) so the UI shows "wizardchen" instead of
  a UUID prefix without storing a denormalised column.

Frontend:
- SystemSettings.vue rewritten against the Settings modal skeleton with
  auto-persisting controls (switch/select @change, input/inputnumber
  @blur, tag-input + per-delta popconfirm for SSRF whitelist and admin
  roster). auth.registration_mode change goes through an inline
  popconfirm; cancel rolls back. Reset / bulk-apply also inline
  popconfirms — no dialog modal for these per-row affordances.
- Priority hint panel surfaces the DB > ENV > default resolver order
  so operators can reason about "I set the env but it doesn't show up".
- Router: /platform/system, /platform/system/settings, /platform/system/
  admins are now compatibility redirects to /platform/settings?section=
  system-global.
- Settings modal sized 900x700 → 1080x780 and content-wrapper 600 → 760
  so the wider tables (members, system settings) breathe; <1100px
  viewport still flexes to the screen.
- i18n: system.globalSettings.* keys for title / description / loading /
  empty / badges / priority hint / per-key labels / reset / bulkApply /
  admins (label, placeholder, save messages, popconfirm copy) across
  zh-CN, en-US, ko-KR, ru-RU.

Misc:
- internal/utils/filesize.go doc: clarify MAX_FILE_SIZE_MB is a
  deploy-time-only knob (nginx + docreader + frontend each cache the
  env at startup); a SystemAdmin UI override would mislead operators
  because nginx would still 413. Until all four layers can hot-reload
  the limit in lockstep, this stays env-only.
- internal/utils/security.go: SSRF whitelist parser/runtime now drives
  off SystemSettingService for live updates; ENV remains the fallback
  for never-overridden deployments.
2026-05-26 21:13:56 +08:00
wizardchen 47a183aa65 feat(system-admin): implement bootstrap for system admin promotion and enhance system settings management
- Added WEKNORA_BOOTSTRAP_SYSTEM_ADMIN_EMAIL environment variable to promote a specified user to system admin on startup.
- Introduced a new bootstrap process in `bootstrap.go` to handle the promotion logic.
- Updated `.env.example` to document the new environment variable and its behavior.
- Created new views for managing system administrators and system settings, including listing, promoting, and revoking admin privileges.
- Enhanced the frontend to reflect the new system admin features, including UI elements for admin management and settings configuration.
- Updated API interfaces to support system admin functionalities, ensuring proper data handling and user management.
2026-05-26 21:13:56 +08:00
Phuong Tran 0f57bf3ff8 fix (chat history): #1431 user's multilines query formatting was lost 2026-05-22 17:26:05 +08:00
wolfkill 5bdaf58d45 fix(session): scope wiki fixer to shared KB tenant 2026-05-21 11:19:19 +08:00