Commit Graph

34 Commits (22cbcdfa5789b33dbb8d47532c21b24da2fdf5aa)

Author SHA1 Message Date
nullptr-error 22cbcdfa57
feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549)
* feat(sandbox): resolve workspace paths through WorkspaceLayout

Keep remote sessions on the /workspace contract, while file, shell, and
artifact code follow the session layout instead of hardcoded paths.

* fix(sandbox): fail closed on host layout lookup and restore the shell blacklist

Treat host vs remote as an explicit origin, refuse inspect outside ReadRoots,
and skip collection when the output tree is missing or equals the project root.

* fix(sandbox): close leftover fail-open holes in the layout path

Keep host copy, collection, and prompt text from falling back to /workspace when a lookup fails or an adapter returns unclean roots.

* fix(sandbox): satisfy golangci-lint on the layout helpers

Wrap new copy and test lines to 120 columns, and add the exported comments revive requires.

* fix(sandbox): gofmt the read_file layout copy wrap

* feat(sandbox): add host as an OS-enforced backend type

Host is not a named/pinnable config: each command is a fresh local process, unlike Docker, Cube, and E2B.

* feat(session): persist Lite host workspace directory

Empty means an auto-allocated folder. Standard edition leaves the column unused so the shared session model still loads. SQLite also gains the tenant skill tables Lite shell_exec already expects.

* feat(localsandbox): adapt the host OS sandbox to session interfaces

File tools stay outside Seatbelt and are bounded by PathGuard. A revoked project directory fails closed instead of allocating a replacement workspace.

* feat(sandbox): run unpinned Lite sessions on the host backend

Lite opts into host explicitly; web stays disabled when no named remote config exists. Host workspaces are not destroyed or auto-committed.

* feat(desktop): pick and persist host project directories

The native folder dialog is the authorization path; typed paths are rejected. settings.sandbox.host tells the client when this edition can use it.

* fix(sandbox): fail closed on Lite host layout lookup misses

Unpinned host sessions were described as /workspace when pin read or
sandbox resolve failed. Prompts and attachment staging now share the
same fallback so Lite never pretends the remote tree exists.

* fix(frontend): keep settings.sandbox.host in sync with the backend keys

TestDeploymentCapabilityKeysMatchFrontend requires the lists to match. Host stays fail-closed like docker so a missed probe does not show Lite-only UI.

* fix(lint): satisfy golangci-lint on the host sandbox wiring

Check c.Error, wrap lines over 120 columns, put context first on versionsWorkspace, and document exported adapter methods.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
2026-09-22 19:35:03 +08:00
lyingbug 33c0333ec9
fix(chat): restore shared workspace images after stream completion (#3523)
* fix(chat): restore shared workspace images after stream completion

* test(chat): wrap shared image completion cases for lint

* fix(chat): wait for persisted completion and preserve streamed replies
2026-09-22 00:16:58 +08:00
lyingbug 093c16cbb3
fix(agent): end the turn when the completion cap truncates an answer (#3449)
* fix(agent): end the turn when the completion cap truncates an answer

A round that came back with finish_reason=length and no tool calls used to
fall through analyzeResponse as non-terminal. With no tool calls to run,
appendToolResults pushed the half-written answer back into the history as a
plain assistant message with nothing telling the model to continue, so the
next round rewrote the answer from the top, hit the same cap, and repeated
until the round budget ran out or the user cancelled.

Three guards, each independently load-bearing:

- analyzeResponse gains a truncation case: deliver the partial text and end
  the turn. A cap hit that produced no text at all has nothing to hand over
  and defers to the existing bounded empty-content retry. No continuation
  nudge is sent — it only grows the prompt with the discarded fragment, and
  assistant prefill, the reliable form, is unavailable on most
  OpenAI-compatible endpoints.
- A consecutive-length breaker stops the loop when truncation keeps landing
  inside tool-call arguments, the one shape that still legitimately loops.
- The stuck-loop detector now counts an identical lack of content. A model
  burning its whole budget on reasoning returns empty every round, which used
  to land in the else branch and reset the counter, so the guard could never
  fire on the case that needed it most.

The answer event and its Done marker carry a truncated flag through the SSE
metadata, and the chat UI shows the same style of notice it already uses for
fallback answers.

Fixes #3446

* fix(agent): treat unterminated reasoning as no answer and keep the truncation marker

Review follow-up on three ways the truncation path could still mislead.

An inline <think> block that the cap cut off never gets its closing tag, so
the pair regex left it whole and the round looked like it had produced an
answer. The truncation case then skipped the empty-content retry it exists
for and emitted the chain of thought as the final answer. StripThinkBlocks
now drops a dangling <think> through end of input, which is what the
streaming splitter already does with the same text, so the two agree on what
counts as an answer.

The truncation marker only rode on the SSE metadata, and a finished
conversation is rebuilt from agent_steps rather than replayed from stream
events. A reloaded, shared or re-opened turn therefore lost the notice and
its half-written answer looked complete. AgentStep carries the flag now — it
is a jsonb column, so no migration — and the frontend marks the rebuilt
answer event and message from it.

The consecutive-length guard ran before the round was recorded, so a final
round of "let me look that up" plus a truncated tool call handed that
preamble over as the final answer, with its tool calls never refused and the
preamble never retracted from the answer area. The guard now records the
round and fails its tool calls the way every other truncated round does, and
a response carrying tool calls never donates its text as the answer.
2026-09-20 13:17:50 +08:00
养乐多 e3ac49cace fix(agent): report failed tool executions as tool_result, not error
response_type=error was being emitted when a tool execution failed,
but that type is reserved for internal agent failures (LLM/pipeline
errors), which clients treat as a crashed run. A failed tool call is
still a tool result: both the chat SSE handler and the skill install
transcript now emit response_type=tool_result with success=false and
the error string in metadata, so clients render it as a failed tool
card and let the agent recover instead of treating the run as dead.

Frontend needs no change: useChatStreamHandler already resolves tool
outcome from the success metadata field, history replay rebuilds from
agent_steps, and the API playground stops failing a whole request on
a recoverable tool error.
2026-09-18 11:11:21 +08:00
nullptr-error 42e6163981
feat(session): fork conversations from a historical user message (#3304)
* feat(session): add fork schema and history queries

为会话分叉增加数据模型与迁移,并提供截至分叉点的消息查询以及会话内的 request_id 配对。

* feat(sandbox): add git, fork snapshots, and bootstrap hook

沙箱镜像安装 git,Docker 快照使用独立的 fork 命名空间,并在首次创建会话沙箱时接入可选 bootstrap。

* feat(session): checkpoint workspace at the end of each turn

每轮结束后对沙箱 /workspace 打 git checkpoint,并避免 fork 后因 mtime 刷新重复挂上产物、以及附件仍绑在父会话而无法还原。

* feat(session): fork conversations and reap orphan snapshots

支持从历史 user 消息分叉会话、首次打开时从快照 bootstrap,并在失败或删除未打开的分叉时回收孤儿快照。

* feat(chat): add session fork entry, prefill, and badges

在用户消息上提供分叉入口,进入新会话时预填原问题,并展示降级提示与来源角标。

* style: add missing newline so session_fork_delete_test.go is gofmt-clean

* fix(session): prune later git history when rolling a fork workspace back

reset --hard 只还原工作区,分叉点之后的 commit 仍可被 checkout。校验 SHA 后丢掉其他 ref 并 gc,避免错误基线。

* style(session): clear golangci-lint findings on the session-fork diff

Pre-push lints the whole PR against main; wrap long lines, check c.Error,
and fix gofumpt/revive/staticcheck so the merge can be pushed.

* fix(chat): show the fork button on hover without shifting the thread

Overlay the action so revealing it no longer expands the user bubble and janks the messages below.

* fix(chat): drop the fork landing banners

Keep the original question prefilled; do not explain sandbox snapshot vs checkpoint on the new thread.

* fix(agent): dedupe artifacts by content hash, not size

Same-path same-size rewrites were skipped as restores. Keep path+mtime as the cheap skip, hash bytes when mtime moves, and stamp the restored mtime onto this session only.

* feat(session): allow forking from an assistant answer

User forks still copy history before the question and prefill it. Assistant forks copy through that reply and reuse its checkpoint, with a degrade tooltip when none exists.

* fix(session): resolve parent attachments from a forked chat

Fork copies attachment IDs onto the new messages but leaves temporary_documents on the parent. Preview and Get now look up those IDs on the fork's copied history instead of a scoped miss.

* fix(agent): rewrite sandbox refs after a content-hash skip

Fork restores change mtime on unchanged files, so Collect returns empty
and named citations were left as sandbox: paths. Resolve those names
against session artifacts, notify only real uploads, and stamp restored
mtime only on the matching content hash.

* fix(session): drop a dead fork snapshot and retry checkpoint rewrite

Create from a missing template now abandons bootstrap so the next
Resolve uses a normal sandbox. A failed checkpoint rewrite no longer
marks the bootstrap consumed, which previously made nested forks
SANDBOX_REPLACED.

* fix(session): load forked parent attachments into the prompt

ResolveForPrompt used session-scoped lookup, so copied parent attachment
IDs passed Get but never reached the model. Use the same resolveInSession
path as preview.

* fix(chat): keep the user-message fork button visible

The control was hover-only, which hid it on touch. Always show it, and
drop the unused degrade tooltip.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-17 14:20:18 +08:00
wizardchen a799f23b26 feat(sandbox): stream live shell and install command output
Publish a bounded stdout/stderr tail for pending shell_exec and skill installs without putting partial logs into the model context.
2026-09-12 10:16:39 +08:00
nullptr-error fbaff900a0
feat(chat): append messages to a running agent turn (#3123)
* feat(stream): persist live-run markers and per-run steer queues

Mid-run sends need a cross-replica pointer to the generating
assistant and a control-plane list that never hits the SSE stream.

* feat(agent): inject queued user messages at ReAct round boundaries

The engine drains the steer sink after each round, persists accepted
rows, and delays Done until after a loop-end inject so the turn can continue.

* feat(session): expose /steer and hand leftover messages to a follow-up turn

Owner-scoped steer routes queue or inject into the live run, single-flight
AgentQA via the live-run marker, and history replay keeps multiple user rows
inside one turn.

* feat(frontend): queue mid-run messages while an agent turn streams

Stop and Send stay visible together, the overlay follows server delivery,
and history splits the transcript around injected user bubbles.

* fix(frontend): type pending and client_id on SteerQueueItem

vue-tsc failed because the overlay sets those fields while POST /steer is in flight.

* fix(session): harden steer persist, handoff, and new_run

Keep SetLiveRun ahead of SSE and roll back orphan turns, make inject
persist idempotent, append-then-delete on rebind, cap overrun rounds,
and stop aborting a live stream when /steer returns new_run.

* fix(session): satisfy pre-push lint on steer paths

Wrap long lines, check discarded Error/Emit results, and keep
context as the first argument so the fork push hook can pass.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-09 16:43:33 +08:00
lyingbug d5cb842580
feat(agent): unify sandbox tools and drop host preloaded skills (#3033)
* feat(agent): unify sandbox tools and drop host preloaded skills

Read workspace and skill files through read_file, and run installed or staged skills through shell_exec(skill_name). Remove the unused skills/preloaded tree and keep generated-file previews stable across re-renders.

* fix(frontend): do not assume classList in markdown sanitize hooks

Node unit tests stub anchors without a DOM classList. Detect download cards from class/download attributes instead.
2026-09-05 16:45:59 +08:00
wizardchen b747cdb3f6 feat(agent): stream live +/- progress for sandbox write and edit
Large write/edit calls used to sit on a spinner until the full JSON
arrived. Emit path and running line counts from partial arguments so
the UI can update in place, without putting file bodies on the SSE wire.
2026-09-02 23:36:58 +08:00
wizardchen 4c0d42c4cb feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files
Refuse truncated tool calls, compact-and-retry once on overflow, and let
sandbox files be appended, paged, and batch-edited instead of rewritten.
Models can set context_window (default 200K) so compaction sizes itself
from the real window; agent max_iterations of -1 means unlimited.
2026-09-02 22:09:09 +08:00
lyingbug 270b64f6ec
feat(agent): preview skill artifacts and verify installs without running them (#2823)
* feat(agent): preview skill artifacts and verify installs without running them

Generated files open in the chat drawer with overlay/Esc returning to the
list, and install verification no longer executes guessed skill entry
points. A failed install can be retried from the saved archive.

* ui(chat): use a folder icon and corner count for skill artifacts

The eye and document glyphs read as preview or copy, and TDesign's
loading spinner sat beside the icon. A folder with a top-right count
matches the generated-files drawer without covering the toolbar action.

* feat(artifact): implement inline artifact references with previews

Add support for inline artifact references in Markdown, allowing generated files to be displayed as clickable cards. Implemented a new rendering mechanism for artifacts, enabling previews directly from the chat interface. Updated localization files for new inline preview hints and missing file messages. Introduced tests for artifact reference normalization and rendering logic.

* fix(client): document AgentResponseType constants for revive

Touching the const block made golangci-lint require comments on every
exported value. Match the rest of the SDK and describe artifacts_pending.

* fix(agent): wrap artifact-reference prompt lines for lll

Pre-push golangci-lint rejected the branch on three lines over 120
characters in the sandbox: reference guidance.

* fix(agent): scope HTML scripts and tighten install verification

Keep scripted HTML preview on skill artifacts only, recreate sandbox
workspace dirs after they are deleted, and treat nested skill files as
dependencies rather than first-party imports.

* refactor(artifact): enhance artifact reference handling and metadata

Updated artifact metadata to include a stable resource handle for better identification. Adjusted artifact reference resolution to support both handle and name forms, ensuring compatibility with existing references. Enhanced tests to validate the new handling logic and ensure proper rendering of artifacts in chat messages. Improved documentation for clarity on artifact reference formats and their usage in the system.
2026-08-27 10:30:32 +08:00
ochan.kwon 894cbec48a feat(agent): report and attribute per-turn LLM token usage
Accumulate token usage across every LLM round of an agent turn and
expose the aggregate everywhere a turn is observable:

- AgentState.TurnUsage accumulates each round's reported usage,
  including the final-answer synthesis call; TokenUsage.Accumulate
  keeps the cache counters as prompt subsets and recomputes the
  combined cache status (unsupported survives only while every call
  was itself unsupported).
- The agent completion event carries the aggregate, and the complete
  stream event exposes it both as data.usage and as a typed
  StreamEvent.Usage that buildStreamResponse promotes into the
  previously-unused StreamResponse.Usage wire field.
- The assistant message persists it as a nullable usage column on
  both migration targets (postgres 000085 jsonb, sqlite 000012 text),
  for web and IM channels alike, so history reads still carry the
  turn's cost after the live stream is gone.
- "[LLM Usage]" log lines gain ", session_id=..., principal=type:id"
  when the context carries them; calls outside sessions stay
  byte-identical.

Usage from failed retry attempts is not accumulated: the per-call
usage log remains the ground truth, while the turn aggregate serves
as the operational signal.
2026-08-24 10:36:19 +08:00
wizardchen ffcb10f953 feat(memory): add cross-session long-term memory
Persist extracted and explicit memories across sessions with topic
resolution, semantic recall, document affinity, consolidation, workspace
settings, and inbox/interests UI.
2026-08-13 18:00:02 +08:00
nullptr-error 5df788dc77
feat(sandbox) (#2301)
* cubesandbox support

* feat(agent): add sandbox shell execution, file management tools, and Artifacts collection/download system

- Add three sandbox agent tools: shell_exec / sandbox_ls / sandbox_read
  - shell_exec: safely execute shell commands in session sandbox with timeout & output truncation
  - sandbox_ls: enumerate files under sandbox artifact output directory with recursive listing & pagination
  - sandbox_read: read sandbox file content with offset-based chunked reading
- Add Artifacts system:
  - ArtifactCollector: real-time collection of sandbox tool output files, attached to messages
  - /api/artifacts/{session_id}/{message_id}/{filename}: download endpoint for frontend
  - ChatArtifactsDrawer.vue: frontend artifacts sidebar UI
  - DB migration 000066_message_artifacts: add artifacts column to messages table
- Attachment sandbox staging: upload user attachments to session sandbox on QA request for skill tool access
- Update Agent Prompts: add shell/sandbox tool usage constraints and safety rules
- Extend Agent Observe: include agent steps in Langfuse observability tracing
- Refactor Sandbox SessionManager: support session-scoped file listing, reading, and status queries
- Frontend: render artifact events in AgentStreamDisplay, i18n updates for 4 locales
- Add comprehensive tests: shell_exec_test, sandbox_read_test, artifact_collector_test, etc.

* feat(agent):add remote sandbox client abstract  & client errors

* feat(sandbox): implement Redis binding store with distributed lifecycle lock;

* feat(sandbox): add remote session lifecycle with ownership context

Implement remoteSessionLifecycle to coordinate persistent remote sandboxes
across multi-instance deployments. Supports Resolve (lazy provision,
recovery, race-safe create) and Destroy with compare-delete semantics.
Add ownership context isolation to decouple remote resource cleanup from
request cancellation.

* feat(sandbox): add CubeRemoteClient adapter for RemoteSandboxClient

Implement CubeRemoteClient as a provider-neutral RemoteSandboxClient
implementation. Error normalization, timeout mapping, and metadata
support are fully integrated. SDK details remain private to the
adapter.

* refactor(sandbox): decouple SessionBoundManager from Cube

Remove internal/sandbox/cube.go (CubeSandbox, sync.Map reaper). Add
RemoteSandbox as provider-neutral Sandbox interface implementation.
Rewrite SessionBoundManager to depend on RemoteSandboxClient +
SessionSandboxBindingStore + SessionExistenceChecker, with all
lifecycle ops delegated to remoteSessionLifecycle (Redis + distributed
lock). Application-layer interfaces migrated to RemoteDirEntry/StatEntry
with time.Time ModTime. Cleanup delegated to provider idle-timeout.

* refactor(agent): enhance session capability handling in sandbox manager

- Introduced sessionFileStore and sessionShellExecutor functions to centralize capability checks for session-scoped file and shell execution.
- Updated agent service to register tools based on the presence of session filesystem capabilities, improving compatibility with various sandbox backends.
- Refactored tests to validate behavior when session capabilities are absent, ensuring robust handling of different sandbox types.
- Removed Cube-specific checks, promoting a more provider-neutral approach to session management.

* refactor(sandbox): add E2B backend, modernize Cube adapter, and enable multi-instance Redis binding

- Add e2b_remote_client.go (~850 lines) — full RemoteSandboxClient implementation
  covering lifecycle, execution, and filesystem ops (ListDir/Stat/MakeDir/Remove)
- Wire E2B into NewManagerFromType and container/bootstrap path (buildE2BManager)
- Dependency: matiasinsaurralde/go-e2b v0.1.0

- Cut ~1500 lines of legacy SDK wrapper (cube_client.go, cube_envd.go,
  cube_defaults.go) and replace with direct *cubesandbox.Client usage
- Parse CubeProxy URL directly in adapter (parseProxyURL), removing the
  old hand-rolled envd transport layer
- Pass lifecycle extra fields (onTimeout, autoResume) via SDK CreateOptions.Extra
- Target Cube deployment v0.5.1

- Add RemoteNetworkPolicy to RemoteCreateRequest with AllowInternetAccess,
  AllowPublicTraffic, AllowOut/DenyOut — shared by Cube and E2B adapters
- Both adapters default to permissive (internet+public on) for backwards compat

- Add selectSessionBindingStore() — Redis-backed or in-memory, controlled by
  WEKNORA_SANDBOX_REDIS_NAMESPACE / WEKNORA_REDIS_NAMESPACE
- Add repositorySessionExistenceChecker — checks session existence via DB
  (gorm.ErrRecordNotFound → not exists, transient errors → preserve binding)
- newSandboxManager() now accepts redisClient + sessionRepo; deprecated
  hardcoded NewMemorySessionSandboxBindingStore() in production path

- buildSessionCreateRequest default timeout action: Kill → Pause+AutoResume
  (Cube and E2B). Sessions survive brief idle periods; providers still
  handle TTL-based reaping.

- buildSessionCreateRequest() and effectiveHTTPTimeout() parameterized
  by RemoteProvider (was hardcoded to Cube Config fields)
- NewSessionBoundManager validates provider and delegates defaults to
  per-provider apply*Defaults helpers

- Legacy SDK wrapper: cube_client.go, cube_envd.go, cube_defaults.go,
  cube_envd_test.go, cube_net_diag_test.go, cube_test.go (~2500 lines)
- Unused helpers: DefaultManager.validateExecution, IsRemoteTerminal,
  IsRemoteUnsupported, IsRemoteTransient, ShouldPreserveRemoteBinding
- Dead Config fields: CubeEnvdPort, CubeIdleTTL, AllowedCmds
- Dead ExecuteResult method: GetOutput
- Unused test: session_artifacts_integration_test.go
- Preloaded skill: sandbox-test-python

- Extract shared httpErrorKind() from per-adapter normalize functions
- Add SupportsFilesystemEnumeration capability flag — gates SessionFileStore
  advertisement (E2B returns true post-reconnect, Cube always true)
- Fix stale comments: Cube-only → remote backends, remove "tomorrow"/"future"
- Bump session sandbox TTL defaults: 604800s → 300s (.env.example)
- Update docs/agent-skills.md: full multi-instance deployment guide

Total: 29 files, +1188 / -3721

* feat(agent): bind collected skill artifacts to owning message in resource registry

When ArtifactCollector persists a skill-generated file it now registers a
resource:// binding (owner_type=message, owner_id=assistantMessageID,
relation=artifact) in addition to the messages.artifacts JSONB record. This
mirrors how knowledge uploads and chat attachments bind their resources, so
artifacts can be enumerated and garbage-collected through the resource
registry instead of only via the message JSONB column.

- ArtifactCollector: inject ResourceCatalog, add messageID to Collect/
  maybePersist, and add best-effort bindArtifactResource (binding failures are
  logged, never drop the artifact; only resource:// refs are bound).
- agent_stream_handler: pass assistantMessageID into Collect.
- Renumber message_artifacts migration 000066 -> 000077 to resolve the
  version collision with existing migrations (upstream now uses 000075/000076).
- Tests: update collector signatures and add coverage for the binding path
  and for graceful degradation when Bind fails.

Note: binding applies to newly generated artifacts only; existing rows are
not backfilled.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(sandbox): harden remote sandbox security and artifact collection

Disable Cube local fallback so skill scripts never run on the host when
the remote provider is unavailable. Require Redis for cube/e2b binding
unless WEKNORA_SANDBOX_ALLOW_MEMORY_BINDING=true for single-node dev.

Collect artifacts with WithoutCancel so user stop does not drop files,
reject shell_exec work_dir outside /workspace, and refuse session-scoped
Execute when a local fallback would be engaged.

* fix(ci): gofmt sandbox files and resolve artifact drawer type error

Apply gofmt to PR-touched Go files and narrow ChatArtifactsDrawer
props in AgentStreamDisplay so session-id is always a string.

* fix(sandbox): adapt to upstream main after rebase

Renumber the message_artifacts migration 000077 -> 000079: upstream now
uses 000077_remove_wiki_log and 000078_chunk_editing_and_custom_metadata.

Declare empty model-handle policies for shell_exec / list_sandbox_files /
read_sandbox_file. Upstream added a coverage test requiring every
UI-exposed tool to declare one; sandbox output is arbitrary skill-authored
content, so it gets the same treatment as execute_skill_script.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(sandbox): replace single tenant sandbox config column with a configs table to support multi-sandbox config per tenant.

* feat(sandbox): add sandbox config entity, cordon lease and session pin field

* feat(sandbox): add sandbox config repository with tenant scoping and cordon lease

* feat(sandbox): stamp sandbox config ID onto sandbox metadata

* feat(sandbox): scope sandbox listing, release and reaping to a single config

* feat(sandbox): resolve managers per config and refuse cordoned or missing configs

* feat(sandbox): implement session pinning and config identity change detection

- Add SessionSandboxPinner with read/pin/clear to bind session to config
- Refactor four entry points (execute/attachment/artifact/destroy) to use pinned config
- Add config_identity.go to detect identity changes by effective values (APIURL/APIKey/Domain/ProxyURL)
- Add SandboxIdentityChanged with key-masked comparison to prevent false 409 conflicts
- Fix artifact collection for default backend and destroy for legacy empty-pin sessions

* feat(sandbox): add multi-tenant sandbox config management and HTTP APIs

Service layer:
- Add sandbox config business service supporting CRUD and inventory listing
- For non-identity changes (e.g., template, TTL), save directly without locking or remote interactions
- For identity changes (e.g., API key, endpoint), reject save if there are still live sandboxes under this config on the remote side,
  and return structured inventory details (sandbox count, affected sessions, related agents) for frontend prompting
- Before deleting a config, check remote live sandboxes; if occupied, reject and prompt user to end sessions or create a new config
- Identity comparison based on effective values to avoid normal saves being misidentified as credential rotation

HTTP interfaces and routing:
- Add REST endpoints for sandbox config (List/Create/Get/Update/Delete/Inventory)
- Mask sensitive credentials (API key, etc.) in all read responses
- Return 409 with structured inventory details on rejection scenarios
- sandbox-check probing supports specifying an existing config_id directly
- Remove old handler file, consolidate validation logic into service layer
- DI wiring and route registration for all six endpoints

* feat(sandbox): add sandbox_config_id field to allow agent referencing named sandbox config

- Add SandboxConfigID field to CustomAgentConfig with omitempty; empty value falls back to global defaults
- Validate config existence on create/update; return 400 if not found
- Inject sandboxConfigService into CustomAgentHandler
- Add unit tests covering field serialization and validation logic

* feat(sandbox, agent): Migrate sandbox config to multi-config CRUD, support named backend selection and workspace scripts policy

Migrate frontend API from legacy KV single-config to /sandbox-configs REST interface, supplement type definitions, detection methods and policy fields; refactor sandbox settings page to multi-config card grid, manage configs with cube/e2b tabs, add editor drawer for create/edit/delete, connection testing and occupancy hints, and add workspace scripts policy toggle; add sandbox config dropdown in Agent editor skill section, integrate config caching, and support selecting named config or falling back to deployment default.

* feat(sandbox): Unify sandbox disable semantics and add tenant-level policy master switch

- Backend: Unify interception in resolveTenantSandboxForConfig with priority: global disable > tenant disable > default fallback > named config; add deploymentSandboxDisabled() using env var to avoid mis‑impacting named E2B; update WorkspaceSandboxPolicy comment as master switch for entire workspace; add unit tests for various scenarios.
- Frontend: Sync i18n copy: button text changed to "Disable/Enable sandbox execution"; confirmation dialog clarifies impact scope; success/failure messages aligned.
- Semantic layering: WEKNORA_SANDBOX_MODE=disabled disables globally; tenant‑level "Disable sandbox execution" only disables for the current workspace.

* feat(sandbox): Add SSRF protection, Agent binding to named configs, and reference query capabilities

- Add outbound URL security guard to validate Cube/E2B API/Proxy addresses against SSRF (including cloud metadata), support allowing private endpoints via env var, link-local always blocked
- Add named backend type validation, only cube/e2b can be stored as workspace named config
- Add sandbox config ID field to Agent at runtime, resolve sandbox for attachments and execution based on Agent's selected named config instead of always global default
- Add query to list Agents referencing a sandbox config ID, useful for deletion/changes to show dependents
- Adapt artifact collector and other callers to fill missing resolve signature parameter

* fix(sandbox): fix session teardown and config-update, add HTTP pool sharing for Cube

Session teardown & deletion ordering
- Move sandbox destroy (and pin `Clear`) before soft‑delete operations (`DeleteSession`/`BatchDeleteSessions`/`DeleteAllSessions`) to prevent remote sandbox leaks and stale pins.
- Pass deployment‑wide `SessionBoundManager` as fallback into `resolveTenantSandboxForConfig` for sentinel pins (`"-"`) to support deployment‑default sessions.

## Config update edge cases
- When identity fields change but old credentials cannot build a client or list config sandboxes, log a warning and proceed with the update instead of returning `sandbox_inventory_unverifiable`; live sandboxes still return 409.
- UI: move `sandboxes_still_live` / `unverifiable` alerts to the top of the sandbox config drawer and auto‑scroll on save refusal and connectivity check results.

Artifact collection for Cube skills
- Pre‑create `/workspace/output` via envd with correct ownership/permissions for `DefaultSandboxExecUser` to ensure Cube skill artifacts are collected properly by `ArtifactCollector`.

HTTP pool sharing for named Cube configs
- Named sandbox configs rebuilt a client on every `Resolve`, causing fresh TCP connections to CubeAPI and the envd proxy on each request.
- Add `CubeTransportPool` and `cubeSplitTransport`:
  - Control plane traffic reuses the same `SharedTransport` as E2B.
  - Data plane traffic uses a guarded transport cached per proxy host:port, preserving the sandbox `Host` header.
- Wire the pool through the tenant resolver `buildClient`; keep SDK defaults for the deployment‑default manager and connectivity probes.

* feat(sandbox): sandbox execution DefaultUser change from root to user

* fix(multi-sandbox-config-per-tenant): align deployment config with tenant resolver and centralize defaults

- Merge form overrides onto DeploymentConfig() via WEKNORA_SANDBOX_* env vars
  instead of DefaultConfig() hardcoded values, allowing empty fields to inherit
  from .env rather than Go constants
- Centralize deployment baseline parsing in sandbox/defaults.go with
  DeploymentConfig, CubeConfigFromEnv, E2BConfigFromEnv
- Delegate container startup to shared helpers to avoid drift
- Propagate tenant env_vars through ResolveEffectiveConfig into
  buildSessionCreateRequest for both Cube and E2B session sandboxes

* fix(multi-sandbox-config-per-tenant): fix some bugs frontend.

* fix(sandbox): guard check-path dials, fix staging gate and shared-agent binding tenant

Three defects found while reviewing the remote sandbox work:

- The Cube connectivity probe and the deployment-default Cube/E2B managers
  built clients without a guarded transport, so an admin-supplied api_url or
  proxy_url could reach link-local addresses via DNS rebinding even though
  save-time validation passed. All three now dial through the same outbound
  guard the per-tenant resolver enforces.

- Attachment staging was gated on the process-wide manager, so a workspace
  pointing an agent at a named Cube/E2B config while the deployment default
  stayed docker/local would run its tools remotely against an empty
  /workspace/input. The gate now resolves the manager the session actually
  runs on.

- Session to sandbox bindings were keyed by the tenant in context, which a
  shared agent deliberately swaps for the agent owner's so models, KBs and
  named configs resolve in the sharing workspace. Session deletion only ever
  knows the session's own tenant, so teardown looked up a key nobody wrote
  and abandoned a paused MicroVM that keeps billing. The binding tenant is
  now carried separately and survives CloneContext.

* chore(sandbox): renumber migrations above upstream and de-collide test stub

Upstream main took 000079 (knowledge_folder_path) and 000080
(knowledge_base_auto_tag_config) while this branch was open, so the three
sandbox migrations move to 000081-000083. golang-migrate requires unique,
ordered versions, and the RAISE NOTICE labels move with the file names.

Main also added a handler test declaring stubSessionService in the same
package; the artifact download test's configurable variant is renamed rather
than merged, since the two stubs answer GetSession differently.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-10 21:28:20 +08:00
MidoriKurage 58ee0b7d38 refactor(handler): extract shared searchResultFromMap
Deduplicate the two inline SearchResult rebuilds (SSE references and
agent-stream references) into one searchResultFromMap helper.

Add a JSON round-trip test pinning all fields.
2026-08-10 19:34:02 +08:00
wizardchen 48b661134b fix(chunks): retain questions across content edits 2026-07-28 20:23:48 +08:00
wizardchen 6a439a4c3b fix(chunks): keep edited retrieval content consistent 2026-07-28 20:23:48 +08:00
wizardchen 9fd7a7f522 feat(mcp): prompt and resume OAuth authorization mid-conversation
When an agent invokes an OAuth-enabled MCP service that the current user has
not authorized yet, the tool call now pauses and surfaces an "Authorize" card
in the chat instead of failing with an opaque error. After the user completes
the per-user OAuth popup, the backend verifies a token exists and resumes the
paused tool call automatically.

Backend:
- Add mcp_oauth_required / mcp_oauth_resolved events and response types.
- Add OAuthPendingRequest + Gate.RequestOAuthAndWait, reusing the existing
  approval waiter, Resolve, Redis cross-instance fan-out and timeout.
- MCPTool.Execute pauses on an authorization-required error for OAuth services
  and retries once (with a fresh ctx and recycled connection) after the user
  authorizes.
- Forward the new events to the SSE stream.
- Add ResolveMCPOAuth handler + route; it verifies authorization before
  unblocking so a premature resolve cannot resume into another failure.

Frontend:
- Add McpOAuthCard with an Authorize button reusing the popup + status-poll
  flow, then resolve the paused run.
- Handle the new stream events (dedup multiple pending cards per service).
- Add resolveMCPOAuth API client and i18n keys (zh/en/ko/ru).
2026-06-29 23:22:28 +08:00
wizardchen 9e916839ef feat(backend): add RAG pipeline progress events
Emit query-understanding and knowledge-search progress over SSE, emit
references before stream completion, and wire progress into session QA.
2026-06-16 01:09:07 +08:00
wizardchen d38a6efd87 refactor(agent): remove final_answer tool references and update related logic
- Eliminated all instances of the final_answer tool from the agent's configuration and logic, transitioning to a model where the agent concludes responses with plain text instead.
- Updated event handling and response analysis to reflect this change, ensuring that the agent's output is now directly delivered as text without relying on a dedicated final answer tool.
- Adjusted localization strings across multiple languages to remove references to final_answer, enhancing clarity and consistency in user-facing messages.
- Improved the overall structure of the agent's response handling to streamline the process and reduce potential confusion regarding tool usage.

refactor(agent): streamline answer handling and preamble logic

- Updated the agent's response handling to treat any text streamed before a tool call as a preamble, ensuring it is not included in the final answer.
- Removed the superseded marker logic from the event emission process, simplifying the flow of answer content and enhancing clarity in the UI.
- Adjusted the event handling to ensure that preamble segments are correctly retracted and relocated into the steps tree, improving the overall user experience.
- Enhanced the agent's internal logic to better manage answer segments and their states during streaming, leading to a more coherent response structure.
2026-06-04 15:48:04 +08:00
wizardchen 55caeefc3d feat(observability): log end-to-end TTFB on both ends of chat stream
To diagnose where latency lives between "user hits send" and "first
token appears" we need a single number that can be matched across
the browser console and the server log. Add correlated TTFB markers
keyed by X-Request-ID:

Frontend (streame.ts)

* Generate the X-Request-ID once and reuse it for logging.
* Log request:start when fetchEventSource is invoked, response:headers
  when onopen fires, and response:first_answer the first time an SSE
  payload with response_type === 'answer' arrives. Filtering by event
  type avoids treating session_title / references / tool_call as the
  "first token".

Backend (session handler)

* parseQARequest records the wall-clock entry time and logs TTFB:start
  with the same X-Request-ID.
* qaRequestContext carries receivedAt through to the stream handler.
* AgentStreamHandler emits a one-shot TTFB:first_answer_chunk log the
  first time it observes an AgentFinalAnswerData chunk, so the delta
  against TTFB:start is the server's request-in → first-token-out
  budget.

The frontend delta minus the backend delta is then attributable to
network + gin middleware, which were previously invisible.
2026-05-12 21:16:37 +08:00
wizardchen 5510ea8f5a feat(agent): human-in-the-loop approval for MCP tool calls (#1173)
Add an opt-in human approval gate so Agent runs pause before executing
MCP tools that operators flag as dangerous, surface an approval card in
the chat UI, and only resume after the user approves (optionally with
edited args) or rejects.

Backend
- New mcp_tool_approvals table + repo/service to mark per-tool approval
  required (PG migration 000042 + sqlite init).
- approval.Gate coordinates RequestAndWait / Resolve with sync.Once
  delivery, configurable timeout, and Redis Pub/Sub fan-out so multi-
  replica deployments work without sticky sessions.
- MCPTool.Execute integrates the gate; uses a round-level ApprovalCtx
  (without the per-tool 60s timeout) for the wait, and re-derives a
  fresh 60s exec ctx after approval so CallTool keeps a full window.
- New SSE response types (tool_approval_required / _resolved) and
  EventBus events plumb approval state to AgentStreamDisplay.
- REST: list/set per-tool approval flag, resolve pending approval.
- Configurable via agent.tool_approval_timeout_seconds (yaml) or
  WEKNORA_AGENT_TOOL_APPROVAL_TIMEOUT env (accepts seconds or Go
  duration).

Frontend
- MCP settings: per-tool "require approval" switch on the test panel.
- Chat: ToolApprovalCard renders the pause point with editable JSON
  args, validation feedback, mm:ss countdown that turns warning/danger
  near deadline, and a resolved state that retains context.
- i18n strings added for zh-CN / en-US / ko-KR / ru-RU.

Docs
- docs/zh/mcp-approval.md covering behavior, config, API, deployment
  considerations (Redis cross-instance, restart limitations).
2026-05-10 22:57:12 +08:00
wizardchen 72dfb9ce75 feat(prompt): enhance context and intent classification in templates
- Updated context template to include runtime metadata such as current time and week, improving contextual awareness in user queries.
- Enhanced rewrite template with critical instructions for intent classification, ensuring that rewritten questions preserve essential entities and keywords.
- Refined intent classification logic to prioritize user intents more effectively, improving the accuracy of responses based on user queries.
- Added examples to clarify expected input and output formats for intent classification, enhancing usability for developers.
2026-03-26 18:57:12 +08:00
wizardchen 2b56c4fbac feat(agent): enhance streaming diagnostics and event tracking
- Added diagnostics for streaming events in the AgentEngine, including chunk counts and response type distributions.
- Implemented logging for emitted event types during the thinking process to aid in debugging and performance analysis.
- Updated the session handler to provide more informative warnings when fallback answers are emitted due to non-streaming behavior.
- Introduced a dedicated HTTP client for Ollama to improve connection handling and timeout management during streaming calls.
2026-03-25 22:08:29 +08:00
wizardchen 8f1237d64f feat(agent): enhance event emission for final answers and fallback handling
- Updated the AgentEngine to emit final answer events even when the LLM stops without calling the final_answer tool, ensuring the frontend can render responses correctly.
- Added fallback logic in the AgentStreamHandler to emit final answers as fallback events when no answer events are streamed, improving robustness in edge cases.
2026-03-19 18:20:09 +08:00
wizardchen a4b255a456 feat: implement final_answer tool and enhance agent duration tracking
- Introduced the final_answer tool to ensure agents submit their complete responses as the last action, improving response accuracy and consistency.
- Updated the agent engine to log detailed information about tool calls and responses, including the total duration of agent execution.
- Enhanced the chat message model to include agent_duration_ms, tracking the total execution time from query start to answer delivery.
- Implemented JSON field extraction for streaming responses, allowing for more efficient handling of tool call outputs.

These changes significantly improve the agent's response handling and ensure that final answers are consistently delivered, enhancing overall user experience.
2026-03-09 11:46:56 +08:00
wizardchen 44f38ca6c3 feat: add fallback response handling and UI indicators
- Introduced a new `is_fallback` property to track when responses are generated without knowledge base matches.
- Updated the chat components to display fallback hints and indicators in the UI, enhancing user awareness of fallback responses.
- Added translations for fallback hints in multiple languages, improving accessibility for diverse users.
- Modified backend logic to emit fallback status in response events, ensuring consistent handling across the application.

These changes improve the user experience by providing clear feedback on fallback responses, enhancing the overall interaction with the chat system.
2026-03-05 18:14:14 +08:00
wizardchen 397689d2f3 feat: introduce WeKnora Lite edition with lightweight configuration and deployment
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.

This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
2026-03-02 21:21:49 +08:00
wizardchen d954ba22af refactor: 转换 knowledgeRefs 为 References 类型 2025-12-31 15:36:20 +08:00
wizardchen de1ffa7f7c feat: 支持输入框内选择知识库和文件,优化选择交互体验 2025-12-22 13:11:31 +08:00
wizardchen ddf6b4e748 feat: Revamp conversation configuration management and enhance localization
- Removed outdated DEVELOPMENT.md, consolidating development instructions into relevant documentation.
- Updated conversation configuration to include new parameters such as max rounds, embedding thresholds, and rewrite prompts.
- Enhanced UI components to support new conversation settings, including AgentSettings.vue and related views.
- Improved localization in English, Russian, and Chinese for new conversation configuration fields and UI elements.
- Implemented logic to apply tenant-specific conversation settings, ensuring defaults are respected.
2025-11-18 23:18:43 +08:00
wizardchen 38beca1146 feat: Enhance event duration tracking and UI components in chat
- Added duration tracking for agent steps and tool calls, improving event data accuracy.
- Updated AgentStreamDisplay and ToolResultRenderer components to include fallback headers and improved output styling.
- Refactored duration handling in the agent engine and session management to ensure consistent duration reporting.
- Enhanced UI elements for better user experience, including scrollable output areas and improved tooltip visibility.
2025-11-12 14:27:00 +08:00
wizardchen a28c0f8b01 feat: Integrate web search configuration and enhance input field functionality
- Added logic to load and manage web search configuration in the InputField component, ensuring proper handling of web search states.
- Implemented user notifications for unconfigured web search settings, guiding users to the appropriate settings page.
- Enhanced the AgentStreamDisplay component to improve tooltip content rendering for knowledge base citations.
- Updated event data structures to include total duration in milliseconds for better precision in agent completion events.
- Refactored citation handling to ensure inline citations are displayed correctly, improving user experience in content rendering.
2025-11-12 01:18:12 +08:00
wizardchen 83721f98d4 feat: Integrate Swiper library and enhance chat functionality
- Added Swiper library to package dependencies for improved UI components.
- Updated chat API to include summary model ID in message handling, allowing for more flexible model selection.
- Enhanced InputField component to manage model selection and state more effectively.
- Improved event handling in chat components to support new model features and ensure accurate message processing.
- Refactored SVG assets for better visual consistency and updated styles for improved user experience.
2025-11-09 00:02:31 +08:00