Commit Graph

102 Commits (014f95f1394a7bfd1c7bbbdcdc19057a80dcaca5)

Author SHA1 Message Date
lyingbug 03d9d66a1f
feat(agent): size agent history by the window and calibrate token estimates (#3413)
* feat(agent): size agent history by the window and calibrate token estimates

Agent history was capped at history_turns (default 5) because compaction
could not survive a turn. With checkpoints persisted, load history by
tokens instead, and let compaction bound the session.

Budget. The loader drops the oldest turns that do not fit, and what it
drops is lost: neither replayed nor summarized. A budget equal to the
compaction threshold (the earlier attempt, since reverted) let the loader
trim first, so whenever one turn was larger than the system prompt plus
the new question the request never crossed the threshold; nothing was
summarized and the session became a sliding window with no checkpoint.
agent.HistoryTokenBudget is now the whole window: overflow reaches the
first round's compaction, which summarizes it into a new checkpoint.
A turn-by-turn regression test drives the real loader, budget and
compactor and fails as soon as a stored turn after the checkpoint is
missing from the loaded history; restoring the old budget fails it.

Summarizer input. History can now reach the window, so the part being
summarized can outgrow one request. An overflowing request degrades to a
raw archive that is never persisted, so every later turn would fail the
same way. Prepare keeps the newest messages that fit, the prompt states
how many were left out, and file paths still come from all of them.

Calibration. cl100k counts Chinese at about a token a character where
Qwen and DeepSeek spend about 0.6. The estimator now carries a scale
(clamped to 0.5-1.5, text only), learned from consecutive requests in a
turn: they share tools, system prompt and earlier messages, so the
provider's growth between them prices exactly the appended messages,
without mixing in how a provider renders tool schemas. The turn's scale
is stored on its usage (context_token_scale); the next turn's loader
prices history with it and hands it to the engine. Loading, the first
compaction check and compaction itself share one estimator, so they
agree; calibrating only the trigger would let the loader drop turns
again, which the regression test covers.

Also: history is priced as the engine sends it (redacted KB/Wiki results
cost one line); a checkpoint older than the rows read is matched by the
assistant rows' (created_at, id) order; a read ending on a turn's
question keeps that turn; a compaction discarded for freeing under 5%
still persists its checkpoint; a checkpoint write matching no row is an
error; a partial index backs the newest-checkpoint lookup. Agent mode no
longer reads history_turns: the field leaves AgentConfig, the editor
hides it in smart-reasoning mode, and quick-answer keeps its turn cap.

* fix(agent): address review of window-sized history loading

Summarization. The summarizer call had a fixed 60s total timeout. With
history loaded up to the window, a request could carry ~175k tokens and be
cut off while prefilling or writing normally; the degraded result left no
checkpoint, so the next turn reloaded the same history and failed the same
way. Summarization now streams and is cancelled only after no output for the
engine's stall timeout (120s by default, LLMCallTimeout overrides), with the
overall ceiling left to the transport, as for the engine's own rounds and as
in Codex and Pi. A stopped turn is not retried. When the summarizer still
fails, the raw archive is persisted as a checkpoint marked degraded so the
loop is broken, and the next compaction refines it as the previous summary.
The archive is held to the summary budget, newest messages first, so a
degraded compaction always frees room.

Loading. The loader priced turns as sent but returned and held them as
stored, so wiki pages stored in full and sent as one line could pile up in
memory. It now returns the as-sent history and releases each turn's stored
rows once replayed, keeping about a page of rows at a time.

Index. The partial checkpoint index is replaced by a composite
(session_id, created_at DESC, id DESC) index serving the backwards page,
the newest-checkpoint lookup and the recent-messages reads. It is built with
CREATE INDEX CONCURRENTLY so writes to messages are not blocked; the
migration holds that single statement, since a multi-statement query runs as
an implicit transaction. A PostgreSQL integration test, skipped without
WEKNORA_MIGRATION_TEST_POSTGRES_DSN, applies the chain, runs 000106 down and
up through golang-migrate, and checks both queries use the index without a
sort; the SQLite migration test checks the same plans.

Calibration. It no longer waits for a total token count, and a usage with
only a scale is persisted. A turn that measures nothing carries forward the
scale it started from, so the newest turn always has the latest one.
2026-09-18 22:51:31 +08:00
wizardchen 2a6e5dcbb2 fix(agent): allow keyword retry after search_knowledge rerank rejection
Repeating the same query in another mode is usually wasted, but keyword
mode can still recall identifiers that hybrid never surfaced. Stop telling
the model that switching mode cannot help.
2026-09-18 16:06:52 +08:00
wizardchen 62bb9da270 fix(agent): score search_knowledge reranks with the document title
The agent searched seven times for "Echo 开放权重模型 降低成本". Hybrid
retrieval ranked the right chunk first (RRF rank 0) every time, but the
rerank model saw only the chunk text, which never names Echo or cost;
it scored 0.0025 and all 18 candidates fell below the 0.15 fallback
floor. The tool reported "No matching chunks", and the model cycled
hybrid -> semantic -> keyword, which cannot help since every mode shares
the same rerank check. Scoring the same chunk with its document title
("Show HN: Echo - ... at 1/3 the cost using open-weight models") gives
0.45, and 0.55 for the query written as a question.

- Rerank passages are title + enriched chunk (rerankPassage); FAQ chunks
  keep their own question, and the pattern filter still sees chunk text
  only.
- When retrieval found candidates but the rerank model rejected all of
  them, the empty statement says so and that switching mode will not
  help, instead of "No matching chunks".
- The tool description, query schema and agent system prompt ask for a
  complete natural-language question in hybrid/semantic mode rather than
  a keyword list, and drop "switch mode" as the retry advice. Every
  keyword engine matches any token (OR + BM25), so a question costs the
  keyword leg no recall; on the same 20 bases the Echo chunks rank the
  same for both query styles.
2026-09-18 15:33:56 +08:00
lyingbug 92e7c0ab37
refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents (#3381)
* refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents

The agent's retrieval surface had grown to five overlapping tools whose
behaviour was scattered across long descriptions, interleaved "tips" in
tool output (most of which modelcontext discarded before the model saw
them), and a grep_chunks implementation that ran an unindexed regex scan
over the chunks table ordered by created_at, so large knowledge bases lost
recall and paid a full-table scan on every call.

Tool surface
- search_knowledge replaces knowledge_search and grep_chunks: one query,
  mode = hybrid | semantic | keyword, optional knowledge_base_ids, limit.
  Keyword mode runs through the retrieval engine's keyword index instead of
  the DB regex scan. Modes unavailable for the scope are refused with a
  reason. Rerank, MMR and de-duplication are kept; already_seen markers,
  retrieval_statistics and the zero-result "CRITICAL" block are gone.
- read_document replaces list_knowledge_chunks, get_document_info and
  wiki_read_source_doc: id accepts a dN or cN handle, always returns the
  document header, pages by exact offset (fixing the old floor-to-page
  bug), finds passages with query (literal by default, regex opt-in) and
  reads neighbours around a chunk with context.
- list_documents pages through one knowledge base, honouring @file/@tag
  scope.
- wiki_search takes a single query plus regex, knowledge_base_ids, limit;
  literal queries are escaped so the model never reasons about JSON
  escaping. Legacy queries / knowledge_base_id are still accepted.
- query_knowledge_graph is registered only when a graph-enabled KB is in
  scope and its description/tips were trimmed.

Compatibility
- Retired names in stored allowlists, presets and API calls are mapped to
  their successors by NormalizeAllowedTools at registration time; no data
  migration. Stored histories with old names still decode, render and
  redact. Calls to retired names return a replacement hint.
- modelcontext gains an "id" key space that decodes against both the chunk
  and document handle tables (decode-only), plus rendering for the
  document header, chunk roles, in-document matches and next_offset.

Prompts and surfaces
- The RAG, wiki researcher and hybrid templates now name the tools and the
  search -> read -> cite order; tool descriptions are short and contain no
  workflow instructions.
- Engine display names, history redaction, grounding prompt, IM rendering,
  the built-in MCP server (search_knowledge gains mode/limit, grep_chunks
  becomes keyword search, read_document gains query, wiki_search gains
  regex), frontend editor/renderers/i18n, presets, website docs and the
  changelog are updated accordingly.

Tests cover name mapping, mode validation and refusal, output shape,
read_document paging/context/query/scope, list_documents, wiki_search
escaping and legacy arguments, modelcontext decoding and rendering, and
registration-time aliasing and graph gating.

* fix(agent): address review findings on read_document, list_documents and model rendering

- read_document: fetch the second page whenever the first page did not
  reach the end of the document, so an unaligned window near the end no
  longer drops the trailing chunks (also fixes context windows around a
  chunk near the end); a query passed together with a cN handle now
  searches the owning document instead of being ignored; query mode stops
  collecting once the output budget is nearly spent and reports truncated;
  the document row is authorized in place instead of being fetched twice;
  the context read takes the total from the window fetch instead of a
  separate count query.
- list_documents: resolve the page's document scope in one batched tag
  lookup instead of one call per row.
- modelcontext: render read_document output once and read next_offset
  through the existing intValue helper.
- search_knowledge: point title/file-name lookups at list_documents.
- Tests for the boundary window, chunk context near the end, cN + query,
  and budget-bound query results.

* fix(agent): close compatibility gaps in the consolidated retrieval tools

- Wiki-only scopes: read_document and list_documents are registered
  whenever a wiki KB is in scope, since every KB stores chunks. Their
  capability requirement includes wiki but is marked auxiliary, so they do
  not widen a RAG agent's derived "all knowledge bases" filter to wiki-only
  bases (Go and frontend derivations both honour it).
- search_knowledge resolves the retrieval mode per knowledge base instead
  of refusing the call: FAQ and vector-only bases answer keyword requests
  semantically, keyword-only bases answer semantic requests by keyword.
  Results carry requested_mode and mode_fallbacks, the empty-result text
  names the fallback, and the call is refused only when no base in scope
  has a chunk index.
- The model rendering uses the mode the search actually ran with (it was
  always labelled semantic) and emits <mode_fallback> per degraded base.
- list_documents applies the @file/@tag scope before paging: tags are
  pushed into the database filter, pinned documents are loaded by ID, and
  total/next_page count only readable documents.
- read_document resolves chunk context by chunk_index through a new
  ChunkRepository.ListChunkNeighbors, so parent, summary and image chunks
  sharing the index sequence no longer shift the neighbourhood; offset is
  documented as a reading-order position, not a chunk index.
- wiki_search (agent and MCP) keeps its regular-expression default and
  only matches literally when the text is not a valid expression;
  regex=false forces a literal match and regex=true rejects invalid input.
- MCP grep_chunks keeps grep semantics: literal terms from the pattern
  fetch candidates from the keyword index (semantic for bases without
  one), and every returned chunk is verified against the pattern.

* docs(changelog): align the consolidation entry with regex wiki_search and grep_chunks semantics

* docs(mcp): note that grep_chunks recall is bounded by index candidates
2026-09-18 14:12:22 +08:00
lyingbug 5574ebc011
feat(knowledge): derive knowledge base descriptions from document profiles (#3370)
* feat(knowledge): derive knowledge base descriptions from document profiles

Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.

Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.

Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.

Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.

Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.

Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.

* fix(knowledge): keep generated descriptions out of duplicates and refresh clones

DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.

CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.

* fix(knowledge): refresh the loaded clone target and carry document profiles

ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.

cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
2026-09-17 22:21:55 +08:00
lyingbug 182f67bd86
refactor(modelcontext): consolidate source_refs parsing, fixed data_analysis table, and identifier leak report (#3372)
* refactor(wiki): consolidate source_refs parsing and resolution

source_refs entries ("uuid" or "uuid|title") were parsed by ten
independent implementations that disagreed on whitespace handling, on
whether a model-supplied title could be trusted, and on how a leading
pipe was treated. The scope-enforced and legacy agent resolvers also
duplicated the lookup and the routing step looked every document up a
second time.

- types.ParseWikiSourceRef / FormatWikiSourceRef / WikiSourceRefMatchesKnowledge
  are now the only readers and writer of the format.
- resolveWikiSourceDocuments replaces resolveAuthorizedSourceRefs,
  resolveSourceRefs and wikiKnowledgeBasesForSourceRefs: one lookup per
  document, titles always rebuilt from server data, dedupe by canonical ID,
  unknown documents fail closed in every mode.
- removeSourceRef now matches padded or titled refs of a deleted document.

* refactor(data_analysis): expose the selected document as a fixed table name

The tool required the document ID inside the SQL text and blindly
string-replaced it with the physical DuckDB table name, which forced the
model-context codec to rewrite handles inside quoted SQL and let the
UUID-derived k_<uuid> table name reach the model through stored schema
summaries.

The document is now selected only by knowledge_id. Every query runs on a
dedicated connection where "dataset" is a temporary view over the
physical table (DuckDB temporary objects are connection-local, so
concurrent sessions stay isolated). The physical name remains accepted
for internal callers; any other table fails validation with an error
naming the table the model should use. Schema summaries, the schema
tool, the analyst prompt and the tool schema all name the same table, so
data_analysis.sql no longer carries source handles and its text codec
policy is removed.

* feat(modelcontext): report raw identifiers that survive encoding

Text-level compaction exists because raw UUIDs reach model context through
producers the registry never sees: error strings, slugs, results of
unknown shape. Retiring it requires knowing where those producers are.

Registry.LeakedIdentifiers scans an encoded message list for UUID-shaped
values in every application-composed field (user messages are skipped),
classifies each as registered (codec missed a rewrite) or unregistered
(producer emitted a raw ID), and SummarizeLeaks renders one line per
field. The agent engine and the three RAG pipeline call sites log the
report as a warning after EncodeMessages.

* fix(chat_pipeline): align RAG data analysis with the fixed table contract

The RAG data-analysis plugin still asked the planning model to fill in
knowledge_id and to write SQL from a prompt that exposed the raw
knowledge UUID, then passed the model's JSON straight to Execute. Under
the new data_analysis contract a query that names anything but the
dataset table is rejected, and the plugin logged that rejection at error
level and moved on, so the analysis vanished without any signal. A
"no analysis needed" plan took the same error path.

The pipeline now owns document selection: the model returns only
needs_analysis and sql, the prompt names the dataset table and never
shows the knowledge ID or physical table, and the plugin builds the tool
input itself. A rejected query is fed back to the model once for repair;
a query that still fails is reported through pipelineWarn with the SQL
and the tool error, and "not needed" is logged as such instead of as a
failure.

Covered by unit tests with a scripted model and executor: not-needed,
pipeline-owned knowledge_id, one-shot repair, persistent failure
surfaced, unparseable plan rejected.
2026-09-17 22:08:16 +08:00
lyingbug 7a98a8e3de
feat(browser): polish connection settings, open tasks in a new window, and streamline prompts (#3193)
* feat(browser): enhance BrowserIcon component and update localization strings

- Updated BrowserIcon component to accept dynamic width and height props.
- Revised localization strings in English, Japanese, Korean, Russian, and Chinese for improved clarity and user guidance regarding browser connection and usage instructions.
- Enhanced BrowserConnectionSettings view with better structure and user feedback elements.

* Default BrowserSkill tasks to a separate window

* refactor(prompts): streamline agent system prompts and context templates

- Updated agent system prompts to enhance clarity and specificity in user interactions.
- Revised context templates to better separate metadata and user requests, improving overall structure.
- Adjusted fallback and intent prompts to ensure consistent language handling and response requirements.
- Introduced new localization strings for browser search instructions across multiple languages.
- Added tests for agent prompt serialization and hydration to ensure template integrity and functionality.

* chore: wrap prompt strings to satisfy golangci-lint

Keep the settings, separate-window, and prompt commits pushable together
without rewriting their history.

* fix(frontend): match agent editor save test to serialized payload

The prompt-inheritance save path still closes after update; the test
should assert that payload, not the unsanitized form object.

* feat(browser): implement browser connection management and related tests

- Introduced a new `browserConnection` store to manage the state of browser connections, including online status and device information.
- Updated the `Input-field` component to handle browser connection states, enabling/disabling features based on the connection status.
- Added tests for the `browserConnection` store to ensure correct behavior under various scenarios.
- Enhanced the `Input-field` and `BrowserConnectionSettings` components to provide user feedback based on the browser connection state.
- Adjusted related components and views to integrate the new browser connection logic, improving overall user experience.

* fix(frontend): update model-save assertion for serialized agent payload

The agent editor still persists the selected model ID via a spread of
formData; the source test should match that payload, not the raw form.
2026-09-11 21:28:58 +08:00
wizardchen 2d7df096ca fix(skills): pin install guidance and generalize runtime prompts 2026-09-09 23:46:59 +08:00
wizardchen a826a1f12c fix(skills): verify runtime prerequisites and support installer steering 2026-09-09 23:04:59 +08:00
GUO Zhaogong df7d871af0 fix(i18n): restore natural Japanese in proofread strings
Second pass over the strings touched by the proofreading commit,
reverting over-corrections that read worse than the original:

- Progress messages go back to the 〜しています… sentence form; the
  clipped 〜中… labels were unnatural in context
- Keep established katakana terms (プログレッシブエージェンティックRAG,
  エージェンティックRAG, Wikiレビュー, ネガティブ例, 生の出力) instead
  of kanji paraphrases; keep クエリ in the embed widget
- Remove commas placed inside noun phrases (〜のための、より〜) and fix
  stacked-modifier sentences (共有解除の確認文, 権限不足の説明文,
  並列実行数の説明)
- Restore sentence-form placeholders/toasts where the noun-only form
  sounded cut off (モデルに直接質問できます, お待ちください…)
2026-09-07 20:11:56 +08:00
GUO Zhaogong 071a87d5f7 fix(i18n): proofread ja-JP locale strings
Follow-up to #3080. A full proofreading pass over the Japanese locale
against the en-US and zh-CN sources, fixing mistranslations, awkward
phrasing and inconsistent terminology:

- Meaning: "organization" contexts now consistently say 共有スペース
  (not bare スペース); remove-from-space actions no longer read as
  deleting the member/agent/KB; built-in model description says
  "visible to" rather than "usable by" all workspaces; several
  descriptions (parser features, xlsx header option, runtime capacity,
  credentials invalidation) corrected against zh-CN
- Terminology: 改写 -> リライト, 机器人 -> ボット, 知识库 -> ナレッジベース
  (not ナレッジ), ReRank -> リランク, サインイン -> ログイン,
  制御プレーン -> コントロールプレーン, API-key capabilities -> 権限,
  "Agentic RAG" kept in English, self-hosted -> セルフホスト
- Wording: fix broken particles/係り受け (JWTに署名, 〜に失敗), drop
  unnatural potential forms (超えられません), align status chips
  (〜済み), progress messages use the 〜中... form, placeholders follow
  the English form, 未提供 -> なし, remove trailing ! on error toasts
- Conventions: keep ヘッダー / リンター with the long-vowel mark; remove
  remaining Latin/Japanese spaces (チャンク#{index}, unix://を)
- Prompt template and agent preset labels get the same fixes
2026-09-07 19:33:36 +08:00
GUO Zhaogong 6f0ed94a47
feat(i18n): add Japanese (ja-JP) locale support (#3080)
* feat(i18n): add Japanese (ja-JP) locale support

Add a complete Japanese UI locale for the admin frontend and the web
embed widget, and register it everywhere the supported-locale list is
enumerated.

- frontend/src/i18n/locales/ja-JP.ts: full bundle (5,600+ keys) aligned
  with en-US/zh-CN key structure; verified by the locale key audit
- frontend/src/i18n/embed.ts: ja-JP block for the embed widget, plus
  `ja*` normalisation in normalizeEmbedLocale
- Register ja-JP in i18n index, SUPPORTED_LOCALES, localeKeyAudit
  bundles/order, TDesign locale map (ja_JP), login language picker,
  general settings dropdown and embed default-locale options
- Add `language.jaJP` key to every locale so the settings dropdown can
  use $t like its siblings
- Accept ja-JP as an embed channel default_locale on the backend and as
  DEFAULT_LOCALE in the frontend docker entrypoint
- Extend locale-enumerating tests and update accepted-value lists in
  .env.example, public/config.js, widget script and website-docs

* feat(i18n): add ja-JP names and descriptions for prompt templates

Add a ja-JP block to the i18n map of all 32 prompt templates
(system_prompt, context_template, rewrite, fallback, intent_prompts,
agent_system_prompt) so the template picker shows Japanese labels when
the UI locale is ja-JP instead of falling back to the Chinese names.
Prompt content is unchanged.

* feat(i18n): add ja-JP strings for the CLI integration page

main gained the CLI setup page (#3081) with 18 new integrations.cli.* /
integrations.tabs.cli keys; add their Japanese translations so the
locale key audit stays green after rebasing onto main.

* style(i18n): unify ja-JP spacing and long-vowel conventions

- Remove spaces between Latin tokens and adjacent Japanese text
  (e.g. "MCP サービス" -> "MCPサービス"), matching the majority style
  already used in the bundle
- Drop the trailing long-vowel mark on common technical katakana terms
  (サーバ, ユーザ, フォルダ, プロバイダ, パラメータ, ヘッダ, クラスタ, ...)
  so the whole locale follows one convention

* feat(i18n): add ja-JP labels for agent type presets

config/agent_type_presets.yaml carried default / zh-CN / zh-TW labels
only, so the agent editor showed English preset names in the Japanese
UI. Add a ja-JP label and description to all five presets.
2026-09-07 19:20:41 +08:00
nullptr-error 524deeaad5
feat(sandbox): run sandboxed execs as root by default (#3067)
* feat(sandbox): run sandboxed execs as root by default

Every chat session owns its own sandbox, so the in-container account is
not a tenant boundary and root is the least surprising default for an
agent that installs packages and writes wherever it needs. Host and
cross-tenant isolation stay at the container boundary.

DefaultSandboxExecUser becomes "root" (remote_client.go) and the sandbox
image ends with USER root. The three backends keep naming the account
explicitly on every exec so the account a call lands on does not depend
on which backend a workspace selected; the adapter default, the file
operations and the artifact-directory bootstrap all follow the constant.
The template still provisions a uid 1000 "user" account for interop with
E2B/Cube tooling that addresses it by name.

Root also makes the skill-install flow's account hand-over pointless, so
it goes: verification no longer runs after a chown/chmod lock-down and a
repair round no longer reopens the tree (normalizeSkillPermissions and
reopenSkillDirForRepair are gone), and the prepare and scratch commands
stop chowning. The scratch wipe now also clears /tmp, and the cache
budget guard sizes only root's caches, since no second account can grow
one. Verification is asserted to stay on the ordinary executor (no
AsRoot, no AllowSkillsRoot) rather than on which account it ran as.

* refactor(skills): install chat-time extras into the skill's own env

PYTHONPATH / NODE_PATH pointed at a per-session overlay
(/workspace/.skill-packages/<skill>) because the skill tree was frozen
after install: root-owned, mode 555, and often built by `uv venv` with no
pip, so `pip install --target` somewhere else was the only way a chat
session could add a package.

None of that holds now. The lock-down is gone and execs run as root, so a
missing package goes straight into the skill's own environment, and the
write is already session-local: a chat sandbox is built from the config's
image and its filesystem dies with the session, so nothing reaches the
image other sessions start from. The overlay bought nothing over that
while splitting one skill's packages across two locations.

SessionSkillPackageDir goes, and applySessionPackagePath becomes
applySkillNodePath: it appends the skill's own node_modules after
whatever the caller supplied, replacing a `+=` that prepended a bare
separator once the overlay stopped seeding the variable. No PYTHONPATH is
injected at all. Python dependencies are reached through the skill's venv
interpreter, which the shell wrapper already puts first on PATH; the
overlay-era value named the venv root, which is not an importable path.
PYTHONPATH stays on the injected-name blacklist so a stored value cannot
shadow the venv.

The runtime hints and every prompt that described the overlay now name
`<skillDir>/.venv/bin/python -m pip install` and `npm install` in the
skill directory. frozenSkillTreeGuidance / isFrozenSkillVenvFailure are
renamed for what they detect now that nothing is frozen.

* docs(prompts): reserve /workspace/output for finished deliverables

/workspace/output is the one directory an artifact collector reads, so
everything written there is offered to the user as a deliverable. The
prompts described it as the place for "deliverables" or "downloadable
artifacts" without saying it is exclusively that, which reads as "the
directory you write results to" — and a model doing multi-step work has
results at every step. Drafts, intermediate renders and debug dumps land
next to the finished file and the user is handed all of them.

The four places that describe the workspace now say the same thing: the
collector reads only /workspace/output, so finished deliverables go there
and everything in progress goes anywhere else under /workspace. The
attachment block gains the matching half for the other direction —
/workspace/input holds the user's own files and is not ours to write to,
which it previously implied by calling the paths "read-only inputs" while
the tools that enforce it say so plainly.

Only the /workspace/input half is enforced (write_sandbox_file refuses
that tree). Whether a file is finished is a judgement no tool can make,
so the output half stays guidance and is stated wherever the model is
already reading about the workspace rather than in one place it might
not see.

* fix(skills): restore executable bundles and package recovery

* style(sandbox): wrap prompt text for repository lint checks

* fix(sandbox): guard recovery paths and document root execution

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-07 13:03:35 +08:00
lyingbug d5cb842580
feat(agent): unify sandbox tools and drop host preloaded skills (#3033)
* feat(agent): unify sandbox tools and drop host preloaded skills

Read workspace and skill files through read_file, and run installed or staged skills through shell_exec(skill_name). Remove the unused skills/preloaded tree and keep generated-file previews stable across re-renders.

* fix(frontend): do not assume classList in markdown sanitize hooks

Node unit tests stub anchors without a DOM classList. Detect download cards from class/download attributes instead.
2026-09-05 16:45:59 +08:00
wizardchen 22489ba663 feat(chat): keep prompt prefixes stable and send provider cache markers
Provider prefix caches only hit when system/tools stay byte-stable and
cache routing is actually on the request. Stop rewriting those prefixes
mid-turn, attach prompt_cache_key / cache_control, and log hit rate
instead of showing it in the conversation tree.
2026-09-03 00:45:52 +08:00
wizardchen 4c0d42c4cb feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files
Refuse truncated tool calls, compact-and-retry once on overflow, and let
sandbox files be appended, paged, and batch-edited instead of rewritten.
Models can set context_window (default 200K) so compaction sizes itself
from the real window; agent max_iterations of -1 means unlimited.
2026-09-02 22:09:09 +08:00
wizardchen 439443e1e1 feat(agent): tell session agents which workspace they are working in
The installer agent was recently given its working directory up front and
stopped prefixing `cd <skill-dir> &&` onto every command. Session agents
never got the same treatment: shell_exec only mentioned /workspace as the
work_dir default, so the model could not tell it was already there, and
execute_skill_script never said scripts run with /workspace as their cwd —
a relative path inside a skill script silently resolves outside the skill.

Describe the workspace once, where each tool is described: shell_exec gains
a Working Directory section (already in /workspace, relative paths, no `cd`
prefix, and each call starts there again so a `cd` does not carry over),
execute_skill_script states its cwd and points at $WEKNORA_SKILL_DIR, and
the skills prompt gains a Workspace section covering input, output and the
per-session package overlay.

Also drop the installer prompt's paragraph about write_sandbox_file and
friends: those tools are not registered in install mode, so the model never
sees them in its schema and the note only spent tokens describing — and
misdescribing — a scope it could not reach.
2026-09-02 16:16:52 +08:00
wizardchen dd647fefad refactor(skills): introduce write_skill_file and edit_skill_file tools for skill directory management
- Added write_skill_file and edit_skill_file tools to facilitate writing and editing files directly within the skill directory during installation.
- Updated the agent service to register these tools exclusively for the built-in skill installer, ensuring they operate within the correct scope.
- Enhanced error handling and validation to prevent unauthorized file access outside the designated skill directory.
- Improved documentation and prompts to clarify usage and limitations of the new tools, emphasizing the importance of proving import resolutions by executing them.
- Added tests to validate the functionality and security of the new file management tools.
2026-09-02 15:08:05 +08:00
nullptr-error 940799a74d
perf(skills): batch install verification and live installer progress (#2960)
* perf(skills): batch install verification and keep caches under budget

The tail of an install (agent_done -> verified) issued one remote exec per
check: two for permission normalisation, one per bundled script for the tree
check, three for the scratch wipe. A skill with twenty scripts paid thirty-
odd round trips, each carrying a hidden bootstrap exec and a distributed-
lock resolve, while the console sat on a finished agent with no feedback.

- normalizeSkillPermissions sends chmod and chown as one command.
- verifySkillTree folds the SKILL.md probe and one test per script into a
  single command that reports every missing file instead of stopping at the
  first. Exit codes 0/1/3 carry the verdict, stderr lines are the final
  messages, and the two existing error texts survive verbatim.
- cleanImageScratch merges the workspace reset, the directory restore and
  the cache pass into one command whose exit code is still owned by the
  strict workspace half. Package caches are no longer wiped wholesale: uv,
  npm and pnpm trim their own entries, and the pip/uv/npm/pnpm totals of
  both accounts are kept while under 256MB and wiped whole over it, so the
  next install resolves a previous download instead of the network.
- The install prompt hands the agent absolute paths for uv, npm, pnpm, pip,
  python3 and node from the same one-command probe that used to ask only
  whether uv exists, so it neither burns a turn on `which` nor gambles on
  the shell PATH.

Tests pin the new command shapes exactly, cover quoting of archive-chosen
file names, assert the tree check stays one command regardless of script
count, exercise the budget guard against a real shell, and keep the
workspace-restore failure semantics intact.

* fix(skills): credit a script's own sys.path bootstrap for shipped modules

The python check resolved every import against the script's directory, its
ancestors up to the skill root and the venv - and nothing else. A module the
skill ships in a sibling subtree (lib/image_video.py, imported by
scripts/generate.py) was invisible to that set, so a skill whose scripts
bootstrap their own library directory was refused for a dependency no
package provides, and the repair round that followed could not fix it
either: it installs packages and is forbidden from editing the tree. One
real install spent seven minutes discovering that, three LLM stream
timeouts included.

The verifier now scans what the tree ships and statically evaluates the
sys.path.insert/append calls each script raises before its imports - the
os.path and pathlib idioms, with variables tracked in source order. A
bootstrap proven to raise the directory that provides the module turns the
finding into a note; anything unprovable stays a refusal, now naming the
shipped file instead of pretending a dependency is missing. The refusal is
unrepairable on purpose: no installer round can fix a shipped module the
import cannot reach.

Tests pin the incident's exact shape, the pathlib idiom, and the guarantee
that makes the evaluator safe: a vendored module still cannot satisfy an
import unless the script's own bootstrap provably reaches it, and modules
pip placed under .venv never count as shipped.

* feat(skills): move the install progress bar while the installer works

The progress card sat at the seeded 35% for the whole agent phase and
only jumped at agent_done. Every tool call the transcript already sees
now advances an asymptotic curve 35 + 44·(1 − e^(−k/12)): monotonic for
any round count, converging below the agent_done anchor at 80, topping
out around 75 even when max_iterations is exhausted.

Each step publishes a one-line command summary (engine hint preferred,
80-rune cap) so the admin can see what the installer is doing. After
the first round ends the activity progress is muted — verification and
repair rounds own the stage anchors (80/82) from there, and a repair
round's tool calls must not drag the bar back under them.

* fix(skills): pin the installer agent's thinking mode off

installerAgentConfig left AgentConfig.Thinking nil, which most providers
read as "defer to the model's own default" — a thinking-capable model
would reason on every ReAct round of an install, burning latency and
completion tokens on what are short shell-command turns. Set an explicit
false so every provider path (Qwen enable_thinking, Ollama think, LKEAP
thinking type) receives a deterministic off instead of a default.

* style(skills): gofmt tenant_skill_verify_python_test.go

CI rejects aligned map keys in this table-driven test; gofmt drops the extra spaces.

* docs(skills): drop the duplicated exit 1 line in the python verifier

The shipped-module case was appended by repeating the whole exit 1 block
instead of extending the existing one.

---------

Co-authored-by: hailongzhao <hailongzhao@tencent.com>
2026-09-02 12:23:42 +08:00
wizardchen 4d78f9dc12 feat(agent): keep max completion tokens as default or a custom cap
Unset (0) now resolves at call time (2048 / 4096 / 24576 with sandbox) so the editor can offer Default vs Custom without silently rewriting a saved value.
2026-08-30 10:52:17 +08:00
wizardchen 21244d451a feat(agent): write workspace scripts and run them with installed skills
Generated PPT/report scripts overflowed the shell_exec command cap, and
session agents kept mutating the frozen skill tree or failing SkillHub
installs on display titles. Add write/edit sandbox tools, attach the
skill venv to /workspace scripts, and accept slug-based install names.
2026-08-29 22:45:18 +08:00
wizardchen 83d661af55 ui(sandbox): redesign personal sandbox secrets settings
Lead with skill credentials, hide empty inputs until edit, and rename the page to 沙箱密钥 so it is not mistaken for system env vars.
2026-08-27 16:55:09 +08:00
hailongzhao 67cdf26216 feat(sandbox): add tenant skill install and remove service layer
Introduce the hidden skill-installer agent and TenantSkillService so a
sandbox config can install or remove skills into snapshot images under a
per-config lock, without yet exposing HTTP or UI entry points.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-24 17:51:13 +08:00
wizardchen e88564a7db Remove redundant Wiki operation log 2026-07-28 11:18:33 +08:00
wizardchen b862c2e45d feat(agent): add per-agent citation output toggle
Move citation protocol injection to the system layer so custom prompts no
longer own citation syntax. Agents can disable inline citations while source
aliases still encode and decode tool arguments correctly.
2026-07-17 10:19:43 +08:00
wizardchen 014164ae44 feat(llm): introduce request-local source reference aliases
Replace raw chunk/document/KB IDs in LLM context with short cN/dN/bN/wN
aliases, decode tool arguments on the way in, and expand compact <ref/>
citations to the public <kb>/<web> format before user-facing output.
2026-07-17 10:19:43 +08:00
wizardchen 92ceb3bd7d refactor(chat): enhance follow-up handling for previous image/document context
Update the classification logic for follow-up questions to include reasoning about previously described images or documents. Ensure that follow-up queries referencing earlier attachments are correctly identified and processed without triggering unnecessary knowledge base searches. This change improves the model's ability to maintain context and respond accurately to user inquiries related to past interactions.
2026-07-16 17:41:27 +08:00
wizardchen fbc9db85d1 fix(chat): enforce retrieved image output in answers
When retrieved context or tool results contain Markdown images, dynamically
append output requirements to system prompts and final-answer instructions.
Also normalize fullwidth image parentheses in the frontend renderer so
localized model output still renders correctly.
2026-07-15 18:10:31 +08:00
wizardchen 951f95903b fix(chat): use Markdown for LLM image context and normalize legacy XML
Chat and agent tool context now emits answer-ready Markdown images instead
of internal <image> XML so copied content renders safely. The frontend also
normalizes legacy XML blocks in persisted conversations during streaming.
2026-07-15 18:10:31 +08:00
wizardchen b86efe3f92 fix(agent): harden @mention scope and simplify must_use hints
Tighten per-turn @MCP/@Skill/@tag handling after review: enforce agent
MCP/Skill whitelist, ignore mentions when selection mode is none, tag-scope
grep_chunks, and simplified English must_use lines without tool dumps.
Skip persisting scope envelopes to history rendered_content.
2026-06-29 18:09:44 +08:00
wizardchen 049f85139d feat(chat): add @Skill/@MCP mentions with scoped agent runtime
Wire per-turn @mentions for skills, MCP services, tags, and files through
QA handlers into agent config, runtime_context/must_use user blocks, and
tool registration. Fix tag-only scopes dropping KB tools, refresh suggested
questions on mention changes, and strengthen MCP must_use so @selected
services are invoked before local KB search.
2026-06-29 18:09:44 +08:00
wizardchen 4bb8906520 feat(faq): enhance FAQ handling and localization support
- Updated the agent system prompt to clarify mandatory deep read requirements for FAQ entries, ensuring accurate retrieval and response generation.
- Added new localization strings for FAQ-related labels in English and Chinese, improving user experience and consistency across the application.
- Refactored the document and grep results components to support distinct FAQ identification, enhancing clarity in the output.
- Introduced new tests for FAQ metadata handling, ensuring robust functionality and correctness in the FAQ processing logic.
2026-06-04 21:43:16 +08:00
wizardchen d38a6efd87 refactor(agent): remove final_answer tool references and update related logic
- Eliminated all instances of the final_answer tool from the agent's configuration and logic, transitioning to a model where the agent concludes responses with plain text instead.
- Updated event handling and response analysis to reflect this change, ensuring that the agent's output is now directly delivered as text without relying on a dedicated final answer tool.
- Adjusted localization strings across multiple languages to remove references to final_answer, enhancing clarity and consistency in user-facing messages.
- Improved the overall structure of the agent's response handling to streamline the process and reduce potential confusion regarding tool usage.

refactor(agent): streamline answer handling and preamble logic

- Updated the agent's response handling to treat any text streamed before a tool call as a preamble, ensuring it is not included in the final answer.
- Removed the superseded marker logic from the event emission process, simplifying the flow of answer content and enhancing clarity in the UI.
- Adjusted the event handling to ensure that preamble segments are correctly retracted and relocated into the steps tree, improving the overall user experience.
- Enhanced the agent's internal logic to better manage answer segments and their states during streaming, leading to a more coherent response structure.
2026-06-04 15:48:04 +08:00
jackson.jia d439b3ae07 feat(builtin-models): add YAML-based declarative config with ${ENV} interpolation
Allow built-in models to be declared in config/builtin_models.yaml
instead of inserting rows via SQL. On every startup the file is read
and each entry is UPSERT-ed into the models table (is_builtin=true)
by stable id.

Any string field may reference an environment variable with ${NAME}.
Unset variables are left as the literal placeholder so
misconfiguration surfaces clearly in provider calls rather than
failing silently with an empty token.

The file is optional: missing file, parse errors, and per-entry
upsert failures all log a warning without aborting startup.
docker-compose.yml adds env_file (.env, required:false) so
deployment-specific variables are passed through automatically.
2026-05-26 11:31:01 +08:00
wizardchen e9be53e830 fix: respect multi-turn-disabled flag in KnowledgeQA pipeline
When a custom agent has `MultiTurnEnabled=false`, `applyAgentOverridesToChatManage`
sets `chatManage.MaxRounds = 0` to signal "no history". Two pipeline plugins
mistreated this zero value as "use the global default" and silently re-loaded
session history into the LLM context:

- `PluginLoadHistory` fell back to `Conversation.MaxRounds` when
  `chatManage.MaxRounds == 0`.
- `PluginQueryUnderstand.loadHistory` had the same fallback, and even when
  `LOAD_HISTORY` was skipped it would re-populate `chatManage.History`,
  leaking previous turns into rewrite, image analysis, and the final answer.

The RAG branch in `session_knowledge_qa` also added `LOAD_HISTORY`
unconditionally, unlike the pure-chat branch which guarded it with `hasHistory`.

This change:

- Treats `chatManage.MaxRounds <= 0` as an explicit disable in both plugins;
  no fallback to global config.
- Makes the RAG pipeline consistent with the pure-chat path by gating
  `LOAD_HISTORY` on `hasHistory`.
- Removes the duplicated `Current Time: {{current_time}}` line from
  `agent_system_prompt.yaml`. The agent already receives a fresh
  `<runtime_context><current_time>` block with each turn from
  `observe.buildRuntimeContextBlock`, so the static placeholder was
  redundant.

The ReAct agent path (`session_agent_qa`) already checked `MultiTurnEnabled`
directly and is not affected.

Closes #1479
2026-05-26 10:45:42 +08:00
wizardchen c0e4a1d2f1 fix(summary): preserve image caption/OCR text in document summaries
Documents whose only payload is an embedded image (e.g. a docx with a
single picture) intermittently produced the refusal line "No textual
content was extractable from this document." even though the vision
model had successfully extracted a caption.

Three coordinated fixes:

- Clarify the summary prompt that text inside `<image_caption>` and
  `<image_ocr>` is first-class extracted content, not an image
  reference, so the model only triggers the empty-content branch when
  the body is genuinely textless.
- For image-dominated documents (real text < 200 runes after stripping
  image markup) include OCR alongside captions so screenshots and
  scanned figures contribute their actual content; text-heavy
  documents continue to use caption-only enrichment to avoid OCR
  noise from incidental figures.
- Add `EnrichContentCaptionAndOCR` which embeds caption + OCR text
  inline next to the original Markdown image link, deliberately
  omitting the `<image url=...>` and `<image_original>` wrapper
  blocks. Those wrappers carry only opaque export hashes that consume
  tokens and have been observed to retrigger the LLM's "image
  reference with no extracted text" heuristic.
2026-05-22 17:25:39 +08:00
wizardchen f3c7281f47 refactor(agent): simplify grep_chunks tool to a single regex query
The grep_chunks tool previously accepted an array of regex queries (1-5)
and an optional knowledge_base_ids filter and limit. In practice the LLM
either fired multiple near-duplicate calls or split synonyms across
entries instead of using POSIX alternation, and KB scoping plus result
limit are server-side concerns the model should not control.

Reshape the contract to match `grep -E -i` semantics:

- Schema accepts a single required `query` string. Combine concepts with
  `|` alternation in one regex instead of multiple calls.
- Drop `knowledge_base_ids` and `limit` from the schema; the tool now
  always searches the full agent scope and uses a fixed internal cap.
- Legacy `pattern`, `queries`, `patterns`, `max_results` keys are still
  accepted and joined into a single alternation regex so older callers
  and in-flight model outputs keep working.
- Update the agent system prompt template to document the new single
  `query` field.
- Frontend tool title now reads `query`/`queries`/`pattern`/`patterns`
  in that order so the search text is shown again under the new schema.
- Add a dedicated `grepSearch` / `grepSearchFailed` tool status (zh-CN,
  en-US, ko-KR, ru-RU) and rename the zh-CN tool label to "搜索关键词"
  so the UI no longer prefixes the call with a generic "调用 ..." label.
2026-05-20 19:34:27 +08:00
weitong a27f12a6cb 验证完毕 2026-05-15 20:47:01 +08:00
Claude 15550d14d4 fix(ingest): stop hallucinating summaries from filenames on empty PDFs
Scanned PDFs without a successful OCR pass (e.g. when no VLM is configured
on the KB, or when the VLM returned a "no text content" reply that the
sanitizer discarded) reach the summary and wiki-ingest pipelines with
nothing but markdown image references for content. The downstream LLM
prompts used to receive the filename, file type, and title alongside that
empty content, and the generate_summary template explicitly forbade the
model from refusing — so the model would invent a topic from the filename
alone, producing e.g. a "Canon scanner manual" summary for an MX5280.pdf
that is actually a scanned legal letter.

Two changes shut this down:

1. Drop filename / title / file-type from every LLM prompt input. The
   wiki summary, knowledge extract, candidate slug, and chunk citation
   prompts now see only the document content. The knowledge summary
   path no longer prepends the "Document Type: X / File Name: Y" intro.

2. Add a content-sufficiency guard in front of the LLM call. A new
   helper strips markdown image references and HTML image tags and
   refuses to invoke the model when fewer than 50 real-text runes
   remain. getSummary returns a sentinel error in that case and the
   caller marks the knowledge as SummaryStatusFailed instead of
   surfacing the first chunk's bare image reference. Wiki ingest skips
   LLM extraction with a clear log message.

The generate_summary.yaml prompt is also updated: the "NEVER output
refusal phrases" clause is removed and replaced with an explicit
empty-content rule that returns a deterministic placeholder line.

Real OCR (Tesseract / PaddleOCR) and filename-pattern sanitisation are
intentionally left for separate PRs.
2026-05-06 11:23:32 +08:00
IFeynman cf8e62fa9e fix(wiki): address PR review feedbacks regarding citation format, rule conflicts, and search loops 2026-05-04 23:52:38 +08:00
IFeynman b313da20ae feat(wiki): enforce strict citation tracing and ontology reuse in Wiki prompts 2026-05-04 23:52:38 +08:00
wizardchen d812770806 fix: inject KB document listing on fallback for broad queries (#959)
When users ask broad queries like "请整理知识库中的数据" in RAG mode,
vector/keyword search returns nothing because the query has no specific
content to match. The user needs to see what documents exist, not search
results.

- Add buildKBDocumentListing() to fetch document titles/filenames from
  the knowledge base and inject them into the fallback prompt via
  {{kb_documents}} placeholder
- Update fallback prompt templates (model_fallback, default_fallback_prompt)
  to include document listing context so the LLM can guide users
- Improve rewrite prompt: tighten summarize vs kb_search classification,
  expand kb_search to cover browse/organize/list operations, add examples

Closes #959
2026-04-28 23:31:04 +08:00
wizardchen 59fbf77524 refactor(agent_system_prompt): update synthesis and issue flagging instructions
Revised the agent system prompt to clarify the synthesis process and the handling of factual errors. The instructions now emphasize the mandatory use of the `final_answer` tool for submitting responses and the need to flag issues before calling `final_answer`. This enhances the clarity and consistency of the agent's operational guidelines.
2026-04-24 13:16:47 +08:00
wizardchen 4c906e0431 feat: Enhance agent tool compatibility and knowledge base filtering
- Introduced a unified capability requirements system for agent tools, ensuring consistent filtering of knowledge bases based on their capabilities.
- Implemented derived `kb_filter` logic to streamline compatibility checks between agent tools and knowledge bases, reducing redundancy in configuration.
- Updated frontend components to reflect new filtering logic, including improved handling of empty states and user feedback when no compatible knowledge bases are available.
- Enhanced internationalization support by adding relevant messages for tool compatibility issues across multiple languages.
2026-04-23 17:28:00 +08:00
wizardchen 50c1c70334 feat: Update agent type presets and prompt templates for improved knowledge base handling
- Changed `kb_selection_mode` in agent type presets from "selected" to "all" to allow agents to access all knowledge bases by default.
- Revised prompt templates to clarify the handling of bound knowledge bases, replacing the `{{knowledge_bases}}` placeholder with a reference to the `<bound_knowledge_bases>` block in the user message's `<runtime_context>`.
- Enhanced internal logic to ensure that only searchable knowledge bases are considered during retrieval, improving the efficiency of knowledge searches.
- Added new functions for better formatting and handling of knowledge base metadata in the runtime context.
- Updated documentation and comments for clarity on the changes made to knowledge base interactions.
2026-04-22 21:18:34 +08:00
wizardchen a8a3d9e694 feat: Enhance wiki extraction and configuration options
- Introduced extraction granularity settings in the wiki configuration, allowing users to control the level of detail in entity and concept extraction (focused, standard, exhaustive).
- Updated the UI to reflect these new options, including tooltips for better user guidance.
- Refactored the knowledge base editor to support the new extraction granularity feature, ensuring a seamless user experience.
- Improved the backend logic for wiki ingest tasks to utilize the new granularity settings, enhancing the accuracy of extracted content.
- Added tests to validate the new extraction granularity functionality and ensure robust performance across different configurations.
2026-04-22 21:18:32 +08:00
wizardchen 21862dce00 feat: Remove deprecated tools from built-in agents configuration
- Eliminated the "thinking" and "todo_write" tools from the allowed tools list in multiple sections of the `builtin_agents.yaml` file, streamlining the agent capabilities.
- This change enhances the clarity and relevance of the tools available to built-in agents, ensuring a more focused and efficient configuration.
2026-04-22 21:18:31 +08:00
wizardchen f8a04a0a93 feat: Update agent type for smart-reasoning mode in built-in agents configuration
- Changed the agent type from "custom" to "rag-qa" in the `builtin_agents.yaml` file, aligning with the new agent type presets introduced previously.
- This update enhances the configuration of built-in agents, ensuring they utilize the appropriate agent type for improved functionality and performance.

These changes contribute to a more coherent and effective agent system by standardizing agent types across configurations.
2026-04-22 21:18:21 +08:00
wizardchen b3eab5276c feat: Update agent system prompt and frontend citation handling for wiki sources
- Enhanced the agent system prompt to clarify citation formats for wiki and chunk sources, ensuring accurate inline citations.
- Introduced a new citation format section in the prompt, detailing the required syntax for both wiki pages and knowledge chunks.
- Updated the frontend to visually align wiki citations with knowledge base citations, improving user experience and consistency in citation presentation.
- Implemented truncation for long wiki labels in citations to maintain visual consistency in chat bubbles.

These changes improve the clarity and usability of citations in the agent system, enhancing the overall information retrieval process.
2026-04-22 21:18:21 +08:00
wizardchen 37a517612d feat: Enhance agent retrieval strategies and regex handling
- Updated the agent system prompt to emphasize the use of regex for chunk searches, improving retrieval accuracy and efficiency.
- Introduced a new `Capabilities` field in the `KnowledgeBaseInfo` struct to define the retrieval surfaces available for each knowledge base, guiding the agent's strategy selection.
- Enhanced the `grep_chunks` tool to support regex queries, allowing for more flexible and powerful text pattern matching in knowledge base chunks.
- Improved JSON handling in the `RepairJSON` function to address invalid escape sequences, particularly for regex patterns, ensuring robust parsing and error handling.
- Updated documentation and comments across various files to clarify usage and expectations for regex and knowledge base capabilities.

These changes significantly enhance the agent's ability to retrieve relevant information while ensuring the integrity of input data handling.
2026-04-22 21:18:21 +08:00