* feat(assistant): let the assistant check MCP services and retrieval
Add a probe risk for operations that change nothing but do real work
(connecting to a service, running a retrieval). Probes use POST, run
without approval at every access level, and are capped per run.
Open three operations on top of it:
- mcp_service.test: connect with the saved settings and report why not
- mcp_service.list_tools: list what a service offers before binding it
- knowledge_base.search: raw hybrid recall with thresholds, to explain
why a document is or is not found
x-agent can now name one method when a handler documents several, and
catalog parameters accept numbers.
* feat(assistant): let the assistant try an agent it set up
Add try_agent: the assistant asks an agent one question the way a user
would and gets back its answer, the tools it called and the documents it
cited, so it can check a setup before calling it done.
- The trial runs as the signed-in user in a new agent_trial session kind,
which ordinary listings already exclude and the API cannot create, and
the session is removed afterwards. Runs are non-interactive, so tools
that would ask the user are refused. Long-term memory is untouched: it
is written only by the HTTP handler.
- The assistant itself, the Wiki fixer and the skill installer cannot be
tried, nor any agent granted the operation catalog.
- A trial spends model calls and runs the agent's own tools, so it is a
write for the access level: confirm asks, auto_write runs, read_only
does not offer it. The confirmation card names the agent and question;
ConfirmedTool now lets a wrapped tool supply that text.
- At most 3 trials per turn, 4 minutes each; the tool timeout is raised
above that. Answers, tool errors and references are capped.
- The runner is bound to the session service after the container builds
it, since the session service depends on the agent service.
* feat(assistant): let the user allow similar changes for the conversation
The confirmation card for a recoverable change now offers "allow this for
the rest of the conversation". Once chosen, later calls of the same kind
in the same session run without asking, so a batch tidy-up no longer
needs a click per item.
- What a grant covers is derived on the server, never taken from the
model: the same catalog operation on the same knowledge bases (a
document's knowledge base comes from the target read) or the same
agent; the same Wiki tool on the run's Wiki knowledge bases; trials of
the same agent. Deletions and operations without such a scope, such as
creating a knowledge base, always ask.
- No new storage: the gate looks for an earlier approved prompt in the
session with the same grant and allow_in_session set, for the same
tenant and user. Grants therefore survive suspend/resume and replicas,
and end with the conversation. A failed lookup asks again.
- The response validator accepts allow_in_session only on approval and
only when the prompt offered a grant, so a client cannot widen it.
- The card shows the scope in the user's language. The model is told
such calls may run without a card and that deletions always ask.
- describeTarget now returns its target read so the grant reuses it
instead of reading again.
* feat(assistant): tell the assistant about settings pages and chat answers
Pages now report which page is open and, for the agent editor, which
section. The model and MCP settings name the model or service open for
editing, and chat answers gain an "ask the assistant" button that
attaches that answer. The backend resolves these for the assistant only:
models and MCP services in the caller's tenant without endpoints or
credentials, and messages through the caller's session read access, with
the question, an answer excerpt and cited sources quoted as data.
* feat(assistant): link the user to pages and filled-in forms
The assistant gains open_page, which shows a button to a knowledge base
or document, a Wiki page, an agent's editor at a tab, a model or MCP
service to edit, or a new model or MCP service with its form filled in.
Targets are read through the operation catalog as the user, so a link
never points at something they cannot open. Drafts take no credentials
and only http(s) endpoints without user info; the card shows the
endpoint, and the user enters keys and saves the form themselves.
* feat(assistant): manage MCP services, model names and skills
* fix(assistant): prefill model drafts on initial navigation
* feat(assistant): document tags, model catalog reads, clearer cards, live page refresh
- Operations project lists of objects; knowledge.get/list report tags.
- model.get shows provider, endpoint and capabilities; new
model.list_providers and model.get_provider read the model catalog,
including vendors an admin added.
- open_page and ask_user guidance: offer page buttons whenever the next
step is the user's, word choices for the user.
- Chat cards: redesigned open-page card pinned under the final answer,
ask-user inputs in the dock style, approval card with aligned fields,
translated labels and a question built from the operation summary.
- Open pages reload after assistant changes: deletes and moves wait for
the background task, Wiki edits now report their changes, KB list
counts, tags and chat model selectors refresh.
- The assistant panel stays open across reloads within the tab.
* fix(ci): format assistant operation code and wrap long lines
* feat(assistant): let the assistant see, fill and open the page beside it
- read_page / fill_page tools answered by the browser over a page_action prompt,
with refs, redaction, in-place text edits, undo card and a page-access switch
- composer shows the open page, dialog and selection; a selection bubble and an
editor toolbar entry ask the assistant about the selected text
- edits scroll into view with a highlight and an inline undo bar
- open_page reaches any settings section, toolbox tool and list page, and can
navigate there itself to read it, keeping the button when edits are unsaved
* feat(assistant): let the assistant open the tab it needs
A visible tab can be selected and re-read, and system settings can open straight to user management, so the next step is no longer blocked on the wrong tab.
* feat(assistant): expose read-only conversation operations
Add session.list, message.list, message.search and
message.chat_history_stats to the operation catalog, restrict the
sessions and messages areas to non-mutating operations, and search
messages with the raw query instead of the log-sanitized one.
* style(chat): restyle open-page and page-edit cards
* fix(ci): comply with assistant card styles and Go line limits
* feat(assistant): let the assistant manage knowledge bases
Create, update, and delete knowledge bases, tags, and manual Markdown,
and show those confirmations as labeled rows with model names.
* fix(assistant): satisfy lint on knowledge base operations
Check new error returns, wrap lines past 120 characters, and document the exported tag request types.
* fix(assistant): enforce knowledge base settings permissions and preserve patch fields
* feat(agent): let the assistant delete and move documents in batches
A batch target can now resolve each item: x-agent "items" names the list
parameter and a read run once per item, and the approval lists every
title under the knowledge base name. The list must be capped at
MaxApprovalItems (20) by x-agent-maximum, which now also bounds string
lists, so nothing approved goes unnamed. One item the user cannot read
blocks the whole batch before any prompt.
New operations: knowledge.batch_delete, knowledge.move (to another
knowledge base, destination named on the card),
knowledge.get_move_progress and knowledge_base.list_move_targets.
The move and move-progress Swagger responses now show their data
envelope.
The approval card keeps line breaks in its description and renders
trailing "- " lines as a scrollable list.
* fix(agent): reconcile batch approvals with workspace operation updates
* fix(agent): keep every batch approval item visible
* feat(agent): open more workspace operations to the WeKnora assistant
Reads: knowledge_base.list / list_shared / list_tags, knowledge.list_folders,
chunk.list, wiki.get_page / list_folders / list_revisions; knowledge.list
also takes tag_ids, folder_path and folder_recursive.
Recoverable writes: knowledge.update (title, summary), batch_reparse,
move_to_folder, rename_folder, create_manual, create_from_url,
knowledge_base.generate_profile, wiki.create_folder / move_page / revert.
x-agent gains catch_all for routes registered as *slug, which Swagger
cannot express. run_operation decodes knowledge_ids handles. The shared
knowledge base list, the tag page and the URL import body get concrete
Swagger types instead of free-form objects.
* fix(agent): show the assistant the change it is about to make
Manual documents can reach the API length limit, unpaginated lists cover
a tenant, and tag pages report their total. wiki.list_revisions can
return one stored revision before revert, and the confirmation card
names the URL, folder, version and documents.
* fix(agent): align catalog tests with the knowledge API allowlist
Catch-all assertions follow the knowledge routes, and the nested tag
page fixture stays inside an area the catalog accepts. The rune-limit
error also fits the line-length check.
* feat(agent): add the built-in WeKnora assistant with an operation catalog
Adds builtin-weknora-assistant, which inspects and manages WeKnora with the
signed-in user's permissions:
- find_operations / run_operation over an allow-listed API operation catalog,
dispatched in-process through the real router as the user
- builtin approval for write/destructive operations, refused when
non-interactive
- agent attribution in knowledge base activity, including async tasks
- page_context request field, narrowed to the turn's search targets and
rendered into runtime_context with model handles
* refactor(agent): compile the operation catalog from x-agent Swagger annotations
The catalog was a hand-written copy of what the swag annotations already
describe: routes, parameters, enums and response shapes. It now compiles
from the Swagger document embedded in the binary. Handlers opt in with
// @x-agent {"id":"knowledge.delete","risk":"destructive",...}
// @x-agent-hint "Cannot be undone."
// @Param id path string true "知识ID" extensions(x-agent=knowledge_id)
and response types mark the fields the model may see with an
extensions:"x-agent" struct tag. Exposure stays opt-in at every level:
unmarked operations, parameters and fields are not reachable, so a new
field on Knowledge or a new query parameter never reaches the model by
accident. NewCatalog still enforces the safety rules (forbidden areas,
risk versus method, approval targets), and a test fails when the
annotations and docs/swagger.json disagree.
The twelve operations compile to the same routes, parameters and
results as before; document results now share one field set, and the
parse_status filter accepts deleting, which the API supports.
* fix(agent): enforce resume and wiki context scopes
* docs(swagger): add shared response envelope types
* docs(swagger): type knowledge base responses
* docs(swagger): add concrete response body types
* docs(swagger): type chunk list responses
* docs(swagger): type knowledge mutation responses
* docs(swagger): type wiki search and autofix responses
* docs(swagger): type FAQ responses
* docs(swagger): type tag responses
* docs(swagger): type message responses and SSE schemas
* docs(swagger): type agent responses
* docs(swagger): type remaining handler responses
* docs(swagger): regenerate OpenAPI spec
* docs(swagger): type knowledge, activity and wiki maintenance endpoints
Covers the endpoints left out of the earlier passes: knowledge base
detail, document list/detail/spans/reparse/cancel/delete, knowledge
base activity and the Wiki stats, issues, lint and rebuild-links
endpoints. The spans handler now writes the documented structs instead
of gin.H, so the schema cannot drift from the body; the JSON is
unchanged. List filters gain their real enums and bounds, including
the finalizing, cancelled and deleting parse states.
* docs(swagger): document JSON-valued fields as JSON, not byte arrays
types.JSON and json.RawMessage are byte slices, so swag rendered every
metadata, details and config field as an array of integers. A swag
overrides file documents them as objects; the four fields that hold
arrays (relation chunk ids, allowed origins, image refs) say so with a
swaggertype tag.
* chore(lint): exempt swag annotation lines from the line-length limit
swag reads each @Param, @Success and @Failure as a single line, so an
enum list or a composed response type cannot be wrapped to fit.
* chore(lint): exempt struct tags carrying swag attributes from the line limit
Struct tags cannot wrap either; a field documented with swaggertype or
extensions keeps its tags on the declaration line.
* feat(gallery): filter images by source document
Add a gallery toolbar button that opens a paginated, keyword-searchable
document picker; confirming restricts the listing to images owned by
the selected documents. The backend gallery endpoint accepts a
knowledge_ids query param (capped at 50) narrowing chunk_images by the
owning document before de-duplication, so the count query stays index
served.
* feat(gallery): lift knowledge_ids cap via POST body, sortable doc picker
Add POST /knowledge-bases/:id/images accepting the gallery filter as a
JSON body so knowledge_ids escapes the URL-length cap (50 -> 1000, an
abuse guard rather than a transport limit); the GET variant keeps its
cap and both share the contract-constrain logic. The source picker
gains sortable column headers (file name / upload time, newest first
by default), a header select-all that unions and removes whole hit
lists across keyword searches, overflow guards capping the no-keyword
view at the 500 most recent documents, and a wider dialog.
* style(gallery): replace td-token fallbacks and font-size literals with design tokens
styleGuard.test.mjs enforces ratchets: --td-* vars must not carry
fallbacks (theme.css guarantees them) and font sizes must use
--app-text-* tokens. GallerySourcePicker.vue added 4 of each,
pushing the ratchets over baseline.
* fix(gallery): keep shared images in source filter and align picker UI
- Scope the de-duplication probe to the selected documents, so an image
whose newer copy lives in an unselected document is no longer hidden.
- Reset the source filter when the gallery switches knowledge bases.
- Accept an empty POST /images body; drop pruned attribute maps to nil.
- Keep the source filter out of the filter panel's count and clear action;
the empty state's clear still resets it.
- Rework the picker dialog to the BatchTagDialog shell: heading with
subtitle, footer with selection count, clickable rows, debounced search,
select-all toggling against every hit, stale select-all responses dropped.
- Remove the unreachable GET knowledge_ids branch in the API client.
---------
Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(knowledge): reject invalid JSON uploads before async parse
Validate .json payloads at HTTP upload/replace time with the same gate used
by the simple JSON reader, so JSONC or truncated files fail with HTTP 400
instead of after document-process retries. CreateKnowledgeFromFile itself
skips the gate so datasource sync and IM still create a knowledge row that
fails in async parse.
Signed-off-by: xiaojunxiang <xiaojunxiang@kingsoft.com>
* fix(knowledge): localize JSON upload validation messages
The upload queue shows the backend message verbatim and the rest of the
upload handler speaks Chinese. Map ValidateJSONContent's errors (now
sentinels with unchanged text, still shared with the async JSON reader)
to Chinese messages at the upload/replace boundary.
---------
Signed-off-by: xiaojunxiang <xiaojunxiang@kingsoft.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* feat(image): observe image attributes and let the policy decide OCR
Describe each image with the VLM as before, but ask it to also observe two
attributes about the image in the same round — how much body text it carries
(contain.text) and whether it presents data visually (contain.data_visual) —
then let a small pure function decide whether that image is worth a second,
separate OCR round.
Per-class OCR was the first shape of this idea; a class (chart / photo / logo)
is only an indirect proxy for 'is there text worth reading here', and the class
table has to grow whenever a new kind of image shows up. The decision actually
needs the observed attributes, so it reads them directly, and the attribute set
is a backend registry row rather than a new class.
The observation and the caption share one VLM call; OCR stays a separate call
(prompt isolation, see #3291). An attribute the model does not answer for is
left absent rather than filled with a conservative default, so the trace never
claims an observation that did not happen, and an incomplete observation falls
back to the on_unobserved policy (OCR still runs). The registry also carries the
panel's display text, so a new attribute needs no frontend change.
Off by default: image_attrs_enabled=false keeps today's behaviour (one caption
and one OCR per image). KB config: image_processing_config.image_attrs_enabled /
image_actions; per-upload override via process_config. Registry:
GET /api/v1/image-attrs/schema.
* feat(gallery): add image gallery tab with schema-driven filters
Adds a 4th knowledge-base tab parallel to Documents/Wiki/Graph that
browses every image asset in a KB.
Backend:
- GET /api/v1/knowledge-bases/:id/images lists image assets projected
from chunks.image_info, with keyword (caption/OCR), multi-value
attribute (attr_<name>=<value>), is_enabled and sort filters applied
in memory, plus pagination.
- Resolves the source knowledge item title (SourceName) via a batched
GetKnowledgeBatch so the viewer can show a human-readable source.
Frontend:
- ImageGallery.vue: thumbnail grid + search/sort + schema-driven filter
panel (from GET /image-attrs/schema) + single-image viewer with
prev/next, attributes/caption/OCR/source.
- Wired as the gallery tab in KnowledgeBase.vue; i18n keys in all 5 locales.
* feat(gallery): drive filters/search/sort from a configurable attribute contract
The gallery UI is no longer hardcoded: it renders from a self-describing
contract served by GET /knowledge-bases/:id/gallery-config (live attribute
sources, every attribute's definition with usage flags merged across the
config tiers, and the caller's search activation state).
- Attribute sources form an open registry (types.RegisterGalleryAttrSource);
a builtin source models caption/ocr_text/timestamps/enabled so the gallery
has zero special cases.
- Usage resolution: user > KB > system > source, field by field; the system
tier lives in system_settings (gallery.policy, JSON, no rebuild), the KB
tier slot is wired in the engine but not stored yet.
- Search activation is the user's own record: mode (all|custom) plus
per-field on/off in users.preferences.gallery; source declarations never
set defaults (unrecorded = off in custom mode).
- image_info is read through a gallery-local struct, removing the compile
dependency on the attribute-pipeline types; the gallery builds and runs
with or without that feature deployed.
- ListImages validates search_in / sort_by / attr_filters against the
contract; ineligible fields are dropped, not served.
* fix(frontend): inline the shared graph-drawer stylesheet (@import instead of <style src>)
plugin-vue 6.0.6 keys unscoped src-style descriptors by the imported file
path, so AgentStreamDisplay.vue and WikiBrowser.vue — both importing
components/css/wiki-graph-drawer.less — overwrite each other during the
build and the second one crashes with "reading 'scoped'". Adding the
gallery changed the module order enough to make the collision hit every
time. Inlining the same stylesheet keeps the emitted CSS identical and
removes the shared descriptor key.
* feat(gallery): render attributes from short labels with descriptions in tooltips
A source could only say one thing about a value, so the filter panel had to put
a full sentence inside a checkbox. Attributes and their values now carry two
pieces of text: a short label for the space-constrained controls (checkbox,
search field, sort option) and a description for wherever there is room
(tooltip, detail panel).
The contract's value list becomes an array of {value, label, description}, and
the display helpers resolve each field separately: the gallery stamps the short
name on a control and moves the sentence into a tooltip. Translation overlays
are looked up per field, so a source that ships no gallery-specific text still
picks up the attribute pipeline's own wording.
* fix(gallery): let locale text win over the contract's wording
The filter panel read the backend's own label before asking the locale,
so a translated value never got a chance: the contract ships its own
default-language wording, and it shadowed the overlay for every locale.
Built keys were also off by a segment — imageAttr.<name> has to be
imageAttr.<name>.label, and the gallery's own keys live under
knowledgeEditor.wikiBrowser.gallery.attr — so builtin and system
attributes fell through to English on a Chinese UI.
* feat(gallery): top bar layout, pinned panels and per-value verdicts
The toolbar moves onto its own row above the grid, so the controls no
longer eat into the space the images are there for: search box, the two
scope switches, sort field and direction, and the result count all sit in
one flex row. The settings behind them are no longer inline either — each
opens as a popover under its control and can be pinned to a right-hand
rail, where the two panels take turns as tabs instead of both claiming
width at once.
Filtering gains a third state per attribute value. Where a value could
only be picked or not picked, it can now be "off" (hide those images),
"on" (show them whatever else says) or neutral (leave them alone), which
is what lets one attribute's rules talk about an image another attribute
tried to hide without the pair of them cancelling each other out: a
forced display outranks a forced hide, and anything no rule speaks about
stays visible.
The switch that used to only exist inside the panel now has a toolbar
home as well. "筛选:全显示" genuinely drops the attribute constraint
instead of imposing it against a panel the user cannot see.
Filtering used to empty the list the moment it was switched on. Any
image whose attribute was never observed — every image ingested before
the attribute pipeline existed — read as a mismatch for every filter,
which is not narrowing a list but losing it. Absence now stays silence:
an attribute no image carries places no constraint on it, until the
pipeline catches up and fills the value in.
Co-authored-by: WorkBuddy
* feat(gallery): expose the image attribute pipeline as a gallery source
The gallery's contract resolved to a single source, so the filter panel
could only offer "Enabled" while the attributes the observe-and-describe
pipeline fills in — contain.text, contain.data_visual — never reached the
UI. The registry is a gallery attribute source like any other: every
attribute it declares is offered to the gallery, filterable by default,
with the config tiers able to override the usage flags at runtime.
The registration lives in the attribute pipeline's own commit; this branch
carried an earlier, squapshot of that work which predated the split of an
attribute value into a short label and a description, so the registry's
value shape and this file have been brought in line with it. Both PRs now
hold byte-identical copies of the source hook, which is what lets either
one land first and the other merge cleanly afterwards.
Co-authored-by: WorkBuddy
* fix(gallery): viewer race, pin icon, panel anchoring and scope wording
Opening the viewer rendered the img element with the previous or empty
source while the proxy fetch was still producing a blob, and the error
that empty source fired locked the viewer in its failure state even
after the real image arrived. The resolved source now lands before the
flag is consulted, and a token keeps a slow fetch of one image from
overwriting a faster one that navigated past it.
The pin button drew nothing: TDesign has no "pushpin" glyph, so the
button rendered as an empty shell. It now uses pin / pin-filled, keeps
its place beside the tab bar, and a pinned rail assigns its columns
explicitly — the panel precedes the main section in the DOM, so leaving
the grid to source order handed the wide column to the panel and
squeezed the images into the rail.
The floating panel no longer appears at a hardcoded offset: it anchors
just below the arrow that summoned it, the arrow flips up while its
panel is out, a second press retracts, and any click elsewhere in the
toolbar closes it. When a rail is pinned the arrows turn to the
requested tab instead of spawning a second surface.
The scope words (全部 / 全显示 / 自定义) become plain status text rather
than a second dropdown — they name the current mode, and only the arrow
beside them opens the panel that changes it.
Co-authored-by: WorkBuddy
* fix(web): align the OCR-on-unobserved switch with the other setting rows
The switch sat bare inside its setting-row while every other switch on
the page is wrapped in setting-control, which is what reserves the right
column and centers the control vertically. With the row laid out from
the top, the bare switch hugged the row's upper edge and crowded the
description text — visibly out of place next to its neighbors. Wrapping
it in setting-control gives it the same right-aligned, centered slot as
the rest.
Co-authored-by: WorkBuddy
* fix(web): move the OCR-on-unobserved row out of the vertical schema row
The row lived inside setting-row-vertical, whose descendant rule stretches
every setting-info and setting-control beneath it to 100% width. Two
full-width flex children side by side cannot fit, so the control overflowed
the container and the switch landed past the right edge — visibly further
out than the switch on the row above. The row and its note now sit beside
the vertical block instead of inside it, under their own condition, so the
switch shares the ordinary row layout and its right edge lines up with the
"image attribute observation" switch above.
Co-authored-by: WorkBuddy
* fix(gallery): applying a verdict now activates the filter scope
Setting an off/on verdict while the panel's master switch was off
silently ignored it: buildParams only attaches attr_rules in custom
scope, and setVerdict never flipped the scope. A verdict the user just
expressed now switches the scope to custom; clearing the last meaningful
verdict returns to all, since custom with no rules constrains nothing.
* fix(gallery): search checkboxes stay visible and idle panels read as dimmed
The search panel hid its field checkboxes while the master switch was
on, so the only way to reach them was to flip the switch first. They now
sit next to the switch like the filter panel's verdicts do: dimmed in
all-field mode (where they carry no weight) yet clickable, and touching
one drops the mode to custom. The filter panel's verdict rows dim the
same way while the filter switch is off.
* fix(gallery): bulk verdict row, neutral default mark, panel polish
The clear-filters button never matched what the panel is for, so it is
now a bulk verdict row: one set of -/off/on buttons that applies the
chosen position to every value of every value-typed attribute, with the
default mark turned into a language-neutral dash. The panel head drops
its second divider (the tab bar already draws one) and the tabs follow
the toolbar's order: search left, filter right.
* fix(gallery): drop the all-fields caption and center the order arrow
The caption above the search checkboxes showed and hid with the master
switch, shoving the checkboxes down on every flip; it said nothing the
dimmed state doesn't, so it is gone. The order button's arrow is an
inline svg, which sits on the text baseline and reads as riding high;
it now centers against the label.
* fix(gallery): center the order arrow for real
vertical-align never applied: the icon is a flex item inside the
button's inline-flex text wrapper, and the wrapper's default stretch
pins the explicitly-sized 16px icon to the top of the 22px line box.
align-items: center on the wrapper is what actually centers it.
* fix(gallery): shorten panel tab labels in en and ru
The two-word labels overflowed the 300px panel and forced the tab bar
into scroll arrows; single-word labels fit in every locale that was
affected.
* docs: drop the image pipeline rewrite from this branch
The rewritten 'Go 侧接手后' paragraph belongs to the image attribute observation
work (#3635), which this branch carries as a prerequisite and which already
documents the caption_ocr / observation_driven split. Keeping a competing
rewrite of the same paragraph on the gallery branch makes it collide with
upstream, which extended that paragraph with the custom_instructions and inline
HTML table notes. Restoring the shared wording keeps the gallery branch free of
this file so upstream's edits merge cleanly.
* style: gofmt the gallery attribute types
Struct field tags in the resolved gallery config were one column narrow, so
gofmt rebroke the alignment block. Whitespace only.
* style(gallery): use design tokens in the gallery styles
The style guard ratchet fails on any new violation. ImageGallery.vue shipped
with 47 TDesign token fallbacks, 21 literal font sizes, 4 literal radii and a
0.15s transition; all of them now read the --app-* / --td-* tokens, which is
what the guard asks for and what dark mode depends on.
* lint(gallery): fix the golangci-lint findings in the gallery code
lll: the three over-long swagger @Param lines and one registry row were
shortened. ineffassign: ctx was assigned before the access check and then
reassigned, so the first value was dead; it is now declared once, after the
check. errcheck: the three unchecked c.Error calls in ListImages. revive: the
AttrSpec / AttrValue doc comment block was attached to the wrong type and split
into one block each. staticcheck S1016: the gallery registration converts the
registry value instead of restating its fields.
* lint(gallery): wrap the image query select and check the last c.Error
* style: reformat the wrapped image select in chunk.go
* fix(gallery): address review findings on ordering, KB switch and blob lifetime
- sortImageAssets: swap operands for desc instead of negating, keeping a
strict weak ordering so ties page stably
- ImageGallery: reload on knowledgeBaseId change, drop stale list
responses, fetch thumbnails in parallel with one cached blob per URL,
revoke blobs on unmount, default sort to builtin:created_at
- KnowledgeBase: non-wiki KBs with a stale wiki/graph tab fall back to
documents again
- handler: move ListImages godoc out of ListKnowledgeFolders' block and
document attr_filters / attr_rules
- DecideOCR: a payload with no OCR conditions uses the built-in policy
instead of silently skipping OCR
* docs(parsing): describe the caption_ocr and observation_driven image pipelines
Carried over from #3635, which this PR supersedes.
* fix(gallery): load local:// and other storage handles through the KB file proxy
The gallery's own scheme check only caught resource://, provider:// and
storage://, so local:// (the default storage), minio://, cos:// and s3://
images went straight into <img src> and never rendered. Resolve through
isProviderFileURL, the scheme list chat and wiki already use, and lock the
routing with a test.
* fix(kb-editor): keep API-customised OCR conditions on save and upload
The KB editor always wrote the registry default into image_actions.ocr.on,
so any unrelated edit replaced a knowledge base's custom OCR conditions
with the default table; the upload dialog did the same for the per-document
override. Keep a stored custom list and fall back to the registry default
only when there is none, and show the conditions actually in effect instead
of the default table in both panels.
* perf(gallery): de-duplicate, filter, sort and page images in SQL
Every gallery request used to read every image-bearing chunk of the KB,
OCR text included, decode it in Go and page in memory. A single query now
expands the image_info arrays, keeps one row per image URL, applies the
keyword / attribute / verdict / enabled filters and the sort, and returns
only the page with the filtered total.
The expansion carries narrow scalars only; filters and JSON sort keys are
evaluated on the winning entry of each image, and the page's JSON is read
back for its rows alone. Postgres runs the query with JIT off, since the
fixed 100-rows-per-call estimate of jsonb_array_elements otherwise prices
it ~100x high and compilation dominates.
On 20k images with 2 KB of OCR text each (40k chunks), a page drops from
~750 ms to ~100-200 ms on postgres and ~200 ms on sqlite; keyword search
over OCR text stays a full scan (~400-600 ms) until the text is indexed.
The filter semantics are pinned against both databases.
* perf(gallery): serve the gallery from a trigger-maintained chunk_images table
Reading images out of chunks.image_info meant parsing every image chunk of
the knowledge base on each request, so a page still cost time in
proportion to the KB. chunk_images (versioned/000113, sqlite/000032) keeps
one row per image_info entry with caption, OCR text and attributes as
columns; triggers on chunks keep it in step with every write path, and the
migration backfills existing chunks.
Copies of one image stay separate rows and the query lists the copy on the
most recently updated chunk through an indexed anti-join, so writes decide
nothing and concurrent writers need no locking. On postgres, caption and
OCR text get pg_trgm indexes when the extension is available.
On 20k images with 2 KB of OCR text each (40k chunks):
postgres first page 10 ms, page 500 41 ms, keyword 2 ms, filters 23 ms
sqlite first page 3 ms, page 500 39 ms, keyword 147 ms, filters 51 ms
down from ~100-600 ms. Writing those 40k image chunks costs 7.0 s instead
of 1.3 s on postgres (1.9 s instead of 1.0 s on sqlite), mostly the trigram
indexes; status and enable-flag updates patch rows in place.
---------
Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* feat(knowledge): flag documents that stop making progress
A document stuck in processing looked the same as one still working: a
spinner that polled forever, no hint of where it stopped, and, hours
later, a generic housekeeping failure.
- The batch and spans endpoints return last_activity_at for in-flight
rows: the later of updated_at and the newest span write. updated_at
only moves at stage transitions; span writes keep moving while a long
stage still works.
- After 20 minutes with no progress, the list and card views show
"may be stuck" with a tooltip. The timeline shows the stage it
stopped at and a stop-parsing button, plus a "last progress" stat.
Both pollers back off to 15s while everything in flight is stalled.
- When housekeeping fails a row, the error names the stalled stage and
the time of the last progress. It closes the row's open spans with
TASK_STALLED (the running stage fails, the rest are cancelled), so
the timeline shows where the run stopped instead of a spinner.
* fix(knowledge): locate stalls in finalizing and tell backlog from stuck
- Housekeeping reads the stall site from the latest attempt. It takes
the running stages, or, when none runs (post-process closes its stage
once enrichment is fanned out), the innermost running spans, mapped
back to their stage. Those spans fail with TASK_STALLED and are named
in the message; every stalled stage is listed in pipeline order. The
other open spans are cancelled. Every closed span gets a duration, and
the message leaves out the time when the heartbeat query failed.
- The spans tree prefers a TASK_STALLED failure for last_error over a
later subtask failure. current_stage falls back to the stage owning a
running subspan.
- Batch and spans responses flag rows quiet past the stall hint whose
work is still queued (waiting_in_queue), using the sweep's own probes.
At most 20 rows are probed per request, cached for a minute. The UI
shows these as queued, with no stop button in the banner.
* fix(knowledge): answer stall verdicts from one shared queue scan
- Probing a quiet document for a backlog rescanned the whole queue once
per document. When the document was absent, which is exactly the
stuck case, the scan ran to the end, and it read a Redis error as
"nothing queued". QueuedKnowledgeIDs now collects every referenced
knowledge ID in one pass and returns list errors. HousekeepingService
keeps that snapshot for a minute, shared across requests and
single-flight. A failed scan is not cached.
- With the per-request cap of 20 gone, every quiet row on a page gets a
verdict. The response carries it as stall_state (queued or stalled),
replacing waiting_in_queue. The UI marks a row as stuck only on a
"stalled" verdict. A row with no verdict, because the probe failed,
shows as ordinary processing instead of inviting a stop.
Closes#3452.
buildSpanTree fills every canonical stage that has no span row using a
status inferred from knowledge.parse_status, so a failed parse painted
all of them "failed". That status carries information it cannot know:
it cannot tell "this stage ran and failed" from "this stage never ran".
The frontend picks the focused stage with the first failed one in
canonical order, so whenever the real failure left an earlier stage
rowless, the UI named the earlier stage. On main today a text passage is
chunked without ever opening a docreader stage, so an embedding failure
reports 文档解析 / docreader as the broken step for content that has no
document to parse, and four of five segments render as failures.
Once a stage has actually recorded a failure its position is known, so
missing stages are now resolved against it:
- downstream of the failure, per StageDependencies or canonical order →
cancelled, which is what the status is documented to mean
- before it → skipped; the stage either did not apply or its row was
never written, and neither of those is a failure
With no failed row anywhere the parse_status fallback stands unchanged,
which is the case it was written for: legacy knowledge parsed before
span tracking existed still renders done/failed rather than five pending
segments.
Also maps parse_status "cancelled" to a cancelled placeholder; it fell
through the switch to pending, so a cancelled document showed spinners
on the stages it never reached.
The DAG walk is redundant with canonical order for today's stage list,
where every dependency points at an earlier stage. It is there so the
answer stays right if the list is reordered or a stage gains an upstream
that is not its predecessor.
No frontend change: skipped and cancelled already render, with localized
labels in every locale.
* refactor(resource-access): unify scoped authorization and transfer recovery
* fix(resource-access): close reparse and transfer review gaps
* fix: enforce explicit file bindings and recover move cleanup
* fix: align mutation admission and cleanup scope checks
- Updated .env.example and docker-compose.yml to clarify skill bundle size limits and their implications for GitHub source downloads.
- Enhanced frontend logic to differentiate between skill bundle uploads and knowledge uploads, ensuring proper error messages for oversized requests.
- Introduced new utility functions for managing upload limits and added tests to validate behavior for oversized JSON bodies.
- Refactored error handling in various handlers to provide clearer feedback on upload size constraints.
- Updated .env.example to clarify the skill bundle size limits and their impact on Nginx configuration.
- Refactored frontend/docker-entrypoint.sh to separate max file size settings for skills and general uploads.
- Enhanced nginx.conf to apply specific body size limits for skill uploads without affecting other endpoints.
- Added tests to ensure proper handling of skill bundle file indexing and size constraints.
- Improved error handling in tenant_source.go and tenant_sandbox_config.go to manage skill file limits effectively.
* feat: support editing document summaries
* fix: address review feedback for manual summary editing
Reset summary edit state when the drawer closes or generation is in
flight, sync summary_status on manual description updates, and add
service tests plus API docs for partial PUT behavior.
* chore: gofmt client/initialization.go for pre-push hook
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
Document folder tree/move/rename endpoints in docs/api, Swagger, and the Go
client; align frontend path normalization with server caps; guard rename
no-ops and duplicate folder creation; enforce KB ownership on folder rename.
The scope switch is gone. It existed because "select a folder" was ambiguous
in a list that could not show folders, and turning that ambiguity into a
control just moved the problem onto the user.
The list now shows what a folder contains: its sub-folders as entries first,
then the documents directly inside it. So the mode follows what the user is
already doing instead of a switch - browsing shows one level, and any active
filter searches the whole subtree flat, with a breadcrumb note saying so. A
document uploaded on its own needs no special row either: it sits at the top
level next to the folders, exactly where it is.
Folders can also be adjusted now, which they could not before - a mis-filed
upload previously had to be deleted and re-uploaded. Documents move via the
row menu or the batch bar into an existing or newly typed folder, and a
folder can be renamed in place, carrying its subtree. Both are plain
folder_path updates: nothing is re-parsed, re-chunked or re-embedded.
Folder uploads used to encode their relative directory inside file_name,
which made the document list render the whole path as the title and left
no way to browse or query a single folder. The directory now lives in a
dedicated folder_path column (with a backfill for existing rows), the
document list accepts folder_path / folder_recursive filters, and a new
GET /knowledge-bases/:id/knowledge/folders endpoint returns the folder
tree with per-folder document counts.
Reject invalid agent_source_tenant_id instead of silently falling back, fix a resolveAgent nil panic when a shared selector fails, and add regression tests for wiki write-tool filtering and agent resolution.
Discard stale async refreshes without clobbering newer summaries, queue manual refreshes instead of blocking HTTP, sort edited chunks by index for summaries, escape document metadata in chat headers, and distinguish lookup failures from true staleness.
Co-authored-by: Cursor <cursoragent@cursor.com>
Downloading exposes the original source file, which is more sensitive than viewing parsed content or previews. Align backend route guards, handler checks, and frontend UI so tenant Viewers and org-shared read-only access cannot download.
Stop failing distributed-mode knowledge rows on startup; let housekeeping
decide using span activity and Asynq queue state. In Lite mode, resume
wiki work from durable task_pending_ops and surface SERVER_RESTART errors
in the processing timeline with i18n support.
Refactor the error handling logic in the GetKnowledgeSpans function to utilize a new helper function, knowledgeSpansLastError, which consolidates the logic for determining the last error from span failures and knowledge row failures. This improves clarity and maintainability of the code. Additionally, update test cases to cover new scenarios for last error handling.
Latest attempt tabs and header now trust parse_status over a closed root span, surface recovery failures via last_error, and keep attempt hover styles from washing out the active tab.
Align product copy, API docs, Swagger descriptions, and i18n strings
with workspace terminology while keeping internal tenant identifiers
and headers unchanged for API compatibility.
Route summary/sync/maintenance tasks to dedicated queues and servers so parsing, enrichment, and batch work no longer share one concurrency budget, while extending the runtime dashboard and settings validation to reflect the new topology.
Replace scopes JSON with a viewer/contributor/admin role column, move
unsafe-method authorization from path-prefix middleware to declarative
APIKey* guards on routes, and close remaining KB-scope gaps in handlers.
Enforce per-key knowledge base restrictions on batch/search read paths,
centralize scope checks in shared KB access validators, and propagate
TenantAPIKeyScope through logger.CloneContext so async QA cannot bypass it.
Gate wiki GET routes and initialization config reads with KBAccessRead,
and bind async task progress endpoints to the caller tenant via task ID
parsing. Closes cross-tenant read paths reported in GHSA-ph8f-4qwx-89rr
and related initialization/progress leaks.
Wire per-turn @mentions for skills, MCP services, tags, and files through
QA handlers into agent config, runtime_context/must_use user blocks, and
tool registration. Fix tag-only scopes dropping KB tools, refresh suggested
questions on mention changes, and strengthen MCP must_use so @selected
services are invoked before local KB search.
Split tag filtering from CRUD management on the knowledge base document
page, align copy to "tags" across locales, and polish hover/detail surfaces
with setting-drawer section styling. Return tags from GetKnowledge for the
detail drawer.
Clear tag relations on move/delete, validate tags on create, batch-load tags,
restore KB-scoped reference counts, dedupe SetKnowledgeTags, fix filter
double-fetch and ResizeObserver leaks, and regenerate Swagger for tag_ids.
* feat: support multi-tag for document knowledge
- Add knowledge_tag_relations join table for many-to-many tag-knowledge mapping
- Migrate existing single tag_id data to the new table
- Update Knowledge struct, filter, payload types (TagID -> Tags/TagIDs)
- Rewrite repository layer: SetKnowledgeTags, GetKnowledgeTags, ListIDsByTagIDs
- Update tag count references to query knowledge_tag_relations
- Update service/handler layers for multi-tag create/list/batch-update
- Frontend: multi-select tag sidebar, multi-tag chip display, tag toggle in popup
- FAQ tags remain single-tag model (unchanged)
* feat(knowledge): tag edit dialog with dynamic overflow and inline creation
- Add TagEditDialog component with search, multi-select, and confirm
- Replace inline t-popup tag selector with dialog-based editing
- Tags overflow with +N badge, visible count adapts to container width via ResizeObserver
- Date/card layout: fixed date width, tags flex to fill remaining space
- Add persistent new-tag input bar in dialog (Enter to create, auto-select)
- Tag creation emits callback to refresh parent tag list immediately
- Apply same overflow + dialog trigger to both card view and list view
- Add i18n keys: tagEditDialogTitle, tagEditSearch, tagSelectedCount, tagOverflowTip, tagNewPlaceholder
* fix: exclude soft-deleted knowledge from tag reference counts
CountReferences, BatchCountReferences, and DeleteUnusedTags now
JOIN with knowledges table to filter out soft-deleted records
when counting knowledge_tag_relations. Previously they queried
knowledge_tag_relations directly without the deleted_at check,
causing stale tag counts after knowledge deletion.
* fix: refresh tag counts after async knowledge deletion completes
Add a second delayed tag list refresh (2.5s) after tag deletion to capture
updated reference counts for remaining tags. The original immediate refresh
handles removing the deleted tag from the list, while this delayed refresh
ensures other tags' counts are correct after the async deletion finishes.
---------
Co-authored-by: lyingbug <xiayula2014@gmail.com>
Move batch reparse work to an asynq task to avoid HTTP timeouts, add i18n
for all locales, guard in-flight documents, and prevent concurrent delete/reparse.
This commit introduces the Batch Reparse feature for the Knowledge Base.
Users can now select multiple documents and trigger a re-parsing pipeline in batches, streamlining the recovery from parse failures.
Add recent=true to knowledge search so the mention picker can list recent
files without a keyword, and consolidate KB/FAQ/file chip styling across
chat input and message history.
- Enhanced the `reparseKnowledge` function to accept optional process configuration overrides, allowing users to adjust parsing settings during reparse operations.
- Updated the frontend components to integrate reparse capabilities, including the `UploadConfirmDialog` and `KnowledgeBase` views, providing a user-friendly interface for confirming reparse actions.
- Added new localization strings for reparse confirmation and process configuration details in multiple languages.
- Modified the backend API to handle reparse requests with optional configuration, ensuring proper validation and persistence of overrides.
- Introduced new types and interfaces to manage reparse-specific data structures in the store, improving type safety and clarity in the codebase.
- Updated the `CreateKnowledgeFromFile` function to accept an optional `processConfig` parameter for per-upload parse configuration overrides.
- Introduced the `KnowledgeProcessOverrides` type to define various parsing settings, including parser engine rules and chunking configurations.
- Modified the API client and related components to handle the new process configuration, improving flexibility in knowledge file uploads.
- Updated documentation and API specifications to reflect the new `process_config` parameter in the upload process.
- Added a new `UploadConfirmHost` component to facilitate user confirmation for uploads, integrating with the updated upload logic.
- Enhanced user experience by providing detailed configuration options during the upload process.
Moving knowledge with mode=reuse_vectors copies vector index entries
directly between knowledge bases, which only works when both KBs are
bound to the same vector store (CopyIndices is routed through the source
store). Nothing rejected a cross-store reuse_vectors move, so it would
copy target-KB rows into the source store and then delete the source
indices — corrupting vector data. The move endpoint validated only the
embedding model, and ListMoveTargets surfaces same-model KBs regardless
of their store binding, so a cross-store target is reachable.
Reject reuse_vectors moves whose source and target KBs are bound to
different vector stores:
- MoveKnowledge handler: synchronous 400 before enqueueing, with a
message pointing the caller at reparse mode.
- moveOneKnowledge: re-check before marking the knowledge as processing,
so a rejected move leaves it untouched rather than stranded.
- moveKnowledgeReuseVectors: a top-of-function guard as defense in depth
for any directly enqueued task, satisfying the pre-existing contract
noted in the comment ("callers must reject cross-store moves").
reparse mode is unaffected: it re-indexes into the target store from
scratch and is the supported way to move across stores.