Commit Graph

70 Commits (main)

Author SHA1 Message Date
lyingbug 0d7516ea0a
feat(assistant): expand management operations and polish interaction UI (#4427)
* fix(assistant): polish interaction cards and stabilize composer context

Localize operation labels across all six UI languages and expose answer copying in the assistant panel. Refine question and page-link cards with a single open/return action, and keep the current-page context row at a stable height.

* feat(assistant): add common data source, chunk and knowledge base operations

* feat(assistant): support parsing configuration and batch FAQ updates

* feat(assistant): add MCP tool directory query and sync operations

* feat(assistant): manage web search providers and data source sync settings

* feat(assistant): add model configuration copy operation

* feat(assistant): add workspace profile members and system information operations

* feat(assistant): add system administrator queries and runtime diagnostics

* fix(frontend): polish assistant and resource card UI

Refine assistant composer layout, context controls and reply actions, including narrow-window alignment. Unify knowledge base, agent and shared-space cards with consistent icons, hover feedback, favorites, menus and compact localized statistics.

* fix(assistant): preserve concurrent settings and exclude upstream error secrets
2026-10-17 14:35:58 +08:00
lyingbug 7115cb2a61
feat(assistant): trial runs, session grants, page links, and MCP/model/skill management (#4375)
* feat(assistant): let the assistant check MCP services and retrieval

Add a probe risk for operations that change nothing but do real work
(connecting to a service, running a retrieval). Probes use POST, run
without approval at every access level, and are capped per run.

Open three operations on top of it:
- mcp_service.test: connect with the saved settings and report why not
- mcp_service.list_tools: list what a service offers before binding it
- knowledge_base.search: raw hybrid recall with thresholds, to explain
  why a document is or is not found

x-agent can now name one method when a handler documents several, and
catalog parameters accept numbers.

* feat(assistant): let the assistant try an agent it set up

Add try_agent: the assistant asks an agent one question the way a user
would and gets back its answer, the tools it called and the documents it
cited, so it can check a setup before calling it done.

- The trial runs as the signed-in user in a new agent_trial session kind,
  which ordinary listings already exclude and the API cannot create, and
  the session is removed afterwards. Runs are non-interactive, so tools
  that would ask the user are refused. Long-term memory is untouched: it
  is written only by the HTTP handler.
- The assistant itself, the Wiki fixer and the skill installer cannot be
  tried, nor any agent granted the operation catalog.
- A trial spends model calls and runs the agent's own tools, so it is a
  write for the access level: confirm asks, auto_write runs, read_only
  does not offer it. The confirmation card names the agent and question;
  ConfirmedTool now lets a wrapped tool supply that text.
- At most 3 trials per turn, 4 minutes each; the tool timeout is raised
  above that. Answers, tool errors and references are capped.
- The runner is bound to the session service after the container builds
  it, since the session service depends on the agent service.

* feat(assistant): let the user allow similar changes for the conversation

The confirmation card for a recoverable change now offers "allow this for
the rest of the conversation". Once chosen, later calls of the same kind
in the same session run without asking, so a batch tidy-up no longer
needs a click per item.

- What a grant covers is derived on the server, never taken from the
  model: the same catalog operation on the same knowledge bases (a
  document's knowledge base comes from the target read) or the same
  agent; the same Wiki tool on the run's Wiki knowledge bases; trials of
  the same agent. Deletions and operations without such a scope, such as
  creating a knowledge base, always ask.
- No new storage: the gate looks for an earlier approved prompt in the
  session with the same grant and allow_in_session set, for the same
  tenant and user. Grants therefore survive suspend/resume and replicas,
  and end with the conversation. A failed lookup asks again.
- The response validator accepts allow_in_session only on approval and
  only when the prompt offered a grant, so a client cannot widen it.
- The card shows the scope in the user's language. The model is told
  such calls may run without a card and that deletions always ask.
- describeTarget now returns its target read so the grant reuses it
  instead of reading again.

* feat(assistant): tell the assistant about settings pages and chat answers

Pages now report which page is open and, for the agent editor, which
section. The model and MCP settings name the model or service open for
editing, and chat answers gain an "ask the assistant" button that
attaches that answer. The backend resolves these for the assistant only:
models and MCP services in the caller's tenant without endpoints or
credentials, and messages through the caller's session read access, with
the question, an answer excerpt and cited sources quoted as data.

* feat(assistant): link the user to pages and filled-in forms

The assistant gains open_page, which shows a button to a knowledge base
or document, a Wiki page, an agent's editor at a tab, a model or MCP
service to edit, or a new model or MCP service with its form filled in.
Targets are read through the operation catalog as the user, so a link
never points at something they cannot open. Drafts take no credentials
and only http(s) endpoints without user info; the card shows the
endpoint, and the user enters keys and saves the form themselves.

* feat(assistant): manage MCP services, model names and skills

* fix(assistant): prefill model drafts on initial navigation

* feat(assistant): document tags, model catalog reads, clearer cards, live page refresh

- Operations project lists of objects; knowledge.get/list report tags.
- model.get shows provider, endpoint and capabilities; new
  model.list_providers and model.get_provider read the model catalog,
  including vendors an admin added.
- open_page and ask_user guidance: offer page buttons whenever the next
  step is the user's, word choices for the user.
- Chat cards: redesigned open-page card pinned under the final answer,
  ask-user inputs in the dock style, approval card with aligned fields,
  translated labels and a question built from the operation summary.
- Open pages reload after assistant changes: deletes and moves wait for
  the background task, Wiki edits now report their changes, KB list
  counts, tags and chat model selectors refresh.
- The assistant panel stays open across reloads within the tab.

* fix(ci): format assistant operation code and wrap long lines

* feat(assistant): let the assistant see, fill and open the page beside it

- read_page / fill_page tools answered by the browser over a page_action prompt,
  with refs, redaction, in-place text edits, undo card and a page-access switch
- composer shows the open page, dialog and selection; a selection bubble and an
  editor toolbar entry ask the assistant about the selected text
- edits scroll into view with a highlight and an inline undo bar
- open_page reaches any settings section, toolbox tool and list page, and can
  navigate there itself to read it, keeping the button when edits are unsaved

* feat(assistant): let the assistant open the tab it needs

A visible tab can be selected and re-read, and system settings can open straight to user management, so the next step is no longer blocked on the wrong tab.

* feat(assistant): expose read-only conversation operations

Add session.list, message.list, message.search and
message.chat_history_stats to the operation catalog, restrict the
sessions and messages areas to non-mutating operations, and search
messages with the raw query instead of the log-sanitized one.

* style(chat): restyle open-page and page-edit cards

* fix(ci): comply with assistant card styles and Go line limits
2026-10-16 23:36:01 +08:00
tayadwx db7046b85e
refactor(image): turn the hard-coded image flow into configurable pipelines (#3746)
* feat(image): configurable image pipelines, rebuilt on current main

Rebuilds the image-pipeline work on top of the current main, carrying
only the pipeline and request-side changes:

- The hard-coded image flow becomes registry-driven pipelines (default /
  smartocr), selectable per knowledge base, with a frontend panel and
  per-action thinking switches addressed as action-level keys
  (caption.thinking / ocr.thinking) declared once and carried into every
  pipeline.
- Request-side support: VLM.PredictWithOptions carries a per-call
  thinking switch through every implementation and decorator, including
  the background concurrency wrapper.
- OCR failure handling follows main's #4132 semantics: invalid output is
  rejected and recorded (ocr_status / ocr_error_code / .ocr subspan)
  without retrying the whole image; caption/observation failures get
  symmetric subspans instead of leaving the image span green.
- Permanent VLM failures skip the asynq retry budget; a run that
  produced nothing fails honestly instead of passing as skipped.
- UI and locale fixes travel with the panel.

Concurrency, rate limiting and circuit breaking are NOT part of this
branch: they stay with the existing system limiter.

* fix(image): address pipeline configuration and execution regressions

* fix(image): align pipeline overrides and validation semantics

* style(image): keep the new pipeline code inside the lll and gofumpt gates

---------

Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-16 16:18:57 +08:00
lyingbug 2fbfb44715
feat(assistant): let the assistant manage knowledge bases (#4353)
* feat(assistant): let the assistant manage knowledge bases

Create, update, and delete knowledge bases, tags, and manual Markdown,
and show those confirmations as labeled rows with model names.

* fix(assistant): satisfy lint on knowledge base operations

Check new error returns, wrap lines past 120 characters, and document the exported tag request types.

* fix(assistant): enforce knowledge base settings permissions and preserve patch fields
2026-10-15 23:54:14 +08:00
lyingbug aaa1accebd
feat(agent): let the assistant delete and move documents in batches (#4319)
* feat(agent): let the assistant delete and move documents in batches

A batch target can now resolve each item: x-agent "items" names the list
parameter and a read run once per item, and the approval lists every
title under the knowledge base name. The list must be capped at
MaxApprovalItems (20) by x-agent-maximum, which now also bounds string
lists, so nothing approved goes unnamed. One item the user cannot read
blocks the whole batch before any prompt.

New operations: knowledge.batch_delete, knowledge.move (to another
knowledge base, destination named on the card),
knowledge.get_move_progress and knowledge_base.list_move_targets.
The move and move-progress Swagger responses now show their data
envelope.

The approval card keeps line breaks in its description and renders
trailing "- " lines as a scrollable list.

* fix(agent): reconcile batch approvals with workspace operation updates

* fix(agent): keep every batch approval item visible
2026-10-15 15:57:58 +08:00
lyingbug a37a1bc51a
feat(agent): open more workspace operations to the WeKnora assistant (#4318)
* feat(agent): open more workspace operations to the WeKnora assistant

Reads: knowledge_base.list / list_shared / list_tags, knowledge.list_folders,
chunk.list, wiki.get_page / list_folders / list_revisions; knowledge.list
also takes tag_ids, folder_path and folder_recursive.

Recoverable writes: knowledge.update (title, summary), batch_reparse,
move_to_folder, rename_folder, create_manual, create_from_url,
knowledge_base.generate_profile, wiki.create_folder / move_page / revert.

x-agent gains catch_all for routes registered as *slug, which Swagger
cannot express. run_operation decodes knowledge_ids handles. The shared
knowledge base list, the tag page and the URL import body get concrete
Swagger types instead of free-form objects.

* fix(agent): show the assistant the change it is about to make

Manual documents can reach the API length limit, unpaginated lists cover
a tenant, and tag pages report their total. wiki.list_revisions can
return one stored revision before revert, and the confirmation card
names the URL, folder, version and documents.

* fix(agent): align catalog tests with the knowledge API allowlist

Catch-all assertions follow the knowledge routes, and the nested tag
page fixture stays inside an area the catalog accepts. The rune-limit
error also fits the line-length check.
2026-10-15 15:19:23 +08:00
lyingbug 687b178706
feat(agent): add the built-in WeKnora assistant with an operation catalog (#4302)
* feat(agent): add the built-in WeKnora assistant with an operation catalog

Adds builtin-weknora-assistant, which inspects and manages WeKnora with the
signed-in user's permissions:

- find_operations / run_operation over an allow-listed API operation catalog,
  dispatched in-process through the real router as the user
- builtin approval for write/destructive operations, refused when
  non-interactive
- agent attribution in knowledge base activity, including async tasks
- page_context request field, narrowed to the turn's search targets and
  rendered into runtime_context with model handles

* refactor(agent): compile the operation catalog from x-agent Swagger annotations

The catalog was a hand-written copy of what the swag annotations already
describe: routes, parameters, enums and response shapes. It now compiles
from the Swagger document embedded in the binary. Handlers opt in with

  // @x-agent      {"id":"knowledge.delete","risk":"destructive",...}
  // @x-agent-hint "Cannot be undone."
  // @Param        id path string true "知识ID" extensions(x-agent=knowledge_id)

and response types mark the fields the model may see with an
extensions:"x-agent" struct tag. Exposure stays opt-in at every level:
unmarked operations, parameters and fields are not reachable, so a new
field on Knowledge or a new query parameter never reaches the model by
accident. NewCatalog still enforces the safety rules (forbidden areas,
risk versus method, approval targets), and a test fails when the
annotations and docs/swagger.json disagree.

The twelve operations compile to the same routes, parameters and
results as before; document results now share one field set, and the
parse_status filter accepts deleting, which the API supports.

* fix(agent): enforce resume and wiki context scopes
2026-10-15 12:18:30 +08:00
lyingbug 234aa948db
docs(swagger): document the real request and response types (#4304)
* docs(swagger): add shared response envelope types

* docs(swagger): type knowledge base responses

* docs(swagger): add concrete response body types

* docs(swagger): type chunk list responses

* docs(swagger): type knowledge mutation responses

* docs(swagger): type wiki search and autofix responses

* docs(swagger): type FAQ responses

* docs(swagger): type tag responses

* docs(swagger): type message responses and SSE schemas

* docs(swagger): type agent responses

* docs(swagger): type remaining handler responses

* docs(swagger): regenerate OpenAPI spec

* docs(swagger): type knowledge, activity and wiki maintenance endpoints

Covers the endpoints left out of the earlier passes: knowledge base
detail, document list/detail/spans/reparse/cancel/delete, knowledge
base activity and the Wiki stats, issues, lint and rebuild-links
endpoints. The spans handler now writes the documented structs instead
of gin.H, so the schema cannot drift from the body; the JSON is
unchanged. List filters gain their real enums and bounds, including
the finalizing, cancelled and deleting parse states.

* docs(swagger): document JSON-valued fields as JSON, not byte arrays

types.JSON and json.RawMessage are byte slices, so swag rendered every
metadata, details and config field as an array of integers. A swag
overrides file documents them as objects; the four fields that hold
arrays (relation chunk ids, allowed origins, image refs) say so with a
swaggertype tag.

* chore(lint): exempt swag annotation lines from the line-length limit

swag reads each @Param, @Success and @Failure as a single line, so an
enum list or a composed response type cannot be wrapped to fit.

* chore(lint): exempt struct tags carrying swag attributes from the line limit

Struct tags cannot wrap either; a field documented with swaggertype or
extensions keeps its tags on the declaration line.
2026-10-15 01:41:42 +08:00
5quan 3fb6b9d631
fix(kb): let the update API set vlm_config (#3704)
* fix(kb): let update API set vlm_config

UpdateKnowledgeBaseRequest only carried name/description/config, so the
PUT /knowledge-bases/:id endpoint could never change a knowledge base's
multimodal (VLM) config, even though CreateKnowledgeBase accepts a
top-level vlm_config. Thread an optional vlm_config through the handler
and service so the update path mirrors create (nil = no change).

Fixes #1723

* test(kb): pass the vlm_config argument in tests added on main

* fix(kb): accept only model-managed vlm_config fields on update

The update API copied the request's VLMConfig wholesale, so a caller
could set the legacy ModelName/BaseURL/APIKey/InterfaceType fields and
have the VLM client call BaseURL directly, outside model management and
its SSRF checks. Only Enabled/ModelID/DescriptionLanguage/
CustomInstructions are taken now; legacy fields keep their stored value.
A referenced model must exist and be a VLLM model (400 otherwise), and
disabling clears ModelID, matching the initialization config endpoint.

* fix(kb): validate and gate vlm_config on the update API

- Enforce the 4000-character custom_instructions cap
  (types.ValidateKnowledgeBasePromptInstructions), as create and
  /initialization/config already do.
- vlm_config is a KB setting: require kbSettingsAccess (owner workspace
  or admin share) like /initialization/config, so share editors get 403.
- Stored legacy model_name/base_url keep IsEnabled() true on their own,
  so disabling returned 200 but left VLM on, and managed -> disabled fell
  back to the old endpoint. Clear them when the request enables a managed
  model_id, or disables VLM without echoing back the stored
  model_name+base_url (GET-then-PUT keeps legacy on). Request legacy
  fields are only compared, never written. Enabling without model_id on a
  non-legacy KB is now a 400 instead of a silent no-op.
- Document the vlm_config semantics in the KB API reference.

* fix(kb): keep the legacy VLM endpoint only for pure legacy knowledge bases

CreateKnowledgeBase stores vlm_config as sent, so a managed model_id can
sit next to legacy model_name/base_url; resolveVLM then uses the managed
model. The update's keep-legacy paths (disable echoing the stored legacy
fields, enabled=true without model_id) cleared model_id but kept the
legacy fields, so IsEnabled() stayed true and image calls moved to the
stored base_url.

Those paths now apply only when no model_id is stored. On a mixed config
a disable clears everything, and enabled=true without model_id is a 400,
the same as on any managed knowledge base.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-08 14:37:23 +08:00
tayadwx 68fe9599cd
feat(gallery): add an image gallery tab with schema-driven filters (#3694)
* feat(image): observe image attributes and let the policy decide OCR

Describe each image with the VLM as before, but ask it to also observe two
attributes about the image in the same round — how much body text it carries
(contain.text) and whether it presents data visually (contain.data_visual) —
then let a small pure function decide whether that image is worth a second,
separate OCR round.

Per-class OCR was the first shape of this idea; a class (chart / photo / logo)
is only an indirect proxy for 'is there text worth reading here', and the class
table has to grow whenever a new kind of image shows up. The decision actually
needs the observed attributes, so it reads them directly, and the attribute set
is a backend registry row rather than a new class.

The observation and the caption share one VLM call; OCR stays a separate call
(prompt isolation, see #3291). An attribute the model does not answer for is
left absent rather than filled with a conservative default, so the trace never
claims an observation that did not happen, and an incomplete observation falls
back to the on_unobserved policy (OCR still runs). The registry also carries the
panel's display text, so a new attribute needs no frontend change.

Off by default: image_attrs_enabled=false keeps today's behaviour (one caption
and one OCR per image). KB config: image_processing_config.image_attrs_enabled /
image_actions; per-upload override via process_config. Registry:
GET /api/v1/image-attrs/schema.

* feat(gallery): add image gallery tab with schema-driven filters

Adds a 4th knowledge-base tab parallel to Documents/Wiki/Graph that
browses every image asset in a KB.

Backend:
- GET /api/v1/knowledge-bases/:id/images lists image assets projected
  from chunks.image_info, with keyword (caption/OCR), multi-value
  attribute (attr_<name>=<value>), is_enabled and sort filters applied
  in memory, plus pagination.
- Resolves the source knowledge item title (SourceName) via a batched
  GetKnowledgeBatch so the viewer can show a human-readable source.

Frontend:
- ImageGallery.vue: thumbnail grid + search/sort + schema-driven filter
  panel (from GET /image-attrs/schema) + single-image viewer with
  prev/next, attributes/caption/OCR/source.
- Wired as the gallery tab in KnowledgeBase.vue; i18n keys in all 5 locales.

* feat(gallery): drive filters/search/sort from a configurable attribute contract

The gallery UI is no longer hardcoded: it renders from a self-describing
contract served by GET /knowledge-bases/:id/gallery-config (live attribute
sources, every attribute's definition with usage flags merged across the
config tiers, and the caller's search activation state).

- Attribute sources form an open registry (types.RegisterGalleryAttrSource);
  a builtin source models caption/ocr_text/timestamps/enabled so the gallery
  has zero special cases.
- Usage resolution: user > KB > system > source, field by field; the system
  tier lives in system_settings (gallery.policy, JSON, no rebuild), the KB
  tier slot is wired in the engine but not stored yet.
- Search activation is the user's own record: mode (all|custom) plus
  per-field on/off in users.preferences.gallery; source declarations never
  set defaults (unrecorded = off in custom mode).
- image_info is read through a gallery-local struct, removing the compile
  dependency on the attribute-pipeline types; the gallery builds and runs
  with or without that feature deployed.
- ListImages validates search_in / sort_by / attr_filters against the
  contract; ineligible fields are dropped, not served.

* fix(frontend): inline the shared graph-drawer stylesheet (@import instead of <style src>)

plugin-vue 6.0.6 keys unscoped src-style descriptors by the imported file
path, so AgentStreamDisplay.vue and WikiBrowser.vue — both importing
components/css/wiki-graph-drawer.less — overwrite each other during the
build and the second one crashes with "reading 'scoped'". Adding the
gallery changed the module order enough to make the collision hit every
time. Inlining the same stylesheet keeps the emitted CSS identical and
removes the shared descriptor key.

* feat(gallery): render attributes from short labels with descriptions in tooltips

A source could only say one thing about a value, so the filter panel had to put
a full sentence inside a checkbox. Attributes and their values now carry two
pieces of text: a short label for the space-constrained controls (checkbox,
search field, sort option) and a description for wherever there is room
(tooltip, detail panel).

The contract's value list becomes an array of {value, label, description}, and
the display helpers resolve each field separately: the gallery stamps the short
name on a control and moves the sentence into a tooltip. Translation overlays
are looked up per field, so a source that ships no gallery-specific text still
picks up the attribute pipeline's own wording.

* fix(gallery): let locale text win over the contract's wording

The filter panel read the backend's own label before asking the locale,
so a translated value never got a chance: the contract ships its own
default-language wording, and it shadowed the overlay for every locale.
Built keys were also off by a segment — imageAttr.<name> has to be
imageAttr.<name>.label, and the gallery's own keys live under
knowledgeEditor.wikiBrowser.gallery.attr — so builtin and system
attributes fell through to English on a Chinese UI.

* feat(gallery): top bar layout, pinned panels and per-value verdicts

The toolbar moves onto its own row above the grid, so the controls no
longer eat into the space the images are there for: search box, the two
scope switches, sort field and direction, and the result count all sit in
one flex row. The settings behind them are no longer inline either — each
opens as a popover under its control and can be pinned to a right-hand
rail, where the two panels take turns as tabs instead of both claiming
width at once.

Filtering gains a third state per attribute value. Where a value could
only be picked or not picked, it can now be "off" (hide those images),
"on" (show them whatever else says) or neutral (leave them alone), which
is what lets one attribute's rules talk about an image another attribute
tried to hide without the pair of them cancelling each other out: a
forced display outranks a forced hide, and anything no rule speaks about
stays visible.

The switch that used to only exist inside the panel now has a toolbar
home as well. "筛选:全显示" genuinely drops the attribute constraint
instead of imposing it against a panel the user cannot see.

Filtering used to empty the list the moment it was switched on. Any
image whose attribute was never observed — every image ingested before
the attribute pipeline existed — read as a mismatch for every filter,
which is not narrowing a list but losing it. Absence now stays silence:
an attribute no image carries places no constraint on it, until the
pipeline catches up and fills the value in.

Co-authored-by: WorkBuddy

* feat(gallery): expose the image attribute pipeline as a gallery source

The gallery's contract resolved to a single source, so the filter panel
could only offer "Enabled" while the attributes the observe-and-describe
pipeline fills in — contain.text, contain.data_visual — never reached the
UI. The registry is a gallery attribute source like any other: every
attribute it declares is offered to the gallery, filterable by default,
with the config tiers able to override the usage flags at runtime.

The registration lives in the attribute pipeline's own commit; this branch
carried an earlier, squapshot of that work which predated the split of an
attribute value into a short label and a description, so the registry's
value shape and this file have been brought in line with it. Both PRs now
hold byte-identical copies of the source hook, which is what lets either
one land first and the other merge cleanly afterwards.

Co-authored-by: WorkBuddy

* fix(gallery): viewer race, pin icon, panel anchoring and scope wording

Opening the viewer rendered the img element with the previous or empty
source while the proxy fetch was still producing a blob, and the error
that empty source fired locked the viewer in its failure state even
after the real image arrived. The resolved source now lands before the
flag is consulted, and a token keeps a slow fetch of one image from
overwriting a faster one that navigated past it.

The pin button drew nothing: TDesign has no "pushpin" glyph, so the
button rendered as an empty shell. It now uses pin / pin-filled, keeps
its place beside the tab bar, and a pinned rail assigns its columns
explicitly — the panel precedes the main section in the DOM, so leaving
the grid to source order handed the wide column to the panel and
squeezed the images into the rail.

The floating panel no longer appears at a hardcoded offset: it anchors
just below the arrow that summoned it, the arrow flips up while its
panel is out, a second press retracts, and any click elsewhere in the
toolbar closes it. When a rail is pinned the arrows turn to the
requested tab instead of spawning a second surface.

The scope words (全部 / 全显示 / 自定义) become plain status text rather
than a second dropdown — they name the current mode, and only the arrow
beside them opens the panel that changes it.

Co-authored-by: WorkBuddy

* fix(web): align the OCR-on-unobserved switch with the other setting rows

The switch sat bare inside its setting-row while every other switch on
the page is wrapped in setting-control, which is what reserves the right
column and centers the control vertically. With the row laid out from
the top, the bare switch hugged the row's upper edge and crowded the
description text — visibly out of place next to its neighbors. Wrapping
it in setting-control gives it the same right-aligned, centered slot as
the rest.

Co-authored-by: WorkBuddy

* fix(web): move the OCR-on-unobserved row out of the vertical schema row

The row lived inside setting-row-vertical, whose descendant rule stretches
every setting-info and setting-control beneath it to 100% width. Two
full-width flex children side by side cannot fit, so the control overflowed
the container and the switch landed past the right edge — visibly further
out than the switch on the row above. The row and its note now sit beside
the vertical block instead of inside it, under their own condition, so the
switch shares the ordinary row layout and its right edge lines up with the
"image attribute observation" switch above.

Co-authored-by: WorkBuddy

* fix(gallery): applying a verdict now activates the filter scope

Setting an off/on verdict while the panel's master switch was off
silently ignored it: buildParams only attaches attr_rules in custom
scope, and setVerdict never flipped the scope. A verdict the user just
expressed now switches the scope to custom; clearing the last meaningful
verdict returns to all, since custom with no rules constrains nothing.

* fix(gallery): search checkboxes stay visible and idle panels read as dimmed

The search panel hid its field checkboxes while the master switch was
on, so the only way to reach them was to flip the switch first. They now
sit next to the switch like the filter panel's verdicts do: dimmed in
all-field mode (where they carry no weight) yet clickable, and touching
one drops the mode to custom. The filter panel's verdict rows dim the
same way while the filter switch is off.

* fix(gallery): bulk verdict row, neutral default mark, panel polish

The clear-filters button never matched what the panel is for, so it is
now a bulk verdict row: one set of -/off/on buttons that applies the
chosen position to every value of every value-typed attribute, with the
default mark turned into a language-neutral dash. The panel head drops
its second divider (the tab bar already draws one) and the tabs follow
the toolbar's order: search left, filter right.

* fix(gallery): drop the all-fields caption and center the order arrow

The caption above the search checkboxes showed and hid with the master
switch, shoving the checkboxes down on every flip; it said nothing the
dimmed state doesn't, so it is gone. The order button's arrow is an
inline svg, which sits on the text baseline and reads as riding high;
it now centers against the label.

* fix(gallery): center the order arrow for real

vertical-align never applied: the icon is a flex item inside the
button's inline-flex text wrapper, and the wrapper's default stretch
pins the explicitly-sized 16px icon to the top of the 22px line box.
align-items: center on the wrapper is what actually centers it.

* fix(gallery): shorten panel tab labels in en and ru

The two-word labels overflowed the 300px panel and forced the tab bar
into scroll arrows; single-word labels fit in every locale that was
affected.

* docs: drop the image pipeline rewrite from this branch

The rewritten 'Go 侧接手后' paragraph belongs to the image attribute observation
work (#3635), which this branch carries as a prerequisite and which already
documents the caption_ocr / observation_driven split. Keeping a competing
rewrite of the same paragraph on the gallery branch makes it collide with
upstream, which extended that paragraph with the custom_instructions and inline
HTML table notes. Restoring the shared wording keeps the gallery branch free of
this file so upstream's edits merge cleanly.

* style: gofmt the gallery attribute types

Struct field tags in the resolved gallery config were one column narrow, so
gofmt rebroke the alignment block. Whitespace only.

* style(gallery): use design tokens in the gallery styles

The style guard ratchet fails on any new violation. ImageGallery.vue shipped
with 47 TDesign token fallbacks, 21 literal font sizes, 4 literal radii and a
0.15s transition; all of them now read the --app-* / --td-* tokens, which is
what the guard asks for and what dark mode depends on.

* lint(gallery): fix the golangci-lint findings in the gallery code

lll: the three over-long swagger @Param lines and one registry row were
shortened. ineffassign: ctx was assigned before the access check and then
reassigned, so the first value was dead; it is now declared once, after the
check. errcheck: the three unchecked c.Error calls in ListImages. revive: the
AttrSpec / AttrValue doc comment block was attached to the wrong type and split
into one block each. staticcheck S1016: the gallery registration converts the
registry value instead of restating its fields.

* lint(gallery): wrap the image query select and check the last c.Error

* style: reformat the wrapped image select in chunk.go

* fix(gallery): address review findings on ordering, KB switch and blob lifetime

- sortImageAssets: swap operands for desc instead of negating, keeping a
  strict weak ordering so ties page stably
- ImageGallery: reload on knowledgeBaseId change, drop stale list
  responses, fetch thumbnails in parallel with one cached blob per URL,
  revoke blobs on unmount, default sort to builtin:created_at
- KnowledgeBase: non-wiki KBs with a stale wiki/graph tab fall back to
  documents again
- handler: move ListImages godoc out of ListKnowledgeFolders' block and
  document attr_filters / attr_rules
- DecideOCR: a payload with no OCR conditions uses the built-in policy
  instead of silently skipping OCR

* docs(parsing): describe the caption_ocr and observation_driven image pipelines

Carried over from #3635, which this PR supersedes.

* fix(gallery): load local:// and other storage handles through the KB file proxy

The gallery's own scheme check only caught resource://, provider:// and
storage://, so local:// (the default storage), minio://, cos:// and s3://
images went straight into <img src> and never rendered. Resolve through
isProviderFileURL, the scheme list chat and wiki already use, and lock the
routing with a test.

* fix(kb-editor): keep API-customised OCR conditions on save and upload

The KB editor always wrote the registry default into image_actions.ocr.on,
so any unrelated edit replaced a knowledge base's custom OCR conditions
with the default table; the upload dialog did the same for the per-document
override. Keep a stored custom list and fall back to the registry default
only when there is none, and show the conditions actually in effect instead
of the default table in both panels.

* perf(gallery): de-duplicate, filter, sort and page images in SQL

Every gallery request used to read every image-bearing chunk of the KB,
OCR text included, decode it in Go and page in memory. A single query now
expands the image_info arrays, keeps one row per image URL, applies the
keyword / attribute / verdict / enabled filters and the sort, and returns
only the page with the filtered total.

The expansion carries narrow scalars only; filters and JSON sort keys are
evaluated on the winning entry of each image, and the page's JSON is read
back for its rows alone. Postgres runs the query with JIT off, since the
fixed 100-rows-per-call estimate of jsonb_array_elements otherwise prices
it ~100x high and compilation dominates.

On 20k images with 2 KB of OCR text each (40k chunks), a page drops from
~750 ms to ~100-200 ms on postgres and ~200 ms on sqlite; keyword search
over OCR text stays a full scan (~400-600 ms) until the text is indexed.
The filter semantics are pinned against both databases.

* perf(gallery): serve the gallery from a trigger-maintained chunk_images table

Reading images out of chunks.image_info meant parsing every image chunk of
the knowledge base on each request, so a page still cost time in
proportion to the KB. chunk_images (versioned/000113, sqlite/000032) keeps
one row per image_info entry with caption, OCR text and attributes as
columns; triggers on chunks keep it in step with every write path, and the
migration backfills existing chunks.

Copies of one image stay separate rows and the query lists the copy on the
most recently updated chunk through an indexed anti-join, so writes decide
nothing and concurrent writers need no locking. On postgres, caption and
OCR text get pg_trgm indexes when the extension is available.

On 20k images with 2 KB of OCR text each (40k chunks):
  postgres  first page 10 ms, page 500 41 ms, keyword 2 ms, filters 23 ms
  sqlite    first page  3 ms, page 500 39 ms, keyword 147 ms, filters 51 ms
down from ~100-600 ms. Writing those 40k image chunks costs 7.0 s instead
of 1.3 s on postgres (1.9 s instead of 1.0 s on sqlite), mostly the trigram
indexes; status and enable-flag updates patch rows in place.

---------

Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-25 12:33:59 +08:00
lyingbug 9fc08c2079
feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675)
* feat(retrieval): unify rerank across search APIs and expose diagnostics

The two retrieval APIs each missed half of the job: hybrid-search could
tune recall but never reranked, knowledge-search always reranked with the
tenant config and exposed no recall parameters. Rerank itself lived in
four places with drifting thresholds, passages and composite scores.

- internal/reranking: one rerank stage (passage = title + cleaned body +
  captions/OCR + generated questions, threshold with degradation and
  top-1 fallback, composite score, MMR) used by the chat pipeline, the
  agent search_knowledge tool and both APIs.
- hybrid-search: optional `rerank` object; absent keeps the old behavior.
- knowledge-search: vector/keyword thresholds, match_count, recall-path
  switches and a `rerank` object that can override or disable rerank.
- Rerank model resolution: request > tenant RetrievalConfig > first
  rerank model; an unknown requested model is a 400. Load or call
  failures degrade to retrieval order instead of failing the search.
- Responses carry meta.rerank (outcome, model, thresholds, top score) so
  an empty result explains itself (#2391).
- Go SDK and website-docs updated, including a "which API" section.

Refs #2976

* docs(retrieval): lead with which search API to use and when

knowledge-search is the default for callers; hybrid-search is the
low-level recall interface. Add a scenario table so the choice does not
hinge on reading the whole capability comparison.
2026-09-24 20:41:49 +08:00
lyingbug 5574ebc011
feat(knowledge): derive knowledge base descriptions from document profiles (#3370)
* feat(knowledge): derive knowledge base descriptions from document profiles

Knowledge bases only had a hand-written description, which most users
leave empty, so agents bound to several knowledge bases had nothing to
route on and searched all of them. This adds a generated description
that is derived from per-document data instead of maintained by a model.

Document profile: the summary call now returns JSON with a 2-3 sentence
summary plus a gist, 3-5 topic keywords, a document type and one
typical question (knowledges.profile). Input is capped at 8k chars and
output at 1024 tokens; plain-text replies from custom templates still
work and simply carry no profile. The summary chunk embeds the gist and
short summary.

Knowledge base aggregate: a database-only fold over those rows (counts
per file type, tag, normalized topic and doc type; stratified samples of
titles and questions; a hash of the inputs). Deleting, moving or
re-parsing a document changes nothing but the rows, so the next rebuild
is exact without any model judgement.

Description text: one small model call over the aggregate yields a
gist, merged topics and typical questions (knowledge_bases.
generated_profile). The call is skipped when the aggregate hash is
unchanged; failures keep the previous text and record the error; an
empty knowledge base clears it. The manual description is never
touched, and the agent runtime context renders both so the model can
pass knowledge_base_ids instead of fanning out.

Refreshes are requested from summary task exit, post-process completion,
deletion, cross-KB moves and enabling the option, debounced per
knowledge base in 30 s windows via kb:profile on the summary queue.
POST /knowledge-bases/:id/profile/generate regenerates synchronously.
profile_config (enabled, model_id, custom_instructions) opts a document
knowledge base in; the settings dialog shows the generated card with
regenerate and adopt-as-description actions in five locales. Search
results now log hits per chunk type so the value of indexing summary
chunks can be measured.

Migrations 000101 (PostgreSQL) and 000021 (SQLite) add the columns; the
SQLite schema test expects the new version and columns.

* fix(knowledge): keep generated descriptions out of duplicates and refresh clones

DuplicateKnowledgeBase copies the source row through JSON, which carried
generated_profile onto a knowledge base that has no documents yet, so
an empty duplicate advertised the source's topics. The duplicate now
starts without a generated description and keeps profile_config so it
derives its own once documents arrive.

CopyKnowledgeBase builds the target field by field and skipped
profile_config, and the clone task never asked for a description after
copying the documents, so a clone stayed undescribed until its next
upload. The target now carries a copy of profile_config and both clone
paths request a refresh on completion.

* fix(knowledge): refresh the loaded clone target and carry document profiles

ProcessKBClone requested the description refresh with the reservation
stub it builds before CopyKnowledgeBase runs. For the main create path
that stub has neither a type nor profile_config, so the refresh helper
rejected it and nothing was enqueued. The refresh now uses the loaded
target returned by CopyKnowledgeBase.

cloneKnowledge copied the summary text but not the structured profile,
so even a scheduled refresh would have aggregated titles without topics
until every clone was re-summarized. Cloned documents now carry an
independent copy of the profile.
2026-09-17 22:21:55 +08:00
lyingbug 3d81b977f1
refactor(resource-access): unify scoped authorization and transfer recovery (#3097)
* refactor(resource-access): unify scoped authorization and transfer recovery

* fix(resource-access): close reparse and transfer review gaps

* fix: enforce explicit file bindings and recover move cleanup

* fix: align mutation admission and cleanup scope checks
2026-09-08 15:36:33 +08:00
wizardchen 58acd8b33c feat(api): honor resource_urls=public on hybrid-search
Single-KB retrieval was still returning resource:// handles even when
callers asked for public URLs, unlike knowledge-search.
2026-08-26 17:40:54 +08:00
wizardchen cc846d8e29 fix: harden shared agent source selector validation
Reject invalid agent_source_tenant_id instead of silently falling back, fix a resolveAgent nil panic when a shared selector fails, and add regression tests for wiki write-tool filtering and agent resolution.
2026-07-30 12:28:23 +08:00
wizardchen ab9f705843 fix: harden shared agent source permissions 2026-07-30 12:28:23 +08:00
zhangzherui 7fde37eeeb fix(search): validate hybrid search query text 2026-07-23 12:05:57 +08:00
wizardchen a3a0308b62 feat(knowledge): add scoped activity audit trail 2026-07-23 10:50:32 +08:00
wizardchen f8b3e5f148 feat(storage): add multi-instance storage backends per workspace
Introduce tenant-scoped storage backend registry so workspaces can register
multiple storage instances and bind each knowledge base to a specific backend,
with migration from legacy tenant-level storage config and new settings UI.
2026-07-15 14:54:39 +08:00
wizardchen 41975abcc5 refactor(terminology): rename user-facing tenant labels to workspace
Align product copy, API docs, Swagger descriptions, and i18n strings
with workspace terminology while keeping internal tenant identifiers
and headers unchanged for API compatibility.
2026-07-14 16:14:43 +08:00
wizardchen 5eefa70e6f Enhance knowledge base configuration by normalizing and validating custom instruction fields. Implemented length checks for prompt instructions and added tests for validation logic. Updated GraphSettings to retain custom instructions when extraction is disabled. 2026-07-13 12:16:19 +08:00
wizardchen c1cddb4d4d Add instruction fields to knowledge base configuration 2026-07-13 12:16:19 +08:00
wizardchen d496d0fc73 feat(task-queue): split upstream workers into isolated core/enrichment/maintenance pools
Route summary/sync/maintenance tasks to dedicated queues and servers so parsing, enrichment, and batch work no longer share one concurrency budget, while extending the runtime dashboard and settings validation to reflect the new topology.
2026-07-10 23:51:29 +08:00
wizardchen 6ac1703198 Simplify KB duplicate API and improve naming/docs coverage.
Drop client-supplied target_id, generate IDs server-side, localize duplicate
names from request language, document the endpoint, and expand handler/client tests.
2026-07-08 20:55:54 +08:00
wizardchen 58b79876ae Add knowledge base duplicate flow 2026-07-08 20:55:54 +08:00
wizardchen 94d18ea15a fix(api-keys): close KB scope gaps in handlers and CloneContext
Enforce per-key knowledge base restrictions on batch/search read paths,
centralize scope checks in shared KB access validators, and propagate
TenantAPIKeyScope through logger.CloneContext so async QA cannot bypass it.
2026-07-07 12:00:13 +08:00
wizardchen 505bc7ddec feat: add scoped tenant api keys 2026-07-07 12:00:13 +08:00
wizardchen a17a2250ed fix(security): enforce KB access on wiki reads and related IDOR paths
Gate wiki GET routes and initialization config reads with KBAccessRead,
and bind async task progress endpoints to the caller tenant via task ID
parsing. Closes cross-tenant read paths reported in GHSA-ph8f-4qwx-89rr
and related initialization/progress leaks.
2026-07-02 12:06:22 +08:00
wizardchen abf6a99b90 fix(api): add POST for hybrid-search while keeping GET compatibility (#1727)
GET with a JSON body was non-standard and confused integrators. Register
POST as the preferred method and keep GET for existing clients; update SDK,
MCP server, and API docs accordingly.

Refs: https://github.com/Tencent/WeKnora/issues/1727
2026-06-22 17:22:35 +08:00
ochan.kwon e9980c6011 fix: deep-copy stored files and images when cloning a knowledge base
Cloning a knowledge base previously copied only the storage path strings
(knowledge.FilePath and chunk.ImageInfo.URL), so the source and the clone
shared the same physical objects in the storage backend. Once the original
file and extracted images are deleted on source removal, the clone is left
with dangling references and its document and images become unreadable —
data loss that occurs even for same-store clones.

Add a CopyFile primitive to the FileService interface and implement it in
every backend: server-side CopyObject on the object stores
(s3/obs/cos/oss/tos/ks3/minio), io.Copy on local, and a no-op on dummy.
Destinations use the knowledge-owned layout and reuse the existing
path/object-key guards; a sentinel ErrCrossBackendCopy is returned when the
source scheme does not match the backend.

Use CopyFile to deep-copy the document file in cloneKnowledge and the
extracted images in CloneChunk and cloneFAQKnowledgeBase via a shared
cloneChunkImageInfo helper that deduplicates identical image URLs per clone
and rewrites them to the new objects. Copied objects are cleaned up
best-effort if a clone fails partway through. A clone-time preflight rejects
cloning into a target bound to a different storage backend when the tenant
pins providers via StorageEngineConfig.

Adds unit tests for local CopyFile (independent copy survives source
deletion, traversal rejection, cross-backend rejection), cloneChunkImageInfo
(empty/multi/dedup/parse-failure/OriginalURL handling), and the storage
provider preflight.
2026-06-03 14:45:59 +08:00
ochan.kwon 8b7d00b260 feat: expose KB ↔ vector store binding in list, editor, and detail UI
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.

Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.

Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.

KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.

The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.

The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).

Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
2026-05-22 17:40:10 +08:00
ochan.kwon fd3d9f547c feat(search): fan-out KB retrieval across bound vector stores
Multi-KB hybrid search now groups KBs by their bound VectorStore (partition
key (storeID, owner_tenant_id)), retrieves in parallel via errgroup with a
SetLimit(4) cap and a per-group timeout (MULTI_STORE_RETRIEVE_TIMEOUT_SEC,
default 30s), and merges results. When the collected results span more than
one engine type, an EngineAwareNormalizer rescales vector scores to [0, 1];
keyword (BM25) scores pass through to the existing RRF fusion. Single-group
calls take the fast path with zero fan-out overhead, preserving today's
behavior for deployments where every KB has vector_store_id = NULL.

Embedding-model consistency is now enforced explicitly via
ResolveEmbeddingModelKeys. Multi-KB searches across KBs whose resolved
model identities differ return BadRequest instead of silently producing
incomparable scores. Cross-tenant Organization-shared KBs are preserved by
partitioning on KB.TenantID so the factory's ownership lookup runs against
the source tenant. Foreign-tenant KB UUIDs injected via the request body
are rejected via kbShareService.HasTenantKBPermission (Plan 3 of #1303,
3-D capped) before any retrieval; rejected scopes surface as 404 to avoid
leaking foreign KB existence.

Service-layer typed AppErrors (ErrVectorStoreBindingInvalid 2200 /
ErrVectorStoreUnavailable 2201) are mapped from PR2 sentinel hierarchy and
preserved end-to-end: the iterative FAQ path returns them rather than
swallowing, and the HybridSearch handler routes typed AppErrors to the
client unchanged instead of downgrading to 500.

Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 4 (Multi-store fan-out search).
Depends on #994, #1310, #1372.
2026-05-20 22:25:39 +08:00
wizardchen 8d4c58c5fa feat(creator_name): add creator_name field to agents and knowledge bases for enhanced visibility
Introduced a new optional field, creator_name, in both CustomAgent and KnowledgeBase types to allow the front end to display the creator's name. This enhancement enables better differentiation between resources created by the current user and those created by other members of the same tenant. Updated relevant API handlers to populate this field during list operations, ensuring accurate representation in the UI. Additionally, modified the ResourceOriginBadge component to utilize the creator_name for improved context in resource listings.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 8332a0dab6 feat(favorites): implement user favorites and recents functionality
Introduced a new API client for managing user favorites, allowing users to list, add, and remove starred resources. Enhanced the sidebar to display favorites and recents, improving user navigation. Updated internationalization files to include relevant labels and tooltips for the new features across multiple languages.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 928f5abe3f feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"

This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.

Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:

  effective = min(share.Permission, tenant_org_role, tenant_role_cap)

where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.

Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.

Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.

Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-18 17:28:58 +08:00
ochan.kwon 0e8de6192c feat(knowledge-base): validate vector store bindings on create, copy, and delete
Wires KnowledgeBase.VectorStoreID and the ownership-aware retrieve factory
into the user-facing knowledge-base lifecycle:

- POST /knowledge-bases validates the requested vector_store_id against
  the caller's tenant scope and the engine registry. New error codes
  ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable (2201)
  distinguish the typed branches without echoing UUIDs to the client.
- GET / POST / PUT / PUT-pin responses embed the bound store's display
  metadata (name, source, engine_type, status) without exposing any
  connection credentials. Cross-tenant shared KBs receive a suppressed
  payload (vector_store_id stripped, source="shared") so operator-chosen
  store names cannot be enumerated across tenants.
- POST /knowledge-bases/copy synchronously rejects clones whose target
  has a different embedding model or vector store, before the async
  clone task is enqueued. The async clone worker re-applies the same
  checks for defense in depth.
- DELETE /vector-stores/:id refuses to remove a store with bound KBs,
  inside a transaction that row-locks the store on PostgreSQL and
  serializes via WAL on SQLite. unregister-from-registry is wrapped in
  defer/recover so a panic surfaces as a structured warning instead of
  silently leaking a stale engine.
- vector_store_id is immutable after creation. The GORM <-:create tag
  blocks every ORM update path; the service-layer DTO omits the field
  entirely; a reflection-based regression test catches any future
  maintainer who adds it back to either layer.
- Empty-string vector_store_id is normalized to nil at both the create
  path and inside SharesStoreWith, so rows persisted by callers that
  did not run Normalize first cannot trip false same-store comparisons.

Part of #993. Depends on #994 and #1310.
2026-05-18 15:58:46 +08:00
wizardchen 4fb089d4d7 fix(kb): map ErrKnowledgeBaseNotFound to 404 across handler helpers
Five handler helpers (validateAndGetKnowledgeBase in knowledgebase.go,
validateKnowledgeBaseAccessWithKBID in knowledge.go, the kbService
guards in faq.go and tag.go, and getKnowledgeBaseForInitialization +
the per-kb config getter in initialization.go) wrapped every
GetKnowledgeBaseByID error — including the well-known
repository.ErrKnowledgeBaseNotFound sentinel — as
NewInternalServerError. The result was that every probe of a stale or
cross-tenant kb id surfaced as a 500 instead of the 404 it should have
been, both confusing clients ("real 5xx vs. wrong URL") and burning
ops attention on monitoring alerts.

The mapping pattern is the same as PR #1336 for sessions: detect the
sentinel via stderrors.Is and emit NewNotFoundError; everything else
still surfaces as a 500 so genuine DB / repo failures keep firing the
alerts that matter. Caught during the RBAC e2e smoke run on
feat/rbac, where a deliberate cross-tenant kb-id probe produced
HTTP 500 + body "knowledge base not found" — the smoking gun.

Tests: new internal/handler/knowledgebase_not_found_test.go covers
three cases — bare sentinel, fmt.Errorf("%w") wrapped sentinel
(regression guard against a future revert to `==`), and a non-sentinel
infrastructure error that must still 500. All three pass.

The full handler test package is green.
2026-05-14 20:12:41 +08:00
wizardchen d199628701 fix(agent): exclude wiki-only KBs from quick-answer (RAG) mode
Quick-answer agent mode retrieves purely through vector/keyword chunk
search and ships with no `allowed_tools`, so the existing capability
filter (which only reads from `allowed_tools`) let wiki-only KBs through
in every entry point. End result: users could @-mention, select, and
receive suggested questions from wiki-only KBs in quick-answer mode,
but the underlying retrieval always returned empty.

Treat "RAG-only" as an implicit property of `agent_mode = quick-answer`
and union it with the tool-derived filter. The same predicate is now
used everywhere the user can pick or be steered toward a KB:

Backend
- `tools.DeriveKBFilterForAgent` / `KBSatisfiesAgentRequirements`
  layer the implicit quick-answer requirement on top of tool derivation.
- `ListKnowledgeBases`, `SearchKnowledge` (shared-agent `@file`),
  `resolveKnowledgeBasesFromAgent` (chat runtime), `/search` IM command,
  and `GetSuggestedQuestions` now all use the agent-mode-aware variant.
- `GetSuggestedQuestions` also skips the wiki-page fallback for
  quick-answer agents to cover the `selected` / explicit-kb-ids paths
  where a wiki-only KB could still slip through.

Frontend
- `deriveKbFilterForAgent` / `kbSatisfiesAgentRequirements` mirror the
  Go helpers.
- `@` mention dropdown (`Input-field.vue`) uses the new helper.
- Agent editor's "specified KB" picker (`AgentEditorModal.vue`) grays
  out wiki-only KBs for quick-answer agents with a tooltip, and the
  pre-save warning fires for quick-answer mode too.
- i18n: add `agentEditor.agentType.kbMismatch.quickAnswer` across all
  four locales.
2026-05-12 16:27:28 +08:00
nullkey c753c67608 docs(api): add and fix swag annotations on handlers
将 swag 注解覆盖率从 ~88% 提升到 100%,并修复审计中发现的若干既有
注解 bug。

新增注解(24 个):
- im.go: 4 (UpdateIMChannel/DeleteIMChannel/ToggleIMChannel/IMCallback)
  其中 IMCallback 同时注册 GET 和 POST,两个方法都加了 @Router 声明。
- web_search.go: 1 (GetProviders)
- web_search_provider.go: 6
- wechat_qrcode.go: 2
- weknoracloud.go: 2
- knowledge.go: 2 (MoveKnowledge/GetKnowledgeMoveProgress)
- knowledgebase.go: 1 (ListMoveTargets)
- organization.go: 3 (ListSharedAgents/ListOrgAgentShares/RemoveAgentShare)
- mcp_service.go: 2 (SetMCPToolApproval/ResolveToolApproval)
- chunker_debug.go: 1 (PreviewChunking)

修复既有注解 bug(12 处):
- initialization.go: 10 个 @Router 路径与真实路由不符
  (e.g. /initialization/kb/{kbId}/config → /initialization/config/{kbId}
   /initialization/fabri/tag GET → /initialization/extract/fabri-tag POST)
- session/stream.go: ContinueStream 路径
  /sessions/{session_id}/continue → /sessions/continue-stream/{session_id}
- mcp_service.go: ResolveToolApproval 请求体 {approve: bool}
  → {decision: "approve"|"reject", reason?, modified_args?}

收敛响应 schema 到已有命名 struct(之前用 map[string]interface{}):
- knowledge.go MoveKnowledge → handler.MoveKnowledgeResponse
- knowledge.go GetKnowledgeMoveProgress → types.KnowledgeMoveProgress
- web_search_provider.go GetProvider/UpdateProvider → types.WebSearchProviderEntity
- initialization.go InitializeByKB → handler.InitializationRequest
- initialization.go 三个 model-test endpoint → handler.ModelTestRequest

修正 organization.go 中 errors.AppError 应使用 apperrors 别名(该文件
stdlib errors 未别名化)。

不补 /files 和 /api/v1/files/presigned 内联闭包路由(文件服务,非业务 REST)。

Refs #890 #1049 #1168
2026-05-12 13:16:58 +08:00
Windfarer 20be935fe0 Merge branch 'main' into storage-allow-list 2026-04-29 19:35:13 +08:00
Windfarer 6601a483be feat: add STORAGE_ALLOW_LIST env var 2026-04-29 18:58:09 +08:00
wizardchen beca2b89a3 feat(observability): extend Langfuse tracing across asynq pipeline
Previously the Langfuse integration only traced in-process HTTP requests
(chat / search / eval), so file uploads and every downstream asynq task
(document parse, chunk embedding, OCR/VLM, summary / question gen, wiki
ingest, datasource sync, etc.) produced either disconnected shallow
traces or no observation at all.

This change threads one trace end-to-end:

- tracer: add SPAN observation type and StartSpan; add ResumeTrace so a
  worker can attach to an upstream trace without emitting a duplicate
  trace-create; StartGeneration now auto-picks parentObservationId from
  ctx so nested trace -> span -> generation trees render correctly.
- types.TracingContext + LangfuseTracingCarrier: embed on all 17 asynq
  payloads so trace_id / parent_obs_id / user_id / session_id serialise
  into every job.
- langfuse.InjectTracing: injected at 28 enqueue sites before json.Marshal
  so the HTTP-layer trace survives the Redis hop.
- langfuse.AsynqMiddleware: mux.Use hook that peeks the payload, either
  resumes the upstream trace or opens a standalone asynq.<type> trace
  for scheduled jobs, and wraps the handler in a SPAN with task metadata
  (id / queue / retry / payload_bytes) plus ERROR level on failure.
- GinMiddleware.shouldTrace: whitelist ingestion / knowledge-mutation /
  FAQ / wiki / datasource endpoints so the root trace actually starts.
- Tests: tracer_test.go covers span nesting, error status, and
  ResumeTrace no-trace-create guarantee; asynq_test.go covers
  InjectTracing round-trip, middleware resume path, and standalone
  trace fallback.
- Docs: docs/Langfuse\u96c6\u6210.md now lists the covered task types
  and documents the cross-process propagation model.

No behavioural change when Langfuse is disabled (all new code paths are
no-ops and carriers serialise to empty strings with omitempty).
2026-04-24 13:16:47 +08:00
wizardchen 4c906e0431 feat: Enhance agent tool compatibility and knowledge base filtering
- Introduced a unified capability requirements system for agent tools, ensuring consistent filtering of knowledge bases based on their capabilities.
- Implemented derived `kb_filter` logic to streamline compatibility checks between agent tools and knowledge bases, reducing redundancy in configuration.
- Updated frontend components to reflect new filtering logic, including improved handling of empty states and user feedback when no compatible knowledge bases are available.
- Enhanced internationalization support by adding relevant messages for tool compatibility issues across multiple languages.
2026-04-23 17:28:00 +08:00
wizardchen b3484592e7 feat: Remove RebuildKnowledgeBaseIndex and associated handler from knowledge service
- Deleted the RebuildKnowledgeBaseIndex function from the knowledge service, which was responsible for reprocessing all documents in a knowledge base.
- Removed the RebuildIndex handler from the KnowledgeBaseHandler, eliminating the endpoint that triggered the rebuild process.
- Updated the router to remove the route for rebuilding the index, streamlining the knowledge base management functionality.
2026-04-22 21:18:31 +08:00
wizardchen 28deb8282f feat: Add comprehensive documentation for WeKnora knowledge base and wiki features
- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings.
- The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities.
- Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system.

These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
2026-04-22 21:18:10 +08:00
wizardchen 2d3b72e70e feat: enhance knowledge update functionality
- Added support for updating the description field in knowledge records, improving the detail and context of knowledge entries.
- Refactored knowledge base update logic to ensure configuration updates are only applied when provided, enhancing robustness and preventing unnecessary overwrites.
- Modified the request structure to make the configuration field optional, allowing for more flexible updates.

These changes improve the knowledge management system by allowing for more comprehensive updates and better handling of configuration data.
2026-03-12 14:47:56 +08:00
wizardchen 125d3e5e4a feat: implement knowledge move functionality
- Added API endpoints for moving knowledge items between knowledge bases, including validation for source and target compatibility.
- Implemented asynchronous task handling for knowledge moves, tracking progress and status updates.
- Introduced new localization strings for move-related UI elements in English, Korean, Russian, and Chinese.
- Enhanced the frontend to support knowledge move operations, including target selection and progress monitoring.

These changes significantly improve the application's capabilities for managing knowledge across different bases, providing users with a seamless experience in organizing their knowledge assets.
2026-03-09 13:23:16 +08:00
wizardchen 00bf0460cc feat: add pinning functionality for knowledge bases
- Introduced a new API endpoint to toggle the pin status of knowledge bases, allowing users to pin important entries to the top of the list.
- Updated the frontend to include UI elements for pinning and unpinning knowledge bases, with corresponding success and error messages.
- Enhanced localization support by adding translations for pinning actions in English, Korean, Russian, and Chinese.
- Modified the database schema to include `is_pinned` and `pinned_at` fields for knowledge bases, enabling persistent pinning status.

These changes improve the organization and accessibility of knowledge bases, enhancing user experience by allowing quick access to prioritized entries.
2026-03-05 18:14:14 +08:00
wizardchen 397689d2f3 feat: introduce WeKnora Lite edition with lightweight configuration and deployment
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.

This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
2026-03-02 21:21:49 +08:00
wizardchen 9a073d62bf feat(api): enhance organization shared knowledge base and agent functionalities
- Introduced new interfaces for handling shared knowledge bases and agents, including visibility and selection modes for agents.
- Added API functions to list shared knowledge bases and agents within an organization, improving data retrieval for UI components.
- Updated existing components to utilize the new API endpoints and interfaces, ensuring accurate representation of shared resources.
- Enhanced error handling in API calls to provide clearer feedback on failures.

This update improves the functionality and user experience by providing better access to shared resources and enhancing the overall data structure within the application.
2026-02-09 11:40:25 +08:00