Commit Graph

30 Commits (main)

Author SHA1 Message Date
lyingbug 0d7516ea0a
feat(assistant): expand management operations and polish interaction UI (#4427)
* fix(assistant): polish interaction cards and stabilize composer context

Localize operation labels across all six UI languages and expose answer copying in the assistant panel. Refine question and page-link cards with a single open/return action, and keep the current-page context row at a stable height.

* feat(assistant): add common data source, chunk and knowledge base operations

* feat(assistant): support parsing configuration and batch FAQ updates

* feat(assistant): add MCP tool directory query and sync operations

* feat(assistant): manage web search providers and data source sync settings

* feat(assistant): add model configuration copy operation

* feat(assistant): add workspace profile members and system information operations

* feat(assistant): add system administrator queries and runtime diagnostics

* fix(frontend): polish assistant and resource card UI

Refine assistant composer layout, context controls and reply actions, including narrow-window alignment. Unify knowledge base, agent and shared-space cards with consistent icons, hover feedback, favorites, menus and compact localized statistics.

* fix(assistant): preserve concurrent settings and exclude upstream error secrets
2026-10-17 14:35:58 +08:00
lyingbug 7115cb2a61
feat(assistant): trial runs, session grants, page links, and MCP/model/skill management (#4375)
* feat(assistant): let the assistant check MCP services and retrieval

Add a probe risk for operations that change nothing but do real work
(connecting to a service, running a retrieval). Probes use POST, run
without approval at every access level, and are capped per run.

Open three operations on top of it:
- mcp_service.test: connect with the saved settings and report why not
- mcp_service.list_tools: list what a service offers before binding it
- knowledge_base.search: raw hybrid recall with thresholds, to explain
  why a document is or is not found

x-agent can now name one method when a handler documents several, and
catalog parameters accept numbers.

* feat(assistant): let the assistant try an agent it set up

Add try_agent: the assistant asks an agent one question the way a user
would and gets back its answer, the tools it called and the documents it
cited, so it can check a setup before calling it done.

- The trial runs as the signed-in user in a new agent_trial session kind,
  which ordinary listings already exclude and the API cannot create, and
  the session is removed afterwards. Runs are non-interactive, so tools
  that would ask the user are refused. Long-term memory is untouched: it
  is written only by the HTTP handler.
- The assistant itself, the Wiki fixer and the skill installer cannot be
  tried, nor any agent granted the operation catalog.
- A trial spends model calls and runs the agent's own tools, so it is a
  write for the access level: confirm asks, auto_write runs, read_only
  does not offer it. The confirmation card names the agent and question;
  ConfirmedTool now lets a wrapped tool supply that text.
- At most 3 trials per turn, 4 minutes each; the tool timeout is raised
  above that. Answers, tool errors and references are capped.
- The runner is bound to the session service after the container builds
  it, since the session service depends on the agent service.

* feat(assistant): let the user allow similar changes for the conversation

The confirmation card for a recoverable change now offers "allow this for
the rest of the conversation". Once chosen, later calls of the same kind
in the same session run without asking, so a batch tidy-up no longer
needs a click per item.

- What a grant covers is derived on the server, never taken from the
  model: the same catalog operation on the same knowledge bases (a
  document's knowledge base comes from the target read) or the same
  agent; the same Wiki tool on the run's Wiki knowledge bases; trials of
  the same agent. Deletions and operations without such a scope, such as
  creating a knowledge base, always ask.
- No new storage: the gate looks for an earlier approved prompt in the
  session with the same grant and allow_in_session set, for the same
  tenant and user. Grants therefore survive suspend/resume and replicas,
  and end with the conversation. A failed lookup asks again.
- The response validator accepts allow_in_session only on approval and
  only when the prompt offered a grant, so a client cannot widen it.
- The card shows the scope in the user's language. The model is told
  such calls may run without a card and that deletions always ask.
- describeTarget now returns its target read so the grant reuses it
  instead of reading again.

* feat(assistant): tell the assistant about settings pages and chat answers

Pages now report which page is open and, for the agent editor, which
section. The model and MCP settings name the model or service open for
editing, and chat answers gain an "ask the assistant" button that
attaches that answer. The backend resolves these for the assistant only:
models and MCP services in the caller's tenant without endpoints or
credentials, and messages through the caller's session read access, with
the question, an answer excerpt and cited sources quoted as data.

* feat(assistant): link the user to pages and filled-in forms

The assistant gains open_page, which shows a button to a knowledge base
or document, a Wiki page, an agent's editor at a tab, a model or MCP
service to edit, or a new model or MCP service with its form filled in.
Targets are read through the operation catalog as the user, so a link
never points at something they cannot open. Drafts take no credentials
and only http(s) endpoints without user info; the card shows the
endpoint, and the user enters keys and saves the form themselves.

* feat(assistant): manage MCP services, model names and skills

* fix(assistant): prefill model drafts on initial navigation

* feat(assistant): document tags, model catalog reads, clearer cards, live page refresh

- Operations project lists of objects; knowledge.get/list report tags.
- model.get shows provider, endpoint and capabilities; new
  model.list_providers and model.get_provider read the model catalog,
  including vendors an admin added.
- open_page and ask_user guidance: offer page buttons whenever the next
  step is the user's, word choices for the user.
- Chat cards: redesigned open-page card pinned under the final answer,
  ask-user inputs in the dock style, approval card with aligned fields,
  translated labels and a question built from the operation summary.
- Open pages reload after assistant changes: deletes and moves wait for
  the background task, Wiki edits now report their changes, KB list
  counts, tags and chat model selectors refresh.
- The assistant panel stays open across reloads within the tab.

* fix(ci): format assistant operation code and wrap long lines

* feat(assistant): let the assistant see, fill and open the page beside it

- read_page / fill_page tools answered by the browser over a page_action prompt,
  with refs, redaction, in-place text edits, undo card and a page-access switch
- composer shows the open page, dialog and selection; a selection bubble and an
  editor toolbar entry ask the assistant about the selected text
- edits scroll into view with a highlight and an inline undo bar
- open_page reaches any settings section, toolbox tool and list page, and can
  navigate there itself to read it, keeping the button when edits are unsaved

* feat(assistant): let the assistant open the tab it needs

A visible tab can be selected and re-read, and system settings can open straight to user management, so the next step is no longer blocked on the wrong tab.

* feat(assistant): expose read-only conversation operations

Add session.list, message.list, message.search and
message.chat_history_stats to the operation catalog, restrict the
sessions and messages areas to non-mutating operations, and search
messages with the raw query instead of the log-sanitized one.

* style(chat): restyle open-page and page-edit cards

* fix(ci): comply with assistant card styles and Go line limits
2026-10-16 23:36:01 +08:00
lyingbug 4d4acec418
feat(assistant): manage agents and skills, separate history, refresh pages (#4344)
* feat(assistant): let the assistant manage agents

Expose custom agents to the operation catalog (list, get, create, copy,
update, delete) plus read-only model and MCP service lookups, so the
assistant can set up agents end to end.

- Add PATCH /api/v1/agents/:id. PUT replaces the whole config, so a
  one-field change from the assistant would wipe the rest; PATCH only
  overlays the fields it carries and reuses the PUT validation path.
- Add x-agent-detail so long fields such as the system prompt are only
  projected for single-object reads, not lists.
- Add label_param so a create without a target is labelled by its name.
- The approval card lists agent setting changes and resolves model,
  knowledge base and MCP IDs to names.
- Deny sharing, publishing channels, credentials and OAuth routes; model
  and MCP endpoints, headers and secrets are never projected.
- Update the assistant prompt, starters, API docs and Go SDK.

* feat(assistant): list skills and configure agent skill scope

- Expose usable skill names for an agent sandbox without exposing skill
  administration or connection settings to the model.
- Validate PATCH selections against the workspace sandbox usable set so
  invented names and missing sandbox selections cannot be saved.
- Show skill changes in approvals and align the assistant prompt, API
  documentation, and Go SDK with focused refusal and projection tests.

* feat(assistant): isolate conversation history

- Keep assistant conversations out of ordinary session lists, search, artifacts and delete-all cleanup.
- Add private, paginated assistant history with session switching, confirmed deletion and stale-session recovery.
- Backfill session kinds safely on Postgres and SQLite, preserve kinds on forks, and align API docs and SDK coverage.

* feat(assistant): refresh pages after successful data changes

- Carry decoded mutation targets through live tool-result events without
  exposing refresh metadata to the model or persisted chat history.
- Coalesce page and selector refreshes, update document drawers, and warn
  about changed agents while preserving open editor drafts.
- Cover mutation, approval, stream, lifecycle and cache behavior with Go
  and Node tests using the existing frontend runner and component fixtures.

* feat(assistant): add agent page context and resource scope limits

- Attach verified agent context and persist personal scope across turns and approvals.
- Enforce resource boundaries before operation and Wiki approvals, including edited arguments.
- Add scope controls, translations, API and SDK support, and regression coverage.

* refactor(assistant): remove the scope limit

The scope picker confused users and duplicated what RBAC and the access
level already enforce. Keep the agent page context: the agent editor
still publishes agent_id and the backend still verifies it.

* feat(assistant): render operation and product-doc results

Show find/run operations and WeKnora docs as structured cards instead of raw JSON.

* feat(assistant): default new agents to smart-reasoning

Quick-answer cannot call MCP, and agent.update could not change the mode. New agents now start in smart-reasoning, and a patch can switch agent_mode.

* fix: resolve assistant management CI failures
2026-10-15 21:43:20 +08:00
lyingbug 450b6ed1b7
feat(assistant): side panel, access levels and role-aware operations (#4342)
* feat(frontend): add dockable assistant side panel

* feat(assistant): access levels, role-aware operations and assistant settings

* fix(assistant): preserve approval and workspace access safeguards
2026-10-15 18:18:22 +08:00
Ochan Kwon e0712df3e0
feat(mcp): let callers without a browser obtain MCP OAuth tokens (#4229)
WeKnora runs behind other applications: they sign their users in and
call WeKnora with a tenant API key and X-External-User-ID, and WeKnora
keeps sessions and MCP OAuth tokens per external user. When an agent
uses an OAuth-protected MCP service, the only way to get that user's
token is the browser authorization code flow, and in this deployment
the user never sees WeKnora, so there is no browser to complete it.
Independently, an OAuth MCP service can only get its client through
dynamic client registration (RFC 7591), so authorization servers
without a registration endpoint cannot be used at all ("server does not
support dynamic client registration"), and servers that allow a grant
only to clients an administrator registered refuse a dynamic one.

JWT bearer grant. The application in front already knows who the user
is. POST /api/v1/mcp-services/:id/oauth/assertion lets it present a
signed assertion of that identity, such as the Identity Assertion JWT
Authorization Grant (ID-JAG) used by MCP's Enterprise-Managed
Authorization extension. WeKnora sends it to the service's
authorization server with the JWT bearer grant (RFC 7523) and stores
the token exactly like the callback does, so status, revoke, runtime
connections and, when the response carries a refresh token, refresh
use it unchanged. The assertion is forwarded unread; validating it is
up to the authorization server. The request reuses the principal from
authorize-url, the SSRF-safe HTTP client and the callback's connection
recycling, and the client the service already has. It does not run
dynamic client registration, which describes a redirect-based client;
without a client it answers 400. The token request outlives the caller,
so a token issued for an assertion that can only be used once is not
lost. A 4xx response other than 408 and 429, or a 2xx response carrying
an OAuth error, answers 400 with the OAuth error code and is logged; a
5xx, 408 or 429 response or a network failure answers 500. Request
logging masks the assertion field.

Pre-registered client. auth_config.client_id names a client registered
with the authorization server in advance, and client_secret is its
secret for a confidential client. When client_id is set it takes the
place of dynamic registration everywhere a client is read: the
authorization URL, the code exchange, the assertion grant and runtime
connections with their token refreshes. The values live on the service
rather than in mcp_oauth_clients, because a refresh rejected with
invalid_client deletes that row so the next authorization registers
again; an administrator's client would silently disappear. client_id is
non-secret: it is set through the main PUT, where an empty value clears
it, and returned in auth_config. client_secret follows api_key and
token: encrypted at rest, never returned, accepted in the create body,
and changed afterwards through the credentials subresource. Changing
client_id rebuilds the service's connections, and a connection built
with a pre-registered client is not treated as holding a replaced
registration when an authorization recycles connections. The MCP
service dialog adds optional Client ID and Client Secret fields to the
OAuth section, in all six locales.

Nothing changes for existing services: with client_id empty every
existing path reads the stored registration and registers dynamically
as before, and the authorization code flow, its state, the callback and
the dynamic registration request are untouched. The authorization
server HTTP client is moved into a function that both paths call.
2026-10-13 16:37:24 +08:00
lyingbug c3f077ed18
fix(models): align configuration and multimodal embedding workflows (#4080)
* fix(models): align configuration defaults and multimodal workflows

* feat(image): enhance image retrieval diagnostics and filtering

- Implemented `retrieveObservedImagePool` to observe engine calls and record image recall decisions without altering results.
- Added detailed diagnostics in `summarizeImagePool` and `summarizeImageCandidateCut` to track image filtering decisions and scores.
- Introduced tests for image pool diagnostics to ensure accurate filtering and decision recording.
- Updated existing functions to utilize new retrieval and summarization methods for improved observability and accuracy in image handling.

* feat(embeddings): enhance image handling with new formats and message support

- Introduced `EmbeddingImageFormat` to manage image input formats, allowing for both object and messages formats.
- Updated `EmbeddingsSettings` to include new fields for `ImageFormat`, `ImagePrompt`, and `TextAsMessages` to support multimodal embedding workflows.
- Enhanced `Client` methods to build request bodies for images in both formats, ensuring compatibility with vLLM's requirements.
- Added tests to validate the new image message handling and ensure correct request formatting for different scenarios.
- Updated documentation to reflect changes in image input handling and configuration options.

* feat(similarity): add embedding similarity mode for enhanced document scoring

- Introduced a new `similarity` option in `ModelDebugOptions` to score documents and an optional image against a query.
- Updated `ModelDebugDrawer.vue` to support a new input mode for similarity, allowing users to select similarity as an input type.
- Enhanced the UI to display similarity results, including ranked candidates based on cosine similarity.
- Added tests to validate the functionality of the new similarity mode in embedding models.
- Updated localization files to include new strings related to similarity mode in multiple languages.

* feat(embeddings): add Qwen3-VL-Embedding support for multimodal input

- Introduced Qwen3-VL-Embedding model with support for both text and image inputs.
- Updated model configuration files to include compatibility settings for the new embedding type.
- Enhanced tests to validate the functionality of Qwen3-VL-Embedding, ensuring correct handling of inputs and outputs.
- Updated documentation to reflect the new model's capabilities and usage instructions.

* test(models): wrap embedding assertions to satisfy lint

* test(opensearch): isolate HTTP transports between parallel tests
2026-10-10 11:06:00 +08:00
lyingbug 0c5d28abb7
feat: index and recall images with multimodal embedding models (#3683)
* feat(models): expose embedding input modalities

Whether an embedding model maps images into the same space as text is
what will decide if a knowledge base can store image vectors. The
catalog already had the field, but most multimodal embedding models
were missing it and embedding rows never reported it.

- Catalog: declare input modalities for the Doubao and Tongyi vision
  embeddings, the Aliyun multimodal / qwen-vl embedding families and
  jina-embeddings-v4, as listed in each vendor's documentation. The
  migration baseline changes for exactly these nine entries.
- Model responses resolve capabilities for embedding rows too, so
  capabilities.input reflects the catalog merged with spec.input.
- Model editor: an "Image input" switch for embedding models. It follows
  the catalog and writes spec.input only when the row differs from it;
  picking another catalog model or switching vendor drops the
  declaration. Model cards show the image badge for such models.

* feat(embedding): embed images with multimodal embedding models

Adds the image side of an embedder, for models that map images into the
same space as their text vectors so a text query can find an image.

- api: EmbedImage (bytes + MIME type, sent inline as base64 because the
  object store is usually unreachable from the vendor) and an
  ImageEmbedder implemented by all four protocols, each in the image
  shape its reference documents: Ark image_url, DashScope contents[].image,
  Gemini inlineData parts, and on the OpenAI shape an object keyed by the
  vendor's image_field (Jina: "image"). A plain OpenAI-compatible
  endpoint names no image field and takes no images.
- Settings: image_field, max_image_batch_size (default one per request),
  max_image_bytes and image_mime_types, filled from vendor docs for the
  DashScope multimodal models, gemini-embedding-2 and Jina.
- embedding: an optional ImageEmbedder interface reached through
  AsImageEmbedder, so existing Embedder implementations and fakes are
  untouched. An embedder takes images only when the catalog (or the
  row's spec.input) lists image AND the endpoint can carry one. Images
  are checked against the documented limits before any request, then
  split at the documented images per request. The concurrency, debug
  and Langfuse decorators forward it; logs and traces carry the MIME
  type and size, never the bytes.

The migration baseline changes only for the entries whose resolved
embedding settings now carry image limits.

* feat(knowledge): index images by their own vector and recall them

When the knowledge base's embedding model embeds images, each image now
gets a vector of its own next to its caption and OCR chunks, so a query
can find an image by what it shows even where the caption left it out.

Ingestion
- After the caption/OCR chunks are indexed, the multimodal task embeds
  the image (embedding.PrepareImage converts and shrinks it to the
  vendor's documented limits) and stores it as a new image_vector chunk:
  Content is the caption (OCR text when there is none) for reranking and
  the answer context; its index row carries the image's vector with
  SourceType ImageSourceType. Scanned PDF pages are skipped, their OCR
  text serves better. A failure here is logged and never fails the task.
- The image vector gets its own chunk rather than a second index row on
  the caption chunk: OpenSearch and Weaviate key documents by ChunkID,
  so a second row would overwrite the caption's text vector, and copy,
  move and enable/disable all follow chunk IDs.
- CompositeRetrieveEngine.BatchIndexVectors writes precomputed vectors
  through each engine's normal BatchIndex, keeping row order (the
  composite's de-duplication does not).
- Paths that rebuild vectors from text leave image_vector alone:
  syncChunkIndex only syncs its enabled flag, updateChunkVector skips
  it, and a caption edit updates its Content only.
- The multimodal task's caption/OCR index rows now set IsEnabled; only
  backends with a column default were storing them as enabled.

Retrieval
- A text query scores lower against an image than against text, so when
  the model embeds images the document vector search is widened by half
  (capped at 500) and run at min(VectorThreshold, 0.1); each store
  group's hits are then held to their own threshold before score
  normalization: text at VectorThreshold, images at min(VectorThreshold,
  0.1). Keyword hits on image rows are dropped, their Content being the
  already-indexed caption.
- image_vector hits are searchable and merge like caption hits: parent
  context is attached, and the caption is not repeated from ImageInfo.

Adds golang.org/x/image for scaling and WebP/BMP decoding.

* fix(models): report no thinking levels for embedding, rerank and ASR models

Embedding rows now carry capabilities, and the end-to-end run showed them
listing off/auto/minimal/low/medium/high: these references resolve no chat
protocol, and an empty level map reads as "every level supported". A model
without a chat protocol does not think, so its capabilities say so.

* fix(compaction): never read a cancelled turn's stream as a summary

streamSummary selects on the turn's context and the provider stream at
once. When the turn is already cancelled both are ready (providers close
the stream once their context ends), and select picks at random, so a
stopped turn could come back as a completed, partial summary with a nil
error. TestSummarizationIsNotRetriedOnceTheTurnIsCancelled caught it
intermittently in CI and reproduces under -race with several -cpu values.

Check the context before each receive and again when the stream closes.

* feat(knowledge): put image vectors behind a knowledge-base switch, off by default

Image vector indexing and recall used to turn on whenever the knowledge
base's embedding model declared image input. A model that takes images is
often chosen for text alone, so an upgrade would have added an embedding
call per image, widened vector recall 1.5x and lowered the image threshold
to 0.1 for those knowledge bases without anyone asking.

ImageProcessingConfig gains image_vector_enabled (default false; stored
JSON without the key reads as off, so existing knowledge bases stay off
with no migration). Both sides now need the switch and the model:

- indexImageVector returns before touching the model unless the KB opted
  in (KnowledgeBase.IsImageVectorEnabled, which also needs vector indexing).
- applyImageRecall resolves the embedding model only when some KB in the
  search opted in, and records the opted-in KBs per store group
  (storeGroup.ImageKBIDs). Only those groups widen the pool and lower the
  threshold; image hits of any other KB, including ones indexed before the
  switch was turned off, are dropped.

The KB editor shows the switch in the image processing section, with a
warning when the selected embedding model does not declare image input;
docs describe the switch, its cost and what turning it off does.

* fix(knowledge): leave image vectors behind when cloning into a KB that is off

CloneChunk copied every chunk of the source document and then CopyIndices
for all of them, so copying a document from a knowledge base with image
vectors on into one with the switch off (same model and store) still
created image_vector chunks and their index rows, bypassing the switch
that indexImageVector honours.

Image vector chunks are now dropped from the copy when the target has not
opted in. Every engine's CopyIndices copies only the rows of chunks in the
mapping, so their index rows stay behind too.

* fix(search): refill the vector pool past stale image vectors

Image hits of a knowledge base whose switch is off were dropped only after
the engine had ranked and cut its results at TopK. Image vectors indexed
before the switch was turned off could fill that pool and push out valid
text hits: with MatchCount 5 the pool is 50, and 50 stale image rows
ranked above the text left the search empty.

No engine can filter rows by source type (Postgres and SQLite do not even
take ExcludeChunkIDs), so the exclusion cannot move into the query. When
the document vector pool comes back full and holds stale image rows, that
search alone runs again on a pool twice as large, until it holds TopK rows
that are not stale, the index runs out, or the pool reaches
maxRetrievalPoolSize (at most four extra calls from a pool of 50). The pool
is then cut back to TopK such rows, matching an engine-side exclusion
unless more than maxRetrievalPoolSize-TopK stale rows rank above the text.

* feat(i18n): translate the image vector switch and image input keys into zh-TW

The zh-TW locale arrived from main after this branch added five keys: the
knowledge base's image vector switch (label, description, unsupported-model
warning) and the embedding model's image input flag (label, description).

* fix(knowledge): leave image vectors behind when moving into a KB that is off

A reuse_vectors move relocates a document's chunks and index rows in place:
MoveKnowledgeIndices rebinds the rows by document ID and
MoveChunksByKnowledgeID rebinds the chunks. Moving a document from a
knowledge base with image vectors on into one with the switch off therefore
carried its image_vector chunks and their vectors along, the same bypass the
clone path had.

When the target has not opted in, the image_vector chunks are now deleted
during the move, their index rows first and then the chunks, so a retry
finds the chunks again rather than leaving orphaned rows. A target that
opted in still receives them unchanged.

* fix(retriever): keep image vectors out of keyword-only engines

BatchIndexVectors handed image rows to every engine of the composite,
including one that serves keyword retrieval alone (Elasticsearch v7 next to
a vector store). There the row holds nothing but the image's caption, which
the caption's own chunk already indexes, and retrieval discards every
keyword hit on it.

Image rows now go only to engines that serve vector retrieval. An engine
that serves both keeps one row for both kinds, so its keyword search still
meets them; the search side handles that.

* fix(search): refill the keyword pool past image rows

Every engine that serves both kinds of retrieval stores one row per index
entry and runs its keyword search over that row's content, so an image
vector row, whose content is its caption, is a keyword hit wherever the
caption's own chunk is. filterImageHits drops all of them, but only after
the engine has cut its ranking at TopK, so they crowded out text and caption
hits; a keyword ranking that put the image rows first came back empty.

The keyword pool is now refilled the way the vector pool is refilled past
stale image vectors: when it came back full and held image rows, the
keyword search runs again on a pool twice as large, up to
maxRetrievalPoolSize, and is cut back to TopK hits that are kept.

Leaving the rows out of the keyword index instead would mean writing them
without searchable content (Milvus and Tencent VectorDB derive a BM25
sparse vector from it, and how they take an empty one is unverified) or a
source-type filter in every engine's keyword query (ten engines, and
filterable fields that existing collections lack).

* revert: let image vectors move with the document into a KB that is off

This reverts commit e259977fe915f7d0bc30c4b02bc7af04da1c8dd4.

Deleting the image vector index rows by chunk ID before the move is not
reliable: the Weaviate and Elasticsearch v8 delete wrappers ignore partial
failures, and OpenSearch rejects more than 1000 IDs with ErrBatchTooLarge.
A partial delete followed by the chunk delete leaves index rows whose chunks
are gone, which nothing ever cleans up.

Without it, the image_vector chunks and their index rows move together
with the document. While the target's switch is off they are filtered at
query time (the pool refill keeps them from crowding out text), they are
removed with the document on delete, reparse or knowledge base delete, both
chunks and index rows by knowledge ID, and they become active again if the
target turns the switch on. The clone path keeps dropping them, since there
the rows are copied rather than moved.

* fix(search): keep the image row refill from losing hits it already had

refillPool replaced a pool with the answer to each larger search. A refill
that answered only in part (RetrieveResult.Error, swallowed into a warning
by retainPartialResults, as when some collections of the store did not
answer) could hold fewer text rows than the complete smaller pool, or none:
40 stale image rows above 10 text came back from a partial refill as the
40 image rows alone, and the search lost all 10 text hits. A refill that
failed outright failed the whole store group, discarding a first round that
had succeeded.

A refill is now best effort. A failed or incomplete refill search stops
the refill and leaves the last complete pool in place, cut back to TopK
kept rows as before, so a refill can no longer leave a search worse off
than it was without it. Merging the partial answer into the pool was not
done: its rows would need deduplicating and its ranking is cut at a
different TopK over a different set of collections, while keeping the
previous pool is exact.

* docs(knowledge): say what happens to image vectors in a KB that is off and what the refill costs

The image vector section now says that with the switch off, image vectors
that were indexed before it was turned off, or that came along when the
document was moved in from a KB that has it on, are kept but filtered at
query time, are removed when the document is deleted or reparsed or the
KB is deleted, and are recalled again once the switch is turned on; a copy
into such a KB leaves them behind.

It also states the cost of keeping dropped image rows from crowding text
out of the pools: the vector and keyword pools are searched again with
TopK doubled up to the 500 candidate cap, which adds queries for store
groups that hold such rows, and text can still be missed when more than
about 500 candidates are image rows. A failed or partial refill keeps the
previous complete pool.

* fix(search): log unexpected refill result sets, and describe the refill stop condition precisely
2026-10-08 18:19:53 +08:00
zaqqmm 19542f2a1d
feat(datasource): add Seafile data source connector (#3616)
* fix(datasource): omit last_sync_cursor from the data source list response

The settings page polls GET /datasource every few seconds during a sync,
and connector cursors can grow to megabytes. No list consumer reads the
cursor, so NewDataSourceResponses now blanks it; detail, create and
update responses still return it through NewDataSourceResponse.

* feat(datasource): add Seafile connector with fingerprint-based incremental sync

Sync folders and files from one Seafile library per data source. The
connector walks the selected scope one directory level at a time,
downloads files through the link + fileserver flow without credentials,
and reconciles deletions against a per-file oid/mtime/size fingerprint
cursor that is checkpointed every 50 mutations.

- resource ids are "<repo_id>:<path>"; ancestors collapse, one library
- per-file failures become coded error items and keep the path pending
- directory failures and Retry-After > 60s abort without deletions
- forced full runs keep the old cursor as a deletion baseline
- API token never reaches cursors, metadata, logs or error text
- share the supported import extension set through internal/utils

* feat(datasource): add Seafile frontend connector and docs

Wire the Seafile data source into the editor dialog, channel labels
and the five locales, and document how to connect a Seafile 10.0.1
library.

- connector definition with base_url + API token fields and icon
- picker guard: one library per data source, at least one selection
- "library" resource type and icon in the lazy tree
- channel label in the knowledge base filter, document views and chat
- i18n for labels, guide steps and the eight seafile_* sync error codes
- website-docs integration guide, data source and extension point doc
  updates, and Seafile in the README data source lists

---------

Co-authored-by: zaqqmm <q-zhang10@qq.com>
2026-10-08 16:18:09 +08:00
wizardchen 6c83d7ab47 revert: take the plugin system back out of main
Reverts the eleven plugin system PRs merged on 2026-09-26 so they can be
reviewed before landing:

* a829d7cc9 agent tools served by plugins (#3733)
* 4f5d475c3 Jira example plugin (#3730)
* 01222aac8 dynamic form options, OAuth connections and resource limits (#3729)
* cf502ba46 event delivery and webhooks (#3728)
* 996696057 sandboxed plugin pages (#3727)
* da4ef68d9 standalone plugin hosts (#3725)
* 728007ada remote runtime, Python SDK (#3724)
* ff49f9491 Host API and document parsers (#3721)
* 4328ab26a code plugins (#3720)
* ed63e35cc declarative plugins (#3716)
* c91a8d11f plugin system foundation (#3711)

The tree is back to 47773b5bd, migrations included: no environment ran
the plugin migrations (versioned 000115-000119, sqlite 000034-000038).
2026-09-26 14:58:18 +08:00
lyingbug ed63e35cc7
feat(plugin): install declarative plugins — MCP servers, skills and model vendors (#3716)
* feat(plugin): install declarative plugin packages across nodes
* feat(plugin): serve installed plugins' MCP servers in each workspace
* feat(plugin): let installed plugins contribute skills and model vendors
* feat(plugin): register plugin skills into the workspace skill catalog
* feat(frontend): manage installed plugins and their configuration
* test(weaviate): read the class state before the probe barrier
* fix(plugin): keep MCP tool directories reachable behind the plugin wrapper
* test(plugin): wrap a long line
2026-09-26 14:44:34 +08:00
lyingbug e179f44445
refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589)
* refactor(models): separate providers, catalog and runtime configuration

* refactor(models): isolate provider definitions and runtime catalogs

* docs: exclude refactoring analysis notes from the pull request
2026-09-23 14:20:08 +08:00
Leo Dawn a3c7be10ca
fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475)
* fix(agent): scope the @Skill and @MCP pickers to the agent that will run

Both pickers listed the caller's own workspace, but a shared agent invokes its
OWNER's skills and MCP services. The two never line up, so for every shared
agent:

  * @Skill was always empty, in every skills mode. GET /skills resolves the
    sandbox config with (tenant, id), and the agent's config id does not exist
    in the borrower's workspace - effectiveTenantSkills short-circuits before
    it reads a single skill row. The response still reported
    skills_available: true, so the category appeared with nothing under it.
  * @MCP in "selected" mode was empty for the same reason: the preset holds the
    owner's service ids, which match nothing in the borrower's catalog.
  * @MCP in "all" mode was worse than empty. isMCPAllowedByAgent returns true
    for every enabled service, so the picker offered the BORROWER's own
    services; resolvePerRequestMCPScope then intersected the mention against
    the agent's preset, found nothing, and dropped it with only a warning.

Resolve the agent when the request carries agent_id + agent_source_tenant_id
and read its resources where they live. Only an explicit source workspace takes
the share path, so an own-agent request is byte-for-byte unchanged; built-in
agent ids exist in every workspace, and falling back to a share lookup without
one would let an org member take over the caller's default agent, exactly as
session.Handler.resolveAgent already documents.

Both endpoints serve only what the agent can actually invoke:

  * skills: the sandbox config comes from the agent, never from the query, so a
    borrower cannot name another config in the owner's workspace and enumerate
    it; "selected" narrows to the configured names and a disabled picker never
    reaches the owner's inventory at all.
  * MCP: the agent's explicit preset and nothing else, because that is exactly
    what resolvePerRequestMCPScope accepts for a shared agent. "all" therefore
    offers nothing rather than offering services every mention would drop.
    Whether "all" should become mentionable is a separate decision about that
    guard, not about the picker.

The cross-workspace MCP rows use a narrowed response built field by field.
NewMCPServiceResponse reveals endpoint URL, headers, env vars and stdio command
to an Admin, and a borrower can be an Admin of their own workspace - that shape
must be narrow by construction, not by role.

On the client the two lists now follow the selected agent: their cache keys
carry it, and the chat input refetches on a switch behind a request token, so a
slow response for the agent the user just left cannot overwrite the picker with
another workspace's services.

* fix(agent): check the c.Error return on the shared-agent picker paths

golangci-lint judges pull requests on the lines they introduce, so the
repository's older bare c.Error calls are grandfathered while these three are
not. Match the _ = c.Error form already used in mcp_service.go.
2026-09-21 13:02:34 +08:00
lyingbug c40a9dd194
refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470)
把集中式的 OpenAI 方言客户端重建为三层:协议层 internal/models/api/<protocol>(OpenAI Chat Completions / OpenAI Responses / Anthropic Messages / Google GenAI),目录层 internal/models/catalog,厂商层 internal/models/vendors/<id>(27 家,go:embed)。27 家厂商逐条对照官方文档核验,厂商事实以数据声明,协议层不出现厂商名。

既有行的运行时行为会变的四处:
1. extra_config.api 变为保留的协议选择器,非法取值被拒绝;
2. Azure 未填 api_version 的行改走 /openai/v1 GA 数据面;
3. api.openai.com 的一方流量改走 Responses 协议(中转/网关不受影响);
4. 7 家厂商的输出上限字段按文档纠正。

不需要数据库迁移。详见 PR 描述与 website-docs/03-features/06-models.md。
2026-09-20 21:28:23 +08:00
lyingbug 85b76d1a30
feat(mcp)!: persist tool catalogs and add guided configuration (#3118)
* feat(mcp)!: persist tool catalogs and add guided configuration

* fix(mcp): write caller-owned catalogs and require a real tool sync

OAuth viewers can now persist their own directory, and the setup wizard no longer treats an unsynced service as complete. Missing catalogs auto-fetch on the tools step; already-used MCP functions are restored across engine turns.

* fix(mcp): skip redundant list_servers and tighten catalog UI

When the advertised server directory already fits in discover_mcp_tools, models should list_tools or describe directly. Chat and settings now show server names, collapsed docs, and initialize debug metadata for empty optional fields.

* fix(mcp): surface catalog summaries and polish tool detail UI

List responses now include a saved catalog count without live MCP calls, so settings cards and @ mentions can show sync state. Tool details open in a popup instead of expanding the row.

* fix(mcp): drop incomplete Anthropic calls and gate shared catalog refresh

Unclosed streamed tool_use was emitted as {} and executed. list_tools refresh=true now re-lists upstream instead of returning the saved snapshot, and only Admin (or MCP-capable API keys) can overwrite tenant-wide static-auth catalogs.

* fix(mcp): wrap client refresh path to satisfy lll

The metadata refresh URL made doRequest exceed the 120-character limit in the client module CI lint.
2026-09-09 13:34:56 +08:00
wizardchen 4c0d42c4cb feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files
Refuse truncated tool calls, compact-and-retry once on overflow, and let
sandbox files be appended, paged, and batch-edited instead of rewritten.
Models can set context_window (default 200K) so compaction sizes itself
from the real window; agent max_iterations of -1 means unlimited.
2026-09-02 22:09:09 +08:00
wizardchen ffcb10f953 feat(memory): add cross-session long-term memory
Persist extracted and explicit memories across sessions with topic
resolution, semantic recall, document affinity, consolidation, workspace
settings, and inbox/interests UI.
2026-08-13 18:00:02 +08:00
wizardchen e1572b71b9 feat(models): let system admins edit builtin model config
Allow platform system administrators to update builtin model parameters
and credentials from the management UI while keeping secrets masked in API
responses. UI edits clear YAML ownership so startup reconciliation no
longer overwrites runtime overrides.
2026-07-14 15:56:02 +08:00
wizardchen 20256287ec feat(models): add max concurrency setting for background tasks
Introduce a new `max_concurrency` parameter to the model configuration, allowing control over the maximum number of concurrent background tasks (ingestion/enrichment) per model. This feature is applicable to chat, embedding, and VLM models, with a value of 0 or undefined falling back to the global default. Update UI components and localization files to support this new setting, ensuring a seamless user experience.
2026-07-09 20:40:51 +08:00
wizardchen d60614475e test(api-keys): add unit tests for tenant API key validation
Introduce unit tests for the tenant API key validation logic, ensuring that scoped keys require capabilities, while full-access keys are validated without them. The tests cover various scenarios, reinforcing the expected behavior of the API key validation function.
2026-07-07 12:00:13 +08:00
wizardchen c65ffa3d9c fix(api-keys): enforce scope from request context after attach
Use c.Request.Context() in authenticateAPIKeyRequest so scoped keys are
actually restricted, update dto tests for removed tenant.api_key, and add
a regression test for the attach-context contract.
2026-07-07 12:00:13 +08:00
wizardchen 505bc7ddec feat: add scoped tenant api keys 2026-07-07 12:00:13 +08:00
wizardchen fc64751f4e fix(security): close init config and parser-check redaction gaps
Strip model base URLs from viewer-accessible initialization config
responses and merge redacted parser secrets before engine check overrides
so admin test flows keep working after masked GET responses.
2026-07-02 14:41:42 +08:00
wizardchen 1a264e09c1 fix(security): harden secret redaction follow-ups for PR #1884
Restore PreserveIfRedacted merge on tenant KV PUT so Admin settings saves
no longer overwrite stored secrets with placeholder values. Redact cross-
tenant tenant listings, tighten Viewer MCP/model responses, and align init
config frontend types with credentials metadata.
2026-07-02 14:41:42 +08:00
wizardchen 512d2e5c25 fix(security): redact secrets in login and tenant KV update responses
Apply the same tenant/integration secret policy to auth login, OIDC
callback, switch-tenant, and invite-register payloads, and return masked
config objects from Admin-only tenant KV PUT handlers.
2026-07-02 14:41:42 +08:00
wizardchen 0b73080bb8 fix(security): redact integration secrets from viewer-accessible endpoints
Prevent low-privileged tenant members from reading raw API keys and
infrastructure credentials via tenant profile, KV config, model/MCP/web
search listings, and KB initialization config responses.
2026-07-02 14:41:42 +08:00
wizardchen f2686e9e39 feat(mcp): drive auth by AuthType and auto-detect OAuth on test
Select the auth header for the configured strategy only, instead of
emitting both X-API-Key and Authorization whenever the fields happened
to be set (which double-authed after switching strategies). AuthType=""
keeps the legacy infer-from-fields behavior for pre-existing rows.

Add a configurable api_key_header (non-secret, defaults to X-API-Key) so
services expecting the key in a different header work without plaintext
custom headers.

On connection test, upgrade a bare 401 that advertises RFC 9728
protected-resource metadata into an explicit OAuthRequired signal so the
UI can guide the user to switch to OAuth 2.0. Surface the server's
description from the initialize handshake in the test result.
2026-06-29 22:37:52 +08:00
wizardchen 3cb0e906a7 fix(datasource): harden RSS connector and editor UX after review
Scope auth headers to feed fetches, use feed-scoped external IDs and
content-based incremental fingerprints, tolerate partial feed failures,
store feed URLs in settings, and fix credential configured detection.
2026-06-23 17:30:38 +08:00
wizardchen 7f3f219f58 feat(mcp): add OAuth2 authorization for MCP services
Support connecting to MCP servers that require OAuth2, with a Cursor-like
zero-config flow: the user provides only a URL and the system performs
metadata discovery, dynamic client registration (DCR), and the
authorization code flow with PKCE on first connect.

- Persist per-user access/refresh tokens and per-service DCR client
  credentials, encrypted at rest (AES-256-GCM)
- Store the PKCE verifier/state during the flow via Redis with an
  in-memory fallback for Lite mode
- Cache MCP clients per (service, user) so tokens stay user-isolated
- Add endpoints for authorize-url, callback, status, and token revoke;
  the callback is allowlisted for unauthenticated access
- Surface a structured "authorization required" hint when a tool call
  hits an unauthorized MCP service
- Frontend: add an auth-type selector with OAuth config, inline
  authorization status/actions, and render the connection test result
  inline in the editor drawer instead of a stacked centered dialog
2026-06-22 20:19:31 +08:00
wolfkill 43565c5d1b fix: add model display name 2026-05-28 19:22:17 +08:00
wizardchen 7643dd457e refactor(credentials): introduce /credentials subresource pattern
Replace the legacy "redacted placeholder + Clear* boolean" pattern with
dedicated per-resource credential subresources across MCP services,
Models, Web search providers, and Data sources.

Why
---
The previous design had three problems:

1. Main PUT body carried secret fields. The frontend echoed back a
   redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty
   defense in the service layer tried to detect "user did not change this"
   vs "user wants to clear this". A regression in that defense (or a new
   frontend forgetting it) silently overwrites the stored secret with the
   placeholder.

2. The "remove this credential" UX was a red checkbox under a pre-filled
   password input. Three intents (preserve / replace / clear) collapsed
   onto one field, and credential changes were bundled with unrelated
   config edits in the same submit. Users wiped working keys by mistake.

3. Secret presence was inferred from "did the response come back with a
   '***' placeholder", which couples the contract to a magic string.

Design
------
Each of the four resources now exposes:

  PUT    /{resource}/{id}/credentials          # write one or more fields
  DELETE /{resource}/{id}/credentials/{field}  # clear a single field

"Is this configured?" metadata travels on the main resource response as
a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET
endpoint. The frontend reads the configured boolean from the main GET
and never sees secret values at all.

Main PUT endpoints now ignore any secret fields in the request body and
log a deprecation warning if they appear, so legacy clients fail loudly
rather than overwriting silently.

Frontend
--------
- New reusable <CredentialResource> component renders a three-state card
  (unconfigured / configured / editing) and drives the dedicated
  endpoints. Used by MCP, Model, Web search; DataSource has a bespoke
  card with the same behaviour because its credentials are a single
  atomic per-connector map.
- Cancel from edit mode now restores state synchronously from the meta
  prop. The previous async refresh() was a no-op while state was still
  'editing', leaving the input frozen open.
- Remove is single-click + toast. The danger-themed button is the
  deterrent; a modal confirm adds friction without adding safety (the
  plaintext is irrecoverable client-side either way — recovery means
  re-typing).

DTOs (internal/handler/dto/) are deliberately separate from the GORM
models so "no secret in response" is a compile-time invariant: a future
contributor cannot leak a secret without explicitly adding the field to
the DTO, which is review-able in a single diff.

Storage is unchanged — credentials still live in the existing jsonb /
parameters columns. No schema migration.

Cleanup
-------
- types.MCPService / types.Model / types.WebSearchProviderEntity /
  types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret
  / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData().
- utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty.
  RedactedSecretPlaceholder constant is retained because VectorStore
  still uses the old pattern and is out of scope here.
- Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat
  redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 15:27:52 +08:00