Commit Graph

214 Commits (ec66a3dac79a9f15eb7f44eaddffa0eafbd86928)

Author SHA1 Message Date
wizardchen 53e0333a53 Add embedding dimension override support 2026-06-15 19:18:19 +08:00
wizardchen 08852ab19d feat(embed): add widget, visitor chat, and management UI (#1657)
Add embed.html entrypoint, floating widget SDK, secure-mode snippets,
management panel with preview, visitor chat composables, citation rendering,
and integration docs for subdomain and server-side token exchange.
2026-06-12 23:10:39 +08:00
nullkey ac84dacc00 feat(frontend): add session source filter to sidebar (#1560)
IM-channel sessions were mixed in with the user's own chats. The sidebar
now defaults to web sessions; tenants with a configured IM channel get a
source switcher in the "对话" header to reach a specific platform.

- getSessionsList: optional `source` query param
- getMessageList: token-guarded loads (racing source switches / route
  changes can't append stale rows) + per-source pagination reset
- SourceSwitcherDropdown: scope dropdown mirroring KBSwitcherDropdown,
  scales to many platforms, reuses platform logos, closes on select
- empty state when a source has no sessions
- pure logic in sessionSourceFilter.ts with unit tests; zh/en/ko/ru i18n
2026-06-12 17:01:28 +08:00
wizardchen 2aadd61fcc feat: implement reparse functionality with process configuration support
- Enhanced the `reparseKnowledge` function to accept optional process configuration overrides, allowing users to adjust parsing settings during reparse operations.
- Updated the frontend components to integrate reparse capabilities, including the `UploadConfirmDialog` and `KnowledgeBase` views, providing a user-friendly interface for confirming reparse actions.
- Added new localization strings for reparse confirmation and process configuration details in multiple languages.
- Modified the backend API to handle reparse requests with optional configuration, ensuring proper validation and persistence of overrides.
- Introduced new types and interfaces to manage reparse-specific data structures in the store, improving type safety and clarity in the codebase.
2026-06-09 20:38:24 +08:00
wizardchen ed9e780c8c feat(client): enhance knowledge file upload with process configuration support
- Updated the `CreateKnowledgeFromFile` function to accept an optional `processConfig` parameter for per-upload parse configuration overrides.
- Introduced the `KnowledgeProcessOverrides` type to define various parsing settings, including parser engine rules and chunking configurations.
- Modified the API client and related components to handle the new process configuration, improving flexibility in knowledge file uploads.
- Updated documentation and API specifications to reflect the new `process_config` parameter in the upload process.
- Added a new `UploadConfirmHost` component to facilitate user confirmation for uploads, integrating with the updated upload logic.
- Enhanced user experience by providing detailed configuration options during the upload process.
2026-06-09 20:38:24 +08:00
wizardchen 4de3c3efbd fix(frontend): harden chat resources cache invalidation and dedup
- Invalidate the single-KB detail cache when editing a KB from the list
  page, matching the detail page so the sidebar/detail no longer show a
  stale name/config for up to the cache TTL.
- Reject (instead of swallowing) listModels network failures so the
  cache layer can tell a transient failure apart from a genuinely empty
  model list and avoid caching the empty result for the whole TTL. All
  UI callers already wrap listModels in try/catch.
- Guard the kb/agent in-flight handles with a generation counter so a
  concurrent force refresh cannot null out a still-running request, and
  clear all in-flight maps on invalidate() so stale requests cannot
  repopulate the cache after invalidation.
2026-06-09 18:05:33 +08:00
wizardchen dda16436d0 feat(lkeap): integrate Tencent Cloud LKEAP Rerank support
- Added support for Tencent Cloud LKEAP Rerank, including new configuration options for SecretId and SecretKey.
- Updated frontend components to handle LKEAP-specific fields and enhance user experience with appropriate labels and placeholders.
- Enhanced backend logic to manage LKEAP credentials and integrate with the new rerank API.
- Improved localization for LKEAP-related terms across multiple languages, ensuring clarity and consistency in user-facing messages.
- Introduced tests for LKEAP Reranker functionality to ensure reliability and correctness in handling requests.
2026-06-04 17:22:18 +08:00
wizardchen 226a37330c feat(system): add server startup time and uptime tracking
- Introduced `started_at` and `uptime_seconds` fields to the `SystemInfo` structure for tracking server boot time and uptime.
- Updated the `GetSystemInfo` handler to populate these new fields based on server runtime.
- Enhanced the frontend to display server startup time and uptime in the system information view, improving visibility for users.
- Added localization support for new labels and descriptions related to server uptime and startup time in multiple languages.
2026-06-04 15:48:04 +08:00
wizardchen bd1a764a91 feat(chat): add request info button and debug payload for stream requests
- Introduced a new `ChatRequestInfoButton` component to display detailed request information for chat sessions.
- Enhanced the `useStream` function to track the last stream request, including request ID, URL, method, and sanitized body.
- Updated localization files to include new strings for request information in both English and Chinese.
- Integrated the request info button into the chat interface, allowing users to easily access and copy request details.
- Improved handling of stream requests by attaching debug information to messages for better tracking and debugging.
2026-06-04 15:48:04 +08:00
langcaiye 2caa1e0508 fix: allow slower datasource resource listing 2026-06-04 15:32:06 +08:00
wizardchen ef1047bf67 feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements
Introduce opendataloader and PaddleOCR-VL parser engines with tenant-level
settings UI, replace liteparse, and harden Excel/PPT/Markdown parsing.
Optional odl-hybrid sidecar stays local-build only and is excluded from
default dev-start and full profiles.
2026-06-03 12:29:13 +08:00
ochan.kwon 40b74e2efa feat(retriever): activate OpenSearch k-NN driver (PR 3 of 3)
Phase 3 (#1440) gate flip. PR 1 (#1445) + PR 2a (#1481) + PR 2b (#1482)
laid the type prep + driver skeleton + read/write paths as gated dead
code; this PR wires every activation surface so opensearch becomes a
registerable VectorStore engine.

Activation wiring
- internal/types: validEngineTypes / GetVectorStoreTypes (with HNSW
  bounds + knn_engine enum + Immutable hints) / retrieverEngineMapping /
  buildEnvStoreForDriver — every gated surface now recognises
  "opensearch". IndexConfig grows four omitempty HNSW fields (HNSWM /
  HNSWEFConstruction / HNSWEFSearch / KNNEngine), keeping other engines'
  serialised config byte-identical.
- internal/container: createOpenSearchEngine + the switch case in
  createEngineServiceFromStore; the RETRIEVE_DRIVER=opensearch env path
  in initRetrieveEngineRegistry; NewEngineFactory now closes over the
  AuditLogService (the EngineFactory type itself is unchanged).
- internal/application/service/vectorstore_healthcheck.go: a
  testOpenSearchConnection case so CreateStore's connectivity probe
  accepts opensearch instead of returning 400.
- internal/application/repository/retriever/opensearch/transport.go:
  NewOpenSearchClient is exported so the factory and env path can build
  the TLS-hardened client; healthcheck.go reuses the unexported
  probeVersion / probeKNNPlugin for the service-layer probe.

Service-layer validation
- validateOpenSearchIndexConfig validates the HNSW caps (m 2-100,
  ef_construction 2-4096, ef_search 1-10000, knn_engine ∈ lucene|faiss).
  Shards/replicas continue to be enforced by the flat ValidateIndexConfig.
  Create-only: UpdateStore mutates the name only.
- validateConnectionConfig requires addr for opensearch.

Sync implementations (stubs.go shrinks)
- CopyIndices (copy.go) mirrors the Elasticsearch / Qdrant pattern —
  search → BatchSave with the source_id remap for generated questions —
  so dim/keyword routing and the source_id contract come from BatchSave
  for free. embeddingMap is keyed by the *target* SourceID because
  OpenSearch's BatchSave looks up embeddings by SourceID
  (lookupEmbedding), not by chunk_id (the ES driver's convention).
  Pagination is from/size; copies larger than max_result_window
  (default 10000) need the scroll-based async path that lands later.
- BatchUpdateChunkEnabledStatus / BatchUpdateChunkTagID (bulk_update.go)
  group the input by target value and issue one _update_by_query per
  group over the cross-dim <base>_* pattern. Caller values flow through
  bound script params only — never string-interpolated into the Painless
  source — closing the script-injection surface.
- inspectByQueryResponse (byquery.go) mirrors inspectBulkResponse: the
  full failure reason goes to the debug log only; the returned error
  carries the bounded id + type.
- UpdateByQueryParams.Refresh is *bool in opensearch-go v4.6.0 (the same
  shape as DeleteByQuery's quirk), so refresh=wait_for is not
  expressible; we use refresh=true.

Driver-owned audit (DIP)
- A new opensearch.AuditSink interface (with nopSink + WithAuditSink
  functional option) lets the driver emit opensearch.index_created and
  opensearch.reindex_executed events without importing any service
  package — the service layer implements the interface. NewRepository
  takes opts, so existing 4-arg test call sites keep compiling unchanged.
- internal/container/audit_sink.go bridges AuditSink to AuditLogService.
  When the context carries no tenant (the env-path registration ctx
  during boot, for example) the adapter skips the emit with a warning
  rather than silently writing tenant_id=0, which would collide with the
  system-scope sentinel.

Frontend + polish
- FieldSchema (frontend/src/api/vector-store.ts) gains min/max/enum/
  immutable. VectorStoreSettings.vue is now schema-driven: a closed
  `enum` renders a t-select; number inputs use the schema's `:min`/`:max`
  and fall back to the legacy replica-vs-shard heuristic only when the
  schema does not pin them; a danger-coloured warning fires when
  insecure_skip_verify is toggled on (the switch and warning are wrapped
  in a vertical stack so the warning sits on its own row below the switch).
- i18n: labels for hnsw_m / hnsw_ef_construction / hnsw_ef_search /
  knn_engine / insecure_skip_verify plus the warning copy in en-US,
  ko-KR, zh-CN, ru-RU.
- docker-compose.dev.yml: an opensearch profile (single-node 3.3.2 with
  security plugin disabled for dev only). OpenSearch Dashboards lives in a
  separate, opt-in opensearch-ui profile so the heavy UI container is not
  forced up alongside the cluster (the driver e2e is fully curl-verifiable
  against :9200). The new docs/dev/opensearch-integration-test.md covers the
  end-to-end exercise and the single-node guidance (set replicas=0 to keep
  the cluster Green).

Gating-guard tests flipped
- The "OpenSearch is NOT in validEngineTypes / mapping / types list /
  env builder / stubs" guard tests from PR 1 / PR 2 are replaced by
  their positive counterparts in this PR. The test suite was the
  activation checklist; the activation flip is its diff.

Backward compatibility
- Additive everywhere. IndexConfig's new HNSW fields are omitempty so
  other engines' serialised config is byte-identical. Existing
  Elasticsearch / Qdrant / Milvus / Weaviate / Doris / TencentVectorDB
  stores are untouched. No migrations.

Test plan
- go build ./... clean
- go vet ./... clean
- gofmt -l clean on touched files
- go test ./... — only TestOssEnsureBucket_CreateFails (Aliyun OSS
  endpoint), the docreader gRPC tests, and the doris SQL-shape tests
  fail; all three are pre-existing on upstream/main and untouched by
  this PR.
- New tests across internal/types, opensearch, service and container —
  including a full end-to-end env-path test that exercises
  initRetrieveEngineRegistry with RETRIEVE_DRIVER=opensearch against an
  httptest cluster.
2026-05-29 16:32:27 +08:00
wizardchen 44d6175559 feat: add knowledge parse cancellation with finalizing post-process state
Lets users stop an in-flight document parse to free up LLM / worker
resources without losing the chunks and index already written. The
core insight is that the previous parse_status=completed flipped as
soon as primary chunks landed, while the most expensive subtasks
(graph extract = N LLM calls per chunk, plus summary, question
generation) were still running in the background — so "completed"
wasn't actually terminal from a resource standpoint.

State machine

  pending -> processing -> finalizing -> completed
                              |
                              +-> cancelled (any of the three
                                            in-flight states)
                              +-> failed
                              +-> deleting

`finalizing` is the new post-process fan-out window. parse_status
only promotes to `completed` once pending_subtasks_count (a new
column tracking summary + question + per-chunk graph extract)
drains to zero via atomic FinalizeSubtask. Wiki ingest is
intentionally excluded from the counter — it's a KB-scoped
debounced batch and would otherwise pin parse_status in
`finalizing` for the wiki batch window.

Backend

- New ParseStatusFinalizing + pending_subtasks_count column with
  migration 000056.
- knowledgeRepository.SetFinalizing transitions processing -> finalizing
  conditionally so a racing cancel cannot be clobbered.
- knowledgeRepository.FinalizeSubtask atomically decrements the
  counter and self-promotes the row to completed when it hits zero.
- KnowledgePostProcess restructured to compute expected subtask
  count up front, flip to finalizing (or completed when no
  enrichment is enabled), and only then fan out subtasks. Subtask
  handlers (summary, question, graph extract) defer-decrement on
  terminal exit using the existing isFinalAsynqAttempt convention.
- New POST /api/v1/knowledge/{id}/cancel-parse handler accepting
  pending / processing / finalizing. Marks the row cancelled,
  zeroes the counter, best-effort dequeues asynq tasks via a new
  TaskInspector abstraction (asynq-mode walks pending/scheduled/
  retry queues; Lite-mode noop), and scrubs wiki ingest pending op.
- SpanTracker.AbortAttempt flat-sweeps every still-running span
  for the attempt via a new repo.CancelAllOpenSpans helper so the
  trace viewer's striped bars all flip to cancelled, even leaf
  generations whose parent stage already EndSpan'd (multimodal
  fan-out pattern). knowledge_post_process closes its postSpan
  via SkipSpan on the cancel/deleting entry guard so a worker
  that opens a span AFTER the cancel sweep doesn't leak it.
- Housekeeping and resetPendingTasks sweep finalizing rows
  identically to processing so a crash/restart can't strand them.
- DeleteKnowledge/DeleteKnowledgeList proactively dequeue
  downstream tasks via the same TaskInspector path.
- ChunkExtractService gets a cancel entry guard so the most
  expensive enrichment (graph extract) bails immediately when the
  parent knowledge is aborted.

Frontend

- New cancelKnowledgeParse API client + "Stop parsing" entry in
  both list view and card view more menus, gated on
  pending/processing/finalizing.
- Polling predicate refactored to a shared isParseInFlight helper
  that recognises `finalizing` (previously the doc list silently
  stopped polling once parse_status flipped from processing).
- Knowledge processing timeline: isPolling includes finalizing,
  new isHardTerminal short-circuits LIVE for cancelled/failed/
  completed so stranded child spans cannot pin LIVE on.
- DocumentListView.computeStatus distinguishes finalizing
  ("增强中") from completed and shows the previous "生成摘要中"
  copy when summary_status is still pending under finalizing.
  Added cancelled badge as well.
- i18n: statusFinalizing / statusCancelled / cancelParse* keys
  across zh-CN, en-US, ko-KR, ru-RU.

Docs / SDK

- docs/api/knowledge.md: documents the new finalizing state,
  cancel-parse semantics, and which statuses accept cancel.
- client (Go SDK): CancelKnowledgeParse with docstring listing
  the cancellable statuses.
2026-05-28 20:16:02 +08:00
wolfkill 43565c5d1b fix: add model display name 2026-05-28 19:22:17 +08:00
wizardchen 1d8ac301c2 feat(frontend): visualize document parsing pipeline with span timeline
Renders a five-segment timeline (DocReader → Chunking → Embedding →
Multimodal → PostProcess) above each document so users can see exactly
which stage a document is in and which one failed, instead of a flat
"Parsing..." spinner.

- New KnowledgeProcessingTimeline component (full + compact mode):
  full mode lives at the top of doc-content for any document not yet
  in `completed`; compact mode replaces the parsing/failed lines in
  the card hover popover with a five-dot mini-progress.
- Polls GET /api/v1/knowledge/:id/spans every 2s while
  parse_status in {pending, processing}; stops on terminal status or
  unmount. Hover popover instances opt out of polling to avoid
  amplifying requests on grid hover.
- Failed / cancelled steps expand into an error card with the raw
  error_message, an error_code -> localized title + suggestion mapping
  for the twelve canonical codes (DOCREADER_TIMEOUT, EMBEDDING_RATE_LIMIT,
  ...), and a "Copy details" button that pastes the failed span JSON
  for support tickets.
- Attempt selector renders only when latest_attempt > 1 so reparse
  history is browsable; the latest attempt is selected by default.
- Last-error block under the timeline includes a Retry button that
  calls reparseKnowledge() and immediately re-fetches.
- i18n strings added under knowledgeStages for en-US, zh-CN, ko-KR,
  ru-RU. ja-JP locale file does not exist in this repository.
2026-05-28 15:14:45 +08:00
lyingbug 3611a8607d
feat(rbac): add multi-use share-link invitations for invite_only mode (#1504)
* feat(rbac): add multi-use share-link invitations for invite_only mode

When the auth.registration_mode toggle flips to invite_only, the public
/auth/register endpoint returns 403 — but until now there was no
channel for unregistered users to actually join a tenant. The existing
invitation table required a registered invitee, so new emails could
not enter the system at all.

This change introduces a share-link model:

- Owner generates a multi-use registration link with a per-tenant role
- Recipient opens the link, registers with their own email, and is
  added to the tenant on the same request
- Token is stored in plaintext (not hashed) so the management UI can
  re-display the URL on demand without a "copy now or revoke" trap.
  Threat model is bounded by 7-day TTL, revocability, and the fact
  that all the link grants is membership in one tenant.
- accepted_count tracks how many users joined through the link so the
  Owner can tell whether a link is fresh or has already been spread

Per-user invitations (registered email -> in-app inbox accept) are
unchanged. Share-link rows use empty invitee_user_id as discriminator;
the partial unique index on (tenant_id, invitee_user_id) was relaxed
to skip empty values so multiple share links can coexist per tenant.

Frontend reuses Login.vue for /register?token=xxx instead of a
parallel page, and adds icon-only revoke / remove actions plus a
distinct "active" status tag for share-link rows.

Migration 000054 adds the token column, the accepted_count column,
and the relaxed pending unique index in a single step.

* feat(auth): update invitation token handling and add rate limiting

- Refactor the invitation token retrieval to use POST instead of GET, enhancing security by preventing the plaintext token from appearing in logs and browser history.
- Update the API endpoint from `/auth/invitations/:token` to `/auth/invitations/lookup` to reflect the new method.
- Introduce a rate limiter for unauthenticated share-link endpoints to mitigate brute-force attacks and abuse, ensuring a maximum of 30 requests per minute per IP.
- Adjust the registration flow to maintain user experience while securing the invitation process.
2026-05-27 19:42:01 +08:00
wizardchen acdc0a526f feat(system): expose platform audit log + polish audit drawers
The system_settings update, system admin promote/revoke, and
apply-default-storage-quota routes have been writing audit rows since
the prior commits, but with TenantID=0 (system-scope). The per-tenant
GET /tenants/:id/audit-log endpoint filters by tenant_id and never
returns them, so until now those rows existed only in the DB with no UI
surface. This commit closes the loop:

Backend:
- GET /api/v1/system/admin/audit-log: new SystemAdmin-gated endpoint
  reusing AuditLogService.List with tenant_id=0. Same cursor-paged
  shape and query params (after_id / limit / action / outcome / actor)
  as the per-tenant feed, so the frontend reuses the same client logic.
- Wired through RegisterSystemAdminRoutes (mounted on the existing
  adminRoutes group so it inherits the SystemAdmin() guard). The
  handler dependency is optional: nil auditLogHandler skips the route,
  mirroring RegisterTenantRoutes' /audit-log handling.

Frontend — new platform audit drawer in SystemSettings.vue:
- "审计日志" entry button in the section header opens a side drawer
  (880px) listing system-scope events. Lazy-loaded on first open;
  refresh is explicit via a button inside the drawer.
- Table columns: stacked date/time (so 50 events in the same minute
  remain distinguishable), stacked actor/role, action tag, structured
  target (subject key + diff line), outcome. The dead request column
  (system actions don't go through middleware path capture) is dropped
  in favour of richer target rendering.
- Per-action target formatters:
    * system.setting_changed: subject = registry key, diff = `old → new`
      (JSON-encoded, 80-char truncation). Reset shows `old → (空)`.
    * tenant_storage_quota bulk apply: subject = "批量同步", diff =
      "applied to N tenants (X GB)".
    * system.admin_promoted / revoked: subject = "name (email)", diff
      annotates idempotent / noop branches so an audit reader can tell
      a real grant from a probe.
- Click-to-expand row reveals the full audit context: actor UUID,
  target_user_id / target_type / target_id, and raw details JSON in
  monospaced scroll-capped block. No psql round-trip needed for
  forensic spot checks.
- Sticky thead pinned to the scroll container so column labels survive
  long scrolls. Cells vertical-aligned middle to keep single-line tag
  cells visually balanced against multi-line target cells. No zebra
  stripes — the stacked content already provides row separation, and
  stripes on top read as noise.

Frontend — same polish back-ported to TenantMembers.vue audit drawer:
- Same drawer width, stacked time / actor cells, structured target +
  diff layout, expandable raw-details row, sticky thead, vertical-
  align middle, no stripes. Refresh button reformulated as a text
  button with label (was an outlined square icon-only).
- request_path column kept (rbac.access_denied carries meaningful
  paths) but empty values render as a placeholder dash so they don't
  read as broken.
- Diff line now covers rbac.invitation_sent / invitation_revoked role
  in addition to the existing role_changed / access_denied details.

API:
- frontend/src/api/system/index.ts: listSystemAuditLog() reuses the
  AuditLog / ListAuditLogParams types from @/api/tenant/audit-log
  (re-exported) so consumers don't need to cross-import.

i18n (zh-CN / en-US / ko-KR / ru-RU):
- system.globalSettings.audit.*: full drawer copy + per-action labels
  (system.setting_changed / admin_promoted / admin_revoked) + target
  diff templates + expanded-row labels.
- tenantMember.audit.expanded.*: expanded-row labels added so the
  shared drawer treatment renders cleanly under tenant scope.
2026-05-26 21:13:56 +08:00
wizardchen 9cce0c8e5e feat(system): consolidate system admin and settings into one Settings panel
Replace the standalone /platform/system/* routes with a single
"系统设置" section inside the canonical Settings modal. The previous
SystemLayout.vue / SystemAdmins.vue surfaces are removed and their
functionality (system admin roster, global settings) is hosted directly
in SystemSettings.vue under the standard `.section-header` /
`.settings-group` skeleton. Legacy URLs redirect to the modal section
so external bookmarks don't 404.

Backend:
- SystemSettingService.Reset + repo.Delete: drop the DB override for a
  key so the 3-tier resolver falls back to ENV / built-in default.
  Idempotent (resetting a never-persisted key returns nil); emits an
  audit row only on real deletions, invalidates the local cache, and
  publishes to peers via the existing pubsub channel.
- TenantService.BulkSetStorageQuota: overwrite every tenant's
  storage_quota in one statement. Powers POST /system/admin/tenants/
  apply-default-storage-quota; bypasses the per-tenant whitelist on
  PUT /tenants/:id which intentionally forbids storage_quota edits.
- AuditAction{SystemAdminPromoted,SystemAdminRevoked} constants and
  emitAdminAudit() in SystemHandler — promote / revoke now leave an
  audit trail with TenantID=0 and {target_email,target_username,
  idempotent|changed} in details.
- SystemSetting.LastModifiedByName: derived per-request display label
  (username, email fallback) so the UI shows "wizardchen" instead of
  a UUID prefix without storing a denormalised column.

Frontend:
- SystemSettings.vue rewritten against the Settings modal skeleton with
  auto-persisting controls (switch/select @change, input/inputnumber
  @blur, tag-input + per-delta popconfirm for SSRF whitelist and admin
  roster). auth.registration_mode change goes through an inline
  popconfirm; cancel rolls back. Reset / bulk-apply also inline
  popconfirms — no dialog modal for these per-row affordances.
- Priority hint panel surfaces the DB > ENV > default resolver order
  so operators can reason about "I set the env but it doesn't show up".
- Router: /platform/system, /platform/system/settings, /platform/system/
  admins are now compatibility redirects to /platform/settings?section=
  system-global.
- Settings modal sized 900x700 → 1080x780 and content-wrapper 600 → 760
  so the wider tables (members, system settings) breathe; <1100px
  viewport still flexes to the screen.
- i18n: system.globalSettings.* keys for title / description / loading /
  empty / badges / priority hint / per-key labels / reset / bulkApply /
  admins (label, placeholder, save messages, popconfirm copy) across
  zh-CN, en-US, ko-KR, ru-RU.

Misc:
- internal/utils/filesize.go doc: clarify MAX_FILE_SIZE_MB is a
  deploy-time-only knob (nginx + docreader + frontend each cache the
  env at startup); a SystemAdmin UI override would mislead operators
  because nginx would still 413. Until all four layers can hot-reload
  the limit in lockstep, this stays env-only.
- internal/utils/security.go: SSRF whitelist parser/runtime now drives
  off SystemSettingService for live updates; ENV remains the fallback
  for never-overridden deployments.
2026-05-26 21:13:56 +08:00
wizardchen 47a183aa65 feat(system-admin): implement bootstrap for system admin promotion and enhance system settings management
- Added WEKNORA_BOOTSTRAP_SYSTEM_ADMIN_EMAIL environment variable to promote a specified user to system admin on startup.
- Introduced a new bootstrap process in `bootstrap.go` to handle the promotion logic.
- Updated `.env.example` to document the new environment variable and its behavior.
- Created new views for managing system administrators and system settings, including listing, promoting, and revoking admin privileges.
- Enhanced the frontend to reflect the new system admin features, including UI elements for admin management and settings configuration.
- Updated API interfaces to support system admin functionalities, ensuring proper data handling and user management.
2026-05-26 21:13:56 +08:00
helloandyzhang e6a469631b feat(agent): add intent prompt customization in agent editor
Allow per-intent system prompt overrides for non-retrieval intents in normal mode.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-26 20:41:10 +08:00
ochan.kwon 8b7d00b260 feat: expose KB ↔ vector store binding in list, editor, and detail UI
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.

Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.

Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.

KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.

The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.

The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).

Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
2026-05-22 17:40:10 +08:00
knight 513e589494 feat: add vLLM server URL configuration for MinerU
Add support for configuring vLLM server URL when using vlm-http-client or hybrid-http-client backend in MinerU.

Changes:
- internal/types/tenant.go: Add MinerUVLMServerURL field to ParserEngineConfig
- internal/infrastructure/docparser/mineru_converter.go: Pass server_url to MinerU API when backend is vlm-http-client or hybrid-http-client
- frontend/src/api/system/index.ts: Add TypeScript type definition
- frontend/src/views/settings/ParserEngineSettings.vue: Add vLLM server URL input field
- frontend/src/i18n/locales/*.ts: Add translations (zh-CN, en-US, ko-KR)
2026-05-20 16:33:12 +08:00
wizardchen 0cdb922d84 feat(tenant): remember user's last-active workspace across logins
Until now a fresh login (new device, expired refresh token, cleared
browser) always dropped the user into their home tenant, even if they
spend most of their time in a peer workspace. The session-local
X-Tenant-ID override in localStorage gave a "same browser sticky"
effect, but never crossed devices or new sessions.

This adds a small server-side preference, `users.preferences.
last_active_tenant_id`, persisted in the existing jsonb column (no
new migration), and threads it through:

* Backend
  * `UserPreferences` gains `LastActiveTenantID *uint64` with sentinel
    semantics (`*0` from the PATCH endpoint = clear preference).
  * `resolveLoginTenantID` validates the stored id (tenant still
    exists + active membership, or CanAccessAllTenants) and falls
    back to home on any failure, best-effort clearing the stale
    preference so subsequent logins don't pay for it again.
  * `Login` and `LoginWithOIDC` resolve once and use the result for
    both the JWT `tenant_id` claim and the returned `active_tenant`,
    keeping the two in sync. `RefreshToken` rides through
    `GenerateTokens` so refresh rotations also land the user back in
    their preferred tenant instead of bouncing to home.
  * `UpdateUserPreferences` learns to merge the new key.
  * `PUT /auth/me/preferences` accepts the new field.

* Frontend
  * `Login.vue` now writes the user's HOME tenant id into
    `user.tenant_id` (matching the field's documented semantics) and
    expresses any active-vs-home divergence via `setSelectedTenant`,
    so `useHomeTenant` and the "current"/"home" badges stay correct
    after the backend honours a remembered preference. `App.vue`'s
    OIDC sync does the same reconciliation.
  * `TenantSelector`, `UserMenu` and the post-tenant-create handlers
    fire `persistLastActiveTenantPreference` after every successful
    user-initiated switch (switching to home sends `0` to clear).
    The call is raced against the existing reload-grace window so
    most writes finish before the page tears down; lost writes are
    recoverable on the next switch.

No new UI. Users will simply notice that, after re-logging in on
another device, they land back in the workspace they were last
using rather than always in their home tenant.

Note: `make docs` is unrelated-broken on `main` (audit_log.go
references `errors.AppError` which swag can't resolve), so the
Swagger artifacts under docs/ are intentionally not regenerated
in this PR. The handler code is the source of truth.
2026-05-18 20:37:57 +08:00
Li Xianggang b3dbf86bf5 feat(obs): 支持华为云obs存储 2026-05-18 19:38:23 +08:00
wizardchen 68ac7fb973 feat(user-preferences): implement user preferences management with backend support
Added functionality for managing user preferences, including the ability to update preferences via a new API endpoint. The preferences are now stored server-side, allowing for synchronization across devices. Updated relevant components to handle user preferences, including the UserMenu and settings views. Enhanced internationalization support for error messages related to preference updates.
2026-05-18 17:28:58 +08:00
wizardchen b0b05b49c5 feat(tenant): enhance tenant information update with description field and add global invitation bell component
Updated the tenant information API to allow editing of both name and description fields, improving user experience in managing tenant details. Introduced a new GlobalInvitationBell component to display pending invitations in the UI, ensuring users can easily access their invitations without navigating away from their current context. Additionally, updated internationalization files to support new description-related messages.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 8d4c58c5fa feat(creator_name): add creator_name field to agents and knowledge bases for enhanced visibility
Introduced a new optional field, creator_name, in both CustomAgent and KnowledgeBase types to allow the front end to display the creator's name. This enhancement enables better differentiation between resources created by the current user and those created by other members of the same tenant. Updated relevant API handlers to populate this field during list operations, ensuring accurate representation in the UI. Additionally, modified the ResourceOriginBadge component to utilize the creator_name for improved context in resource listings.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 884ac44283 feat(invitations): implement pending invitations feature with polling and dialog
Added functionality for managing pending invitations, including a polling mechanism to update the invitation count in real-time. Introduced a new dialog component for users to view and respond to their invitations without navigating away from their current context. Updated API endpoints for fetching and managing invitations, and enhanced the user interface with relevant internationalization support.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 8332a0dab6 feat(favorites): implement user favorites and recents functionality
Introduced a new API client for managing user favorites, allowing users to list, add, and remove starred resources. Enhanced the sidebar to display favorites and recents, improving user navigation. Updated internationalization files to include relevant labels and tooltips for the new features across multiple languages.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 7d030a6f6a feat(tenant): implement tenant creation limit and error handling
Added a new configuration option to limit the number of tenants a non-superuser can create via self-service. Introduced a new error type for handling cases where users exceed this limit, returning a 429 status code. Updated the tenant creation and update handlers to enforce this limit and provide appropriate feedback to users. Additionally, refactored the tenant update request structure to ensure only mutable fields are allowed.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 29dc99ae79 feat(tenant): implement self-service tenant creation functionality
Added a new API endpoint for creating tenants, allowing any logged-in user to create their own workspace. The backend automatically assigns the creator as the Owner of the new tenant. A new CreateTenantDialog component was introduced in the frontend, enabling users to input the workspace name and an optional description. Internationalization files were updated to include relevant messages for this feature across multiple languages.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen b2411542eb feat(tenant): add tenant name update functionality and enhance UI
Implemented a new API endpoint for updating tenant names, allowing owners to edit tenant information directly within the UI. The frontend now supports inline editing with success and error notifications. Additionally, internationalization files have been updated to include relevant messages for the new functionality across English, Korean, Russian, and Chinese languages, ensuring a consistent user experience.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen dc8d033b4d feat(organization): transition membership model to tenant-level and enhance invite functionality
Updated the organization membership model to operate at the tenant level, aligning with Plan 3. This change introduces new interfaces and modifies existing ones to accommodate tenant-centric operations, including the `InviteMemberRequest` and `TenantInviteCandidate`. The search functionality for inviting members has been updated to reflect this shift, allowing for tenant-based searches instead of user-based. Additionally, internationalization files have been updated to ensure consistent messaging across languages regarding tenant membership and roles.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen cf6c28fe7e fix(rbac-ui): identify org owner row by tenant_id, not user_id
Plan 3 (#1303) lifted Org membership from per-user to per-tenant, and
40fdb978 pinned the org's owner tenant in DB as organizations.owner_tenant_id.
The OrganizationSettingsModal members list, however, still decided
"which row is the owner" by comparing member.user_id against
orgInfo.owner_id. After Plan 3 the members list is keyed on tenant_id,
so the user-id check is structurally wrong: the representative user of
the owner tenant happens to be the owner today, but if that user is
moved to another tenant — or a different user of the owner tenant
becomes the representative — the modal would either lose the "(owner)"
tag, or re-tag a non-owner row, and would offer the role-select /
remove-button affordances on what is in fact the owner tenant row.

This change:

  - Adds OwnerTenantID to OrganizationResponse and populates it in
    toOrgResponse from the persisted column. IsOwner now compares the
    caller's current tenant against owner_tenant_id (with a user-id
    fallback for legacy rows where owner_tenant_id == 0).

  - Adds owner_tenant_id to the frontend Organization type and a small
    isOwnerMember(member) helper in OrganizationSettingsModal, which
    tenant-matches by default and falls back to user-id only on
    pre-000046 rows. The role <select>, role tag suffix, and the
    delete button on member rows now all use the helper.

The "is-me" CSS class and the me-tag in the member name still match
on user_id — that one really is a per-user marker ("this row's user is
me, the signed-in user").
2026-05-18 17:28:58 +08:00
wizardchen 928f5abe3f feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"

This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.

Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:

  effective = min(share.Permission, tenant_org_role, tenant_role_cap)

where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.

Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.

Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.

Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-18 17:28:58 +08:00
wizardchen 1e8b5c8d40 refactor(api): drop deprecated tenant ConversationConfig field and KV write path
The Tenant.ConversationConfig field was marked deprecated when
CustomAgent (builtin-quick-answer) replaced the per-tenant defaults
path, but the JSONB column, GET/PUT KV handlers, write helpers and
read fall-throughs all stayed wired. This had two concrete failure
modes:

  - Input-field.vue silently PUT-ed per-user model picks back into
    /tenants/kv/conversation-config, which requires Admin+ — Viewers
    and Contributors got a 403 every time they switched chat models.
  - The same write also overwrote the tenant-shared default for every
    other user, turning a per-user UX into a per-tenant footgun.

Remove the field, the struct, the validate helper, the PUT case and
the two dead Summary helpers in session/helpers.go. KnowledgeSearch
loses its tenant-conversation fallback (one chunk-tool reader); the
global config + hardcoded defaults still cover it. session_agent_qa
already required agents to declare their own rerank_model_id, so no
behaviour change there.

Frontend follows:
  - Input-field.vue: per-user 'last selected chat model' moves to
    localStorage, no API call.
  - AgentEditorModal: defaults for builtin-quick-answer creation now
    come from prompt-templates (system/context/rewrite/fallback) and
    retrieval-config (top_k / threshold), the two endpoints that
    actually own those settings now.
  - AgentSettings.vue: orphan settings page with no router/import
    referencing it; deleted.
  - api/system: getConversationConfig / updateConversationConfig and
    the ConversationConfig interface go away.

Net: 137 insertions, 2761 deletions.
2026-05-18 17:28:58 +08:00
wizardchen b9a1ce7617 fix(rbac): drop X-Tenant-ID short-circuit in SSE / fetch paths
The previous PR removed the
"selectedTenantId === weknora_tenant.id → skip header" short-circuit
from the axios interceptor, but the same copy-pasted logic lived in
three more places that bypass axios:

- `api/chat/streame.ts` — `fetchEventSource` for the streaming chat
  endpoint. Without this fix the chat reply would 404 with
  "流式连接失败:HTTP 404" on the first message after switching to
  a non-home tenant: the SSE call landed on the home tenant where
  the freshly-created session did not exist.
- `api/initialization/index.ts` — native `fetch` for multimodal
  test (FormData) requests. Misroutes the test to the home tenant.
- `utils/security.ts` — `getProtectedFileRequestHeaders` used by the
  Markdown image proxy. Authenticated file fetches landed on the
  home tenant, breaking image rendering after a tenant switch.

All three now mirror the simplified rule: if `weknora_selected_tenant_id`
is set, attach `X-Tenant-ID` unconditionally. The server's
`IsTenantAccessible` already accepts a header pointing at the user's
home tenant, so always attaching is safe.

Same root cause as the axios fix: any code path that writes the
active tenant into `weknora_tenant` (UserMenu's `loadUserInfo`, OIDC
sync, router hydrate) made the two values equal, after which these
non-axios paths silently dropped the header and re-routed to home.
2026-05-18 17:28:58 +08:00
wizardchen 1a46fcf8ca fix(rbac): role-aware UI gates leaked write affordances after tenant switch
When an Owner of tenant A invited user B as a Viewer, B could log in,
switch the active tenant from B's home to A via the user menu, and
still see every write button — Create KB, Upload Document, Edit/Delete
agent — because the role-gate plumbing on the client was wired to the
wrong tenant. The server's RBAC enforcement (PR 2 / PR 5) still blocked
the actual mutations with 403, so no data leaked, but the UI presented
authority the user did not have, which is a confusing and embarrassing
defect.

Five distinct root causes, all on the client / server boundary:

1. `currentTenantRole` (frontend/src/stores/auth.ts) read
   `tenant.value.id` — the user's *home* tenant — instead of the active
   tenant id. After switching to A, the helper still returned B's home
   role (Owner), so every `hasRole(...)` gate in the codebase silently
   short-circuited. Fix: read `selectedTenantId` first, fall back to
   `tenant.id`.

2. `setMemberships` was only called once, on /auth/login. Hard reloads
   (which the tenant switcher triggers) re-ran `getCurrentUser()` via
   `App.vue` and `router/index.ts`, but those callers discarded the
   `memberships` field of the response, so the auth store kept the
   login-time snapshot forever. Role changes between sessions stayed
   invisible. Fix: extract memberships in both /auth/me consumers and
   feed them into `setMemberships`.

3. `KnowledgeBase.vue`'s `isOwner` compared `kb.tenant_id` to the
   active tenant id, so any KB inside the current tenant was treated
   as "owned by the caller" — bypassing every role gate below. Fix:
   compare against `kb.creator_id` (the field PR 5 introduced) and
   require an actual creator match. `canEdit` / `canManage` now layer
   `hasRole('admin')` on top so a non-creator with sufficient role can
   still manage.

4. `KnowledgeBaseList.vue` and `AgentList.vue` rendered the per-card
   more-menu (Settings, Delete, Edit) unconditionally. With the
   ownership column from PR 5 this is straightforward to gate on the
   client: Settings/Delete now require creator-match or `hasRole('admin')`
   per the server's `OwnedKBOrAdmin` / `OwnedAgentOrAdmin` matrix.

5. The axios interceptor (frontend/src/utils/request.ts) had a
   "selectedTenantId === defaultTenantId → don't attach X-Tenant-ID"
   short-circuit. Any code that wrote the active tenant into
   `weknora_tenant` (UserMenu's loadUserInfo, OIDC sync, router
   hydrate) made the two values equal, after which the header silently
   stopped attaching and subsequent navigation requests landed on the
   home tenant. Fix: drop the short-circuit; always attach when
   selectedTenantId is set.

Server-side, /auth/me read `user.TenantID` (the immutable home tenant
id stamped at signup) when fetching the tenant for the response, so
the `tenant` field always pointed at home even after the auth
middleware had already resolved an active tenant from X-Tenant-ID.
The frontend then re-keyed `authStore.tenant` to the home tenant on
every refresh, undoing the switch. Fix: read the active tenant id from
context (`types.TenantIDFromContext(ctx)`) and only fall back to
user.TenantID when context is unset.

The server-side enforcement was correct throughout — every fix here is
purely about not lying to the user with the rendered UI.
2026-05-18 17:28:58 +08:00
wizardchen 766a6c482f feat(rbac): per-tenant audit log for RBAC events (PR 6, #1303)
PR 1-5 of the multi-tenant RBAC series enforce the role matrix; what
they don't do is keep a durable record of who did what. Today the
RBAC middleware logs "[rbac] role insufficient ..." to stderr -
perishable, not searchable. Operators cannot answer "who removed
Alice from tenant T-foo on Tuesday" or "did we 403 a probing user
200 times last hour" after the fact.

This PR adds a generic per-tenant audit log that:

- Captures mutating RBAC actions (member add/remove/role-change/leave)
  durably from the tenant_member service after the repo write succeeds.
- Captures enforcement rejections (RequireRole / RequireOwnershipOrRole
  denies under EnableRBAC=true) via a gin-context-injected hook.
- Sliding-window dedup (1 minute) on (tenant, actor, action, path)
  for denied events so a probing client cannot flood audit_logs.
- Default-on - no extra env flag. Schema is generic enough for future
  KB / agent / datasource events to reuse via new action namespaces
  without another migration.

Schema (migration 000044) mirrors wiki_log_entries' shape: a single
audit_logs table with action, target_type/id, request path/method,
outcome, and a JSONB details column. Three indexes: (tenant_id, id
DESC) for the per-tenant cursor feed, actor_user_id for "what did
Alice do", (tenant_id, action) for both the action filter and the
dedup lookup. Sqlite mirror added for Lite mode.

Wiring is nil-safe: the audit field on tenantMemberService and the
gin-context lookup in rbac.go both tolerate a nil service so existing
tests construct without a stub and dormant deployments degrade
gracefully if the provider is misconfigured.

The query API is GET /tenants/:id/audit-log, gated by Admin+ on top
of the existing PathTenantMatch (denial histories should not surface
to ordinary members). Cursor-paginated by descending id (sidesteps
the duplicate-timestamp tie-breaking that BIGSERIAL trivially solves).

Frontend: TenantMembers.vue gets a t-tabs wrapper with Members
(existing) and Audit log (new) tabs; the audit tab is gated to
Admin+ to mirror the server, lazy-loads on first open, paginates via
the next_cursor pattern, and renders coloured chips per
action/outcome so an operator can scan for anomalies. i18n keys
added in all four locales.

Tests: 14 new tests across service (6), middleware (4), and handler
(4). Service tests pin the dedup window behaviour (writes again
after expiry, dedup is per-actor-per-path, degrades on lookup
error). Middleware tests pin both reject sites' audit hook and the
"dormant mode does not fire" + "nil audit does not panic"
invariants. Handler tests pin the cursor envelope and that an empty
page returns next_cursor=0.

Out of scope: per-action ACL on the audit log, retention/TTL job,
KB/agent/datasource event wiring (schema supports it - just no
emitters yet), async writes, frontend filter UI.
2026-05-18 17:28:58 +08:00
wizardchen ae7cd02cd6 fix(rbac): polish tenant member UI; add /leave endpoint
Self-review of PR 3's TenantMembers.vue surfaced a handful of real bugs
and a few rough edges. This commit fixes them and adds the self-leave
endpoint that PR 3's design called out as a follow-up.

Frontend bug fixes
- Read err.status (not err.response.status). The axios interceptor in
  utils/request.ts already flattens errors into { status, message,
  ... }, so the original branch-by-status code never matched and every
  4xx fell through to the generic toast. Fixes 404/409 surfacing as
  "userNotFound" / "alreadyMember" / "lastOwner" instead of "Something
  went wrong".
- t-dialog @confirm replaces :on-confirm so the dialog stays open on
  validation failure and on API error. Users can fix and retry without
  retyping the email.
- Drop the t-select revert hack; with :model-value the rendered value
  is already source-of-truth-on-row.role, so no setTimeout dance is
  needed.
- Don't toast "no active tenant" on cold mount; watch the activeTenantId
  ref and load once the auth store hydrates.

Frontend UX additions
- Empty state via t-empty (also covers the search-no-results case).
- Compact role-permissions matrix above the table, mirroring
  OrganizationSettingsModal so Owners can see what each role grants
  before promoting/inviting.
- Search box filtering by username/email (client-side, sufficient for
  expected per-tenant member counts).
- "Leave Tenant" button — visible to any active member except the
  last-Owner case (server still enforces).

Backend
- New POST /tenants/:id/leave route. Calls TenantMemberService.RemoveMember
  with the caller's own user id so non-Owners can quit on their own.
  Gated Viewer+ in the matrix; the existing last-Owner invariant in the
  service still rejects last-Owner self-leave with 409.
- 3 new handler tests cover the happy path, the last-Owner 409 path,
  and the missing-caller-id 401 path (defensive — auth middleware
  always attaches a user, but the test pins the contract).

i18n
- Add tenantMember.permissions / search / leave / empty strings in all
  four locales (en/zh/ko/ru).
2026-05-18 17:28:58 +08:00
wizardchen 3f7150cb5c feat(rbac): tenant member management + invite_only gate (PR 3 of 3, #1303)
Wire the human-facing surface for the tenant-RBAC system that PR 1
introduced and PR 2 enforced: REST endpoints + UI for managing tenant
members, plus the long-pending invite_only registration gate.

Backend
- internal/handler/auth.go: Register now rejects with 403 when
  auth.registration_mode=invite_only. PR 1 already exposes the mode via
  /auth/config so the frontend hides the sign-up link; this is the
  server-side enforcement for direct API calls.
- internal/handler/tenant_member.go: new TenantMemberHandler with
  ListMembers / AddMember / UpdateMemberRole / RemoveMember. Maps
  service-layer sentinels onto HTTP semantics (ErrLastOwner -> 409,
  ErrMembershipAlreadyExists -> 409, ErrUserNotFound on AddMember -> 404,
  ErrInvalidTenantRole -> 400). AddMember requires the invitee to
  already have an account; SMTP-driven email invites are deferred.
- internal/types/tenant_member.go: new TenantMemberResponse projection
  carrying the fields the management UI renders (email, username,
  avatar, role, joined_at).
- internal/router/router.go: four routes under /tenants/:id/members
  gated per the matrix (Viewer+ for list, Owner+ for every mutation).
- internal/container/container.go: dig provider for the new handler.

Tests
- handler/tenant_member_test.go: 12 table-driven cases covering happy
  path, dangling-user tolerance, ErrLastOwner -> 409, ErrMembershipNotFound
  -> 404, ErrMembershipAlreadyExists -> 409, unknown email -> 404,
  invalid role rejected before user lookup, bad tenant id -> 400.
- handler/auth_register_invite_only_test.go: invite_only -> 403,
  self_serve -> 201, nil Auth config falls through.

Frontend
- src/api/tenant/members.ts: typed client (TenantRole/TenantMember/...)
  for the four endpoints.
- src/views/settings/TenantMembers.vue: members table with add dialog,
  inline role select, confirm-remove modal. All mutation widgets gated
  behind currentTenantRole === 'owner'; server still enforces.
- src/views/settings/Settings.vue: register the Members tab; show only
  when currentTenantRole is loaded (avoids tab flash on cold nav).
- i18n in en-US/zh-CN/ko-KR/ru-RU: new tenantMember namespace covering
  all UI strings and structured error messages (lastOwner / userNotFound
  / alreadyMember / invalidRole etc).

This completes the #1303 RBAC rollout. With cfg.Tenant.EnableRBAC=true
operators now have a full management surface; with EnableRBAC=false
the routes still exist but mutations log-and-pass per PR 2 contract.
2026-05-18 17:28:58 +08:00
wizardchen 7f7a48880f feat(rbac): add tenant_members schema and auth wiring (PR 1 of 3, #1303)
Lay the foundation for tenant-level RBAC without enforcing it yet.
Schema, types, middleware lookup, and login-flow shape changes ship now;
RequireRole/route annotations and member-management endpoints follow in
PR 2 and PR 3.

Highlights:
- New tenant_members table (Postgres + SQLite) with (user_id, tenant_id)
  unique key and (owner|admin|contributor|viewer) role values. Migration
  000043 backfills one row per existing user, marking the earliest-created
  active user in each tenant as Owner and the rest as Contributor.
- KnowledgeBase.creator_id and CustomAgent.runnable_by_viewer added so
  PR 2 can wire ownership-based and Viewer-runtime checks.
- Auth middleware loads the active TenantMember row, sets a new
  TenantRoleContextKey, and falls back via three escape hatches:
  cross-tenant superusers (CanAccessAllTenants) get Admin in the target
  tenant; orphan tenants (zero members) auto-promote the first authenticating
  human to Owner; and when tenant.enable_rbac is false (the v1 default)
  unmatched lookups fail open with Admin to preserve current behaviour.
- Register / OIDC provisioning now insert an Owner membership for the
  registrant via TenantMemberService.EnsureOwner.
- LoginResponse renamed tenant -> active_tenant and gained a memberships
  array. POST /auth/switch-tenant issues a fresh token pair scoped to a
  target membership; GET /auth/config exposes auth.registration_mode for
  the frontend register-tab gating.
- New config keys: auth.registration_mode (self_serve | invite_only,
  defaults to self_serve) and tenant.enable_rbac (defaults to false).
  Both have WEKNORA_* env overrides.
- Frontend store persists memberships and exposes currentTenantRole;
  Login.vue calls /auth/config and hides the Register tab in invite_only
  mode. Go SDK types updated to match the new LoginResponse shape.
2026-05-18 17:28:58 +08:00
wizardchen a9d37970ae fix(initialization): fall back to stored apiKey when test-connecting an existing model
Direct regression from the previous credential subresource refactor: the
edit-mode ModelEditorDialog stopped binding apiKey into formData (the
field now lives in <CredentialResource>), so "Test connection" sent
apiKey="" and every upstream call (DeepSeek / lkeap / OpenAI etc.) 401'd.

Fix in the backend so the frontend stays simple:

- ModelTestRequest gains an optional ModelID field.
- New fillSecretsFromStoredModel helper: when ModelID is set AND a secret
  field on the request is empty, copy it from the stored (decrypted) model.
  Non-empty request values still win, so users typing a new key to verify
  it before saving behave the same as before.
- The four test entrypoints (chat / embedding / rerank / asr) call the
  helper right after ShouldBindJSON.

Frontend: ModelEditorDialog.testConnection forwards modelId when isEdit
is true. The four API client signatures accept the optional field.

WebSearch and MCP are unaffected: WebSearch already has a TestProviderByID
ID-based path, MCP has no standalone raw test endpoint.
2026-05-17 15:27:52 +08:00
wizardchen 7643dd457e refactor(credentials): introduce /credentials subresource pattern
Replace the legacy "redacted placeholder + Clear* boolean" pattern with
dedicated per-resource credential subresources across MCP services,
Models, Web search providers, and Data sources.

Why
---
The previous design had three problems:

1. Main PUT body carried secret fields. The frontend echoed back a
   redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty
   defense in the service layer tried to detect "user did not change this"
   vs "user wants to clear this". A regression in that defense (or a new
   frontend forgetting it) silently overwrites the stored secret with the
   placeholder.

2. The "remove this credential" UX was a red checkbox under a pre-filled
   password input. Three intents (preserve / replace / clear) collapsed
   onto one field, and credential changes were bundled with unrelated
   config edits in the same submit. Users wiped working keys by mistake.

3. Secret presence was inferred from "did the response come back with a
   '***' placeholder", which couples the contract to a magic string.

Design
------
Each of the four resources now exposes:

  PUT    /{resource}/{id}/credentials          # write one or more fields
  DELETE /{resource}/{id}/credentials/{field}  # clear a single field

"Is this configured?" metadata travels on the main resource response as
a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET
endpoint. The frontend reads the configured boolean from the main GET
and never sees secret values at all.

Main PUT endpoints now ignore any secret fields in the request body and
log a deprecation warning if they appear, so legacy clients fail loudly
rather than overwriting silently.

Frontend
--------
- New reusable <CredentialResource> component renders a three-state card
  (unconfigured / configured / editing) and drives the dedicated
  endpoints. Used by MCP, Model, Web search; DataSource has a bespoke
  card with the same behaviour because its credentials are a single
  atomic per-connector map.
- Cancel from edit mode now restores state synchronously from the meta
  prop. The previous async refresh() was a no-op while state was still
  'editing', leaving the input frozen open.
- Remove is single-click + toast. The danger-themed button is the
  deterrent; a modal confirm adds friction without adding safety (the
  plaintext is irrecoverable client-side either way — recovery means
  re-typing).

DTOs (internal/handler/dto/) are deliberately separate from the GORM
models so "no secret in response" is a compile-time invariant: a future
contributor cannot leak a secret without explicitly adding the field to
the DTO, which is review-able in a single diff.

Storage is unchanged — credentials still live in the existing jsonb /
parameters columns. No schema migration.

Cleanup
-------
- types.MCPService / types.Model / types.WebSearchProviderEntity /
  types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret
  / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData().
- utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty.
  RedactedSecretPlaceholder constant is retained because VectorStore
  still uses the old pattern and is out of scope here.
- Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat
  redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 15:27:52 +08:00
ochan.kwon d869cc1ff6 fix: redact sensitive fields and prevent credential loss on edit
Closes #988

Unify the "write-only secrets" pattern across MCP services, Models,
WebSearch providers, and DataSources. Fix two related defects:

1. GET responses leaked plaintext credentials — GetMCPServiceByID
   while List masked, all Model / WebSearchProvider / DataSource GET
   responses unconditionally.
2. When the frontend echoed a masked placeholder back on save, the
   update path overwrote the stored secret with the mask — silently
   wiping the real credential. MCP surfaced this most visibly:
   ListMCPServices masked Token to "first4****last4" and
   McpServiceDialog round-tripped that masked string back into the DB.

- internal/types/secret.go introduces the shared sentinel
  RedactedSecretPlaceholder = "***" and two helpers:
  - IsRedactedOrEmpty(s): true for "" or the sentinel
  - PreserveIfRedacted(incoming, existing): merge predicate
- Each resource adds a RedactSensitiveData (mutates-in-place) or
  Redacted (returns-a-copy) method and a MergeUpdate contract:
  empty or placeholder preserves, any other value replaces, and a
  write-only Clear* bool flag (gorm:"-", json omitempty) wipes with
  precedence over preserve / replace. Clear* never persists.
- Every GET / List / Create / Update response path now redacts.
  VectorStore's pre-existing "***" literal is refactored onto the
  shared constant for consistency.

All four dialogs — McpServiceDialog, ModelEditorDialog,
WebSearchSettings, DataSourceEditorDialog — never pre-fill secret
inputs. The stored-state signal is collapsed into the input
placeholder: "•••••••• (Enter new value to replace)" when the server
reports a stored value, falling back to the connector-specific
placeholder otherwise. The input itself carries the "something is
there" cue, matching how password dots render.

An opt-in "Remove this credential" checkbox appears in edit mode when
a value is stored; checking it disables the input and triggers a
danger-themed confirmation dialog before submit. DataSource rolls the
clear up to a connector-level "Remove all credentials" flag since its
credentials are per-connector atomic bundles (e.g. Feishu app_id +
app_secret).

i18n updates across en-US, zh-CN, ko-KR, ru-RU:
secret.storedPlaceholder / clearHint / confirmClearTitle /
confirmClearBody.

- MCP update handler's manual map-to-struct projection was silently
  dropping clear_api_key / clear_token / custom_headers on partial
  updates. Pull them off the map explicitly.
- MCP configChanged detection now snapshots URL / StdioConfig /
  TransportType before the merge so connection recycling still
  triggers on real transport changes (not just whenever AuthConfig
  is present).
- MCP CustomHeaders: nil preserves existing, non-nil replaces. Was
  always overwriting, silently wiping headers on any auth-bearing
  update.
- GetMCPServiceByID now redacts (was plaintext while List masked).
- WebSearchProvider update preserves Name / Description when empty
  instead of clobbering them.
- WebSearchProvider TestProviderRaw rejects the "***" placeholder up
  front with an actionable message rather than forwarding it to the
  upstream provider API as a literal key.
- Model builtin hideSensitiveInfo now preserves non-secret metadata
  (SupportsVision, Provider, ExtraConfig, InterfaceType,
  ParameterSize, EmbeddingParameters) that were previously zeroed
  out for builtin model cards.
- Replace local equalStringSlices / equalStringMaps helpers with
  stdlib slices.Equal / maps.Equal (Go 1.21+).

- internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go:
  table-driven coverage of merge, redact, clear, and write-only flag
  invariants for every resource.
- internal/application/service/mcp_service_test.go: full service layer
  with a fake repo; covers ListMCPServices / GetMCPServiceByID
  redaction, UpdateMCPService partial-update honoring of Clear*,
  CustomHeaders preserve-on-nil / replace-on-non-nil, and
  clear-flag precedence over submitted values.

Additive only. No changes to request or response JSON shape beyond
the write-only Clear* flags (omitted by default via omitempty). No DB
schema changes. No SDK struct field removals. Existing clients that
don't know about the new flags continue to function unchanged.

- DataSource.Config jsonb at-rest encryption. Pre-existing residual
  risk; tracked as a follow-up.
- Renaming VectorStore.MaskSensitiveFields to match the new
  RedactSensitiveData / Redacted naming convention. Left alone to
  minimize diff; a follow-up rename PR can unify.
2026-05-17 11:37:53 +08:00
wizardchen 6f95c75ed2 feat(system-info): surface DB migration errors with troubleshooting links
When a startup database migration fails (e.g. issue #1319: pg_trgm not
available so 000041 cannot build its trigram index), the application
intentionally keeps booting so operators can reach the UI to diagnose.
However, before this change the system info page silently dropped the
"DB Version" row because the value was empty:

  - migration.go only cached the version after a successful m.Up(); the
    error path returned early and left migrationVersionSet=false.
  - system.go used CachedMigrationVersion's ok=false to skip emitting
    db_version, and the JSON tag was already omitempty.
  - SystemInfo.vue gated the entire row on v-if="systemInfo?.db_version".

The end result was the most useful diagnostic surface vanishing in the
exact failure mode that needs it most — Wiki ingest and KG features
would silently produce nothing with no UI hint.

Changes:

  - migration.go: replace sync.Once-based setter with an RWMutex-guarded
    state struct holding {version, dirty, err}. Every failure path now
    calls captureMigrationFailure(m, err), which best-effort reads
    m.Version() so the cached value still reflects the partial state.
  - system.go (handler): always emit db_version (falling back to
    "unknown" when no version could be read), append " (failed)" when an
    error is recorded, and add db_migration_error to the response.
  - swagger / client SDK: keep the API contract in sync with the new
    response field.
  - SystemInfo.vue: render the DB version row whenever either field is
    present, show a "Migration failed" danger tag, and add a full-width
    alert below the row carrying the error message plus two links:
      1. View troubleshooting guide -> new docs/migration-troubleshooting.md
      2. Report an issue -> github.com/Tencent/WeKnora/issues/new,
         prefilled with the captured error and environment metadata.
  - docs/migration-troubleshooting.md: new self-service guide covering
    the common failure modes (missing extension, dirty state, privileges,
    out of disk, schema drift) with concrete psql / make commands.
  - i18n: add the new keys to zh-CN, en-US, ko-KR, ru-RU.

Refs #1319.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-14 16:34:50 +08:00
wizardchen 55caeefc3d feat(observability): log end-to-end TTFB on both ends of chat stream
To diagnose where latency lives between "user hits send" and "first
token appears" we need a single number that can be matched across
the browser console and the server log. Add correlated TTFB markers
keyed by X-Request-ID:

Frontend (streame.ts)

* Generate the X-Request-ID once and reuse it for logging.
* Log request:start when fetchEventSource is invoked, response:headers
  when onopen fires, and response:first_answer the first time an SSE
  payload with response_type === 'answer' arrives. Filtering by event
  type avoids treating session_title / references / tool_call as the
  "first token".

Backend (session handler)

* parseQARequest records the wall-clock entry time and logs TTFB:start
  with the same X-Request-ID.
* qaRequestContext carries receivedAt through to the stream handler.
* AgentStreamHandler emits a one-shot TTFB:first_answer_chunk log the
  first time it observes an AgentFinalAnswerData chunk, so the delta
  against TTFB:start is the server's request-in → first-token-out
  budget.

The frontend delta minus the backend delta is then attributable to
network + gin middleware, which were previously invisible.
2026-05-12 21:16:37 +08:00
wizardchen cacca049d9 feat(knowledge-base): document list filters and explicit batch-management UX
Add three new optional filters to the document list under a knowledge base
detail page — parse status, source/channel, and updated time range — and
rework multi-select to no longer cause the card title to jitter on hover.

Backend
- Introduce types.KnowledgeListFilter to aggregate optional filter dimensions
  (tag, keyword, file_type, parse_status, source, updated_from/to) and switch
  ListPagedKnowledgeByKnowledgeBaseID (repository/service/interface) to accept
  it instead of a growing positional parameter list.
- The ListKnowledge HTTP handler accepts new parse_status, source, start_time
  and end_time query params; time params accept RFC3339, "YYYY-MM-DD HH:MM:SS"
  and "YYYY-MM-DD". The repository routes source="manual"/"url" onto the type
  column to stay consistent with file_type semantics; other source values match
  the channel column.
- Update the four other callers (agent_service, initialization) to pass an
  empty filter struct, preserving prior behavior.

Frontend
- Add three controls in the doc-filter-bar (status select, source select,
  date-range picker with future-date disabled) wired through getKnowled /
  listKnowledgeFiles into the new backend params.
- Replace the hover-triggered card checkbox with an explicit "批量管理" mode
  (mirrors the session list UX): in card view the checkbox only renders while
  batch mode is on, entered via the per-card "..." menu; the list view keeps
  its leading checkbox column. Switching from list to grid auto-enables batch
  mode when something is already selected, so the selection stays visible.
- DocumentBatchBar now stays open whenever batch mode or selection > 0, and
  its "取消选择" button both clears the selection and exits batch mode.

API surface sync
- Regenerate Swagger artifacts (docs/docs.go / swagger.json / swagger.yaml).
- Update docs/api/knowledge.md with the new query parameters.
- Add backward-compatible ListKnowledgeWithFilter + KnowledgeListFilter to the
  Go SDK; the existing ListKnowledge keeps its signature.

i18n
- New filter labels in zh-CN / en-US / ko-KR / ru-RU; reuse existing
  menu.batchManage / batchManage.cancel for the multi-select strings.
2026-05-12 18:29:47 +08:00
wizardchen 1110615300 feat(web-search): add SearXNG provider (#1166)
支持对接自建/公共 SearXNG 实例作为网络搜索引擎,缓解免费搜索引擎在国内
网络环境下访问受限的问题。

- types: 新增 WebSearchProviderTypeSearxng 与 BaseURL 参数字段;
  类型元数据新增 RequiresBaseURL,前端可动态渲染 Instance URL 表单。
- infrastructure/web_search/searxng.go: 调用 /search?format=json,强制
  utils.ValidateURLForSSRF 校验 base_url,可选 api_key 透传给反代鉴权。
- service: isValidProviderType 与参数校验接入 searxng。
- container: 注册 NewSearxngProvider 工厂。
- frontend: WebSearchSettings 表单根据 requires_base_url 渲染 Instance
  URL 输入框;编辑回填、free 判定同步更新。
- docker: 新增可选 searxng 服务(profile=searxng/full),附带最小化
  settings.yml(启用 JSON 格式、关闭 limiter、关闭遥测),
  docker-compose 默认 SSRF_WHITELIST 包含 searxng 容器名。
- .env.example: 补充 SEARXNG_PORT / SEARXNG_SECRET 说明。

Closes #1166
2026-05-11 16:53:47 +08:00
wizardchen 5510ea8f5a feat(agent): human-in-the-loop approval for MCP tool calls (#1173)
Add an opt-in human approval gate so Agent runs pause before executing
MCP tools that operators flag as dangerous, surface an approval card in
the chat UI, and only resume after the user approves (optionally with
edited args) or rejects.

Backend
- New mcp_tool_approvals table + repo/service to mark per-tool approval
  required (PG migration 000042 + sqlite init).
- approval.Gate coordinates RequestAndWait / Resolve with sync.Once
  delivery, configurable timeout, and Redis Pub/Sub fan-out so multi-
  replica deployments work without sticky sessions.
- MCPTool.Execute integrates the gate; uses a round-level ApprovalCtx
  (without the per-tool 60s timeout) for the wait, and re-derives a
  fresh 60s exec ctx after approval so CallTool keeps a full window.
- New SSE response types (tool_approval_required / _resolved) and
  EventBus events plumb approval state to AgentStreamDisplay.
- REST: list/set per-tool approval flag, resolve pending approval.
- Configurable via agent.tool_approval_timeout_seconds (yaml) or
  WEKNORA_AGENT_TOOL_APPROVAL_TIMEOUT env (accepts seconds or Go
  duration).

Frontend
- MCP settings: per-tool "require approval" switch on the test panel.
- Chat: ToolApprovalCard renders the pause point with editable JSON
  args, validation feedback, mm:ss countdown that turns warning/danger
  near deadline, and a resolved state that retains context.
- i18n strings added for zh-CN / en-US / ko-KR / ru-RU.

Docs
- docs/zh/mcp-approval.md covering behavior, config, API, deployment
  considerations (Redis cross-instance, restart limitations).
2026-05-10 22:57:12 +08:00