WeKnora/internal/types
lyingbug 967ed09706
feat(models): add system admin model catalog console (#3662)
* feat(models): add system admin model catalog console

System administrators can now maintain the model catalog from
Settings → System administration → Model catalog instead of editing
models.json on every replica. Administrator changes are stored as a
versioned overlay in the database (new model_catalog_config table),
layered between the deployment file and explicit per-model settings,
and synchronised to other instances within about 5 seconds.

Backend
- GET/POST preview/PUT /system/admin/model-catalog with optimistic
  version + deployment baseline checks (409 on conflict), 20-version
  history and audit logging.
- Console overlays reject credentials, headers, env interpolation and
  non-inline icons; those stay deployment-owned.
- Catalog views include each chat model's resolved thinking levels and
  the vendor defaults, so the console never re-implements level
  inheritance.

Frontend
- Model list in the shared data-table style with provider/type filters,
  "modified only" filter and source tags (built-in / deployment / admin).
- Edit drawer saves and applies immediately: display name, context
  window, max output, input modalities, reasoning, thinking levels,
  embedding dimension and picker visibility, each with its inherited
  default and a layer comparison table.
- Add-model drawer, JSON editor with server-side change check before
  publishing, and version history with one-step restore.
- Model settings gets a system-admin shortcut to the catalog; model
  editor reloads providers when opened so published catalogs apply.

* fix(models): harden the admin model catalog console

- Reject console base_url, base_urls, url_patterns and auth: moving a
  vendor's default URL would send its deployment api_key (and keys of rows
  without their own base URL) to another host. With every interpolated
  field now deployment-owned, '$' in names and descriptions is plain text.
- Drop deployment base URLs from catalog views; they may carry
  interpolated tokens.
- Remember a stored version this replica cannot compile and skip it until
  the version changes, instead of re-reading and recompiling every poll.
- Resolve vendor default thinking levels with reasoning forced on, like
  per-model levels.
- Compute builtin/deployment views once; preview returns only the
  candidate catalog and normalized overlay.
- Share deployment file lookup via runtime.ReadDeploymentOverlay.
- The model editor force-refreshes its own type instead of resetting the
  provider store for every type; the refreshed list wins in the id index.
2026-09-24 16:31:45 +08:00
..
interfaces fix(tenant): stop wiki tasks of soft-deleted tenants from issuing model requests (#3602) 2026-09-23 12:13:32 +08:00
agent.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
agent_share_source.go fix: harden shared agent source selector validation 2026-07-30 12:28:23 +08:00
agent_share_source_test.go fix: harden shared agent source selector validation 2026-07-30 12:28:23 +08:00
agent_test.go feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
agent_type_preset.go feat: Introduce agent type presets for smart-reasoning mode 2026-04-22 21:18:20 +08:00
artifact_versions.go feat(agent): unify sandbox tools and drop host preloaded skills (#3033) 2026-09-05 16:45:59 +08:00
artifact_versions_test.go feat(agent): unify sandbox tools and drop host preloaded skills (#3033) 2026-09-05 16:45:59 +08:00
audit_log.go feat(system-admin): add endpoint for creating users (#2722) 2026-08-20 19:51:10 +08:00
audit_log_test.go feat(system-admin): add endpoint for creating users (#2722) 2026-08-20 19:51:10 +08:00
browser_search.go feat(browser): polish connection settings, open tasks in a new window, and streamline prompts (#3193) 2026-09-11 21:28:58 +08:00
builtin_agent_config.go fix(i18n): 修复内置智能体名称与描述不随界面语言切换的问题 (#2828) 2026-08-27 11:53:45 +08:00
builtin_agent_localization_test.go fix(i18n): 修复内置智能体名称与描述不随界面语言切换的问题 (#2828) 2026-08-27 11:53:45 +08:00
builtin_models_config.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
builtin_models_config_test.go feat(models): let system admins edit builtin model config 2026-07-14 15:56:02 +08:00
caller.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
chat.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
chat_history_config.go feat: implement chat history and retrieval settings 2026-03-09 11:46:56 +08:00
chat_manage.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
chunk.go perf(chunk): speed up chunk ingestion and batch updates (#3369) 2026-09-17 21:38:15 +08:00
chunk_contextheader_test.go test(types): regression test that ContextHeader is hidden from JSON 2026-05-06 17:17:07 +08:00
cleanup.go init commit 2025-08-05 15:08:07 +08:00
config_redaction.go fix(sandbox): always require inbound credentials 2026-09-03 13:34:40 +08:00
config_redaction_sandbox_test.go feat(sandbox): add a graphical desktop to session sandboxes (#3245) 2026-09-14 20:37:37 +08:00
config_redaction_test.go feat(storage): support AWS S3 NoAk default credentials (#2008) 2026-08-05 11:59:02 +08:00
const.go refactor(embedding): move the wire formats into protocol packages, and correct the vendor facts (#3500) 2026-09-21 11:55:38 +08:00
context_checkpoint.go feat(agent): size agent history by the window and calibrate token estimates (#3413) 2026-09-18 22:51:31 +08:00
context_clone.go feat(audit): 知识库活动记录对 API Key 操作额外显示 Key 名称 (#3405) 2026-09-18 19:39:34 +08:00
context_clone_coverage_test.go refactor(context): replace middleware.EmbedChannelContextKey with types.EmbedChannelContextKey 2026-08-13 18:00:02 +08:00
context_helpers.go refactor(embedding): move the wire formats into protocol packages, and correct the vendor facts (#3500) 2026-09-21 11:55:38 +08:00
context_helpers_test.go fix: retry document summaries before fallback 2026-08-05 19:29:32 +08:00
custom_agent.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
custom_agent_reasoning_effort_test.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
custom_agent_test.go feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
dataset.go init commit 2025-08-05 15:08:07 +08:00
datasource.go feat(datasource): persist source created time; Feishu uses content edit time for source_updated_at (#3083) 2026-09-07 19:39:01 +08:00
datasource_test.go feat(feishu): sync docx via blocks API with per-type downdrill (#2087) 2026-07-27 16:04:56 +08:00
docparser.go fix(chunker): trim surrounding whitespace before embedding 2026-05-06 21:16:48 +08:00
embed_channel.go feat(im): add per-channel reply locale (#3455) 2026-09-23 12:01:01 +08:00
embed_channel_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
embedding.go feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
errors.go init commit 2025-08-05 15:08:07 +08:00
evaluation.go fix: support custom jieba dictionary directory 2026-06-08 20:10:37 +08:00
event_bus.go refactor: Implement event-driven streaming and enhance session management 2025-11-07 22:41:41 +08:00
extract_graph.go Refactor knowledge post-processing into orchestrator task 2026-04-16 18:13:20 +08:00
faq.go fix: remove GPL converter and bundle third-party license sources 2026-09-08 15:19:10 +08:00
faq_sync.go feat(knowledge): enhance FAQ export functionality 2026-07-23 10:50:32 +08:00
faq_sync_test.go feat(knowledge): enhance FAQ export functionality 2026-07-23 10:50:32 +08:00
faq_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
file_reference.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
file_reference_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
generated_question_revision_test.go fix(chunks): retain questions across content edits 2026-07-28 20:23:48 +08:00
generated_question_source_id_test.go fix(knowledge): bound generated question source IDs 2026-07-28 20:23:48 +08:00
graph.go feat: implement preview of user level memory graph 2026-02-25 14:29:54 +08:00
indexing_strategy.go feat: Add comprehensive documentation for WeKnora knowledge base and wiki features 2026-04-22 21:18:10 +08:00
json.go test(types): add unit test for JSON unmarshalling and validation 2026-04-22 21:17:36 +08:00
json_map.go refactor(knowledge): replace flat stage table with langfuse-style span tree 2026-05-28 15:14:45 +08:00
json_test.go test(types): add unit test for JSON unmarshalling and validation 2026-04-22 21:17:36 +08:00
knowledge.go feat: add configurable sorting to document and resource lists (#3318) 2026-09-23 12:00:21 +08:00
knowledge_custom_metadata_test.go feat(knowledge): support editable chunks and metadata 2026-07-28 20:23:48 +08:00
knowledge_folder.go fix(knowledge): keep long non-ASCII folder names valid UTF-8 2026-09-07 11:50:32 +08:00
knowledge_folder_test.go fix(knowledge): keep long non-ASCII folder names valid UTF-8 2026-09-07 11:50:32 +08:00
knowledge_process.go feat(knowledge): make document summary generation optional 2026-09-17 19:39:52 +08:00
knowledge_process_test.go feat(docreader): support forcing PDF scanned-page OCR parsing 2026-06-25 17:30:22 +08:00
knowledge_profile.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
knowledge_profile_test.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
knowledge_span.go fix(spans): expand knowledge span name column and truncate long wiki names 2026-07-13 10:35:44 +08:00
knowledgebase.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
knowledgebase_auto_tag_test.go fix(knowledge): harden auto-tagging and let it respect manual tags 2026-08-07 15:35:58 +08:00
knowledgebase_profile.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
knowledgebase_test.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
legacy_persisted_json_test.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
locale.go feat(im): add per-channel reply locale (#3455) 2026-09-23 12:01:01 +08:00
locale_test.go feat(im): add per-channel reply locale (#3455) 2026-09-23 12:01:01 +08:00
mcp.go feat(mcp)!: streamline setup and use short routing handles 2026-09-09 16:11:52 +08:00
mcp_endpoint.go feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
mcp_endpoint_tools.go feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
mcp_endpoint_tools_test.go feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
mcp_metadata.go feat(mcp)!: persist tool catalogs and add guided configuration (#3118) 2026-09-09 13:34:56 +08:00
mcp_oauth.go fix(mcp): coordinate OAuth token refresh and authorization polling 2026-07-23 15:35:24 +08:00
mcp_test.go feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling 2026-05-17 15:27:52 +08:00
memory.go fix(memory): rank memories by the query, not by importance 2026-09-13 11:43:17 +08:00
memory_extraction.go fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
memory_test.go feat(browser): polish connection settings, open tasks in a new window, and streamline prompts (#3193) 2026-09-11 21:28:58 +08:00
message.go feat(artifacts): let users delete generated files (#3447) 2026-09-20 13:01:46 +08:00
message_artifact.go feat(artifacts): let users delete generated files (#3447) 2026-09-20 13:01:46 +08:00
message_attachment_test.go feat(im): process file attachments in QA 2026-08-12 16:08:51 +08:00
message_suggestion.go feat(agent): add question suggestions with after-answer follow-ups 2026-07-14 11:17:25 +08:00
message_test.go feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
model.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
model_catalog_config.go feat(models): add system admin model catalog console (#3662) 2026-09-24 16:31:45 +08:00
organization.go fix(security): close the low-severity shared-space findings (#3416) 2026-09-18 22:58:46 +08:00
parser_engine_config_test.go fix(parser): default ppt/pptx to markitdown when no engine rules (#2882) 2026-08-31 14:58:15 +08:00
placeholder.go feat(chat): keep prompt prefixes stable and send provider cache markers 2026-09-03 00:45:52 +08:00
placeholder_time_test.go feat(chat): keep prompt prefixes stable and send provider cache markers 2026-09-03 00:45:52 +08:00
principal.go fix(mcp): resolve MCP endpoint sessions by their storage ID 2026-09-22 16:53:10 +08:00
principal_test.go test(api-keys): add unit tests for tenant API key validation 2026-07-07 12:00:13 +08:00
prompt_instructions.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
prompt_instructions_test.go Enhance knowledge base configuration by normalizing and validating custom instruction fields. Implemented length checks for prompt instructions and added tests for validation logic. Updated GraphSettings to retain custom instructions when extraction is disabled. 2026-07-13 12:16:19 +08:00
qa_request.go feat(chat): add session reasoning controls and polish model configuration 2026-09-23 17:12:19 +08:00
qa_request_test.go fix(chat): preserve steering continuity and composer focus 2026-09-09 20:15:23 +08:00
question_suggestion_test.go feat(agent): add question suggestions with after-answer follow-ups 2026-07-14 11:17:25 +08:00
resource.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
resource_test.go feat(agent): preview skill artifacts and verify installs without running them (#2823) 2026-08-27 10:30:32 +08:00
retrieval_config.go Fix/hybrid search match count normalization (#2750) 2026-08-20 11:36:38 +08:00
retriever.go feat(retriever): add OpenSearch type prep ahead of Phase 3 driver 2026-05-22 20:43:50 +08:00
runtime_task.go feat(system): add cursor pagination for runtime queue tasks 2026-07-14 16:53:07 +08:00
sandbox_network_policy.go fix(sandbox): always require inbound credentials 2026-09-03 13:34:40 +08:00
sandbox_network_policy_test.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
sandbox_tenant_context_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
search.go fix(chunks): 流水线重写后的检索结果不用位置匹配 (#2558) 2026-08-06 14:32:24 +08:00
search_scope_test.go refactor(agent): introduce modelcontext registry and fix handle-codec bugs 2026-07-27 11:05:43 +08:00
secret.go fix(security): harden secret redaction follow-ups for PR #1884 2026-07-02 14:41:42 +08:00
secret_test.go fix(security): harden secret redaction follow-ups for PR #1884 2026-07-02 14:41:42 +08:00
session.go feat(chat): add session reasoning controls and polish model configuration 2026-09-23 17:12:19 +08:00
session_access_test.go feat(sandbox): expose tenant skill management over HTTP and in the console (#2789) 2026-08-25 11:28:12 +08:00
session_fork.go feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
session_fork_test.go feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
shared_agent_access.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
shared_agent_access_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
storagebackend.go feat(storage): add multi-instance storage backends per workspace 2026-07-15 14:54:39 +08:00
storagebackend_test.go feat(storage): add multi-instance storage backends per workspace 2026-07-15 14:54:39 +08:00
sync_item_error_test.go fix(feishu): resilient large-wiki sync — retry, streaming resume, failure visibility 2026-07-21 16:14:22 +08:00
system_setting.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
tag.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
task.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
task_dead_letter.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
task_initiator_test.go feat(audit): 知识库活动记录对 API Key 操作额外显示 Key 名称 (#3405) 2026-09-18 19:39:34 +08:00
task_pending_op.go fix(queue): order ClaimBatch by fail_count so retries cannot starve new work (#3444) 2026-09-21 13:00:29 +08:00
task_queue_test.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
temporary_document.go fix: harden shared agent source permissions 2026-07-30 12:28:23 +08:00
tenant.go feat(sandbox): add a graphical desktop to session sandboxes (#3245) 2026-09-14 20:37:37 +08:00
tenant_api_key.go feat(audit): 知识库活动记录对 API Key 操作额外显示 Key 名称 (#3405) 2026-09-18 19:39:34 +08:00
tenant_api_key_scope_test.go Add API key member and space capabilities 2026-07-07 12:00:13 +08:00
tenant_api_key_test.go Refine scoped API key capabilities 2026-07-07 12:00:13 +08:00
tenant_env_vars.go fix(sandbox): make skill env capture fill blanks only 2026-08-27 14:32:00 +08:00
tenant_env_vars_test.go fix(sandbox): make skill env capture fill blanks only 2026-08-27 14:32:00 +08:00
tenant_invitation.go feat(rbac): add multi-use share-link invitations for invite_only mode (#1504) 2026-05-27 19:42:01 +08:00
tenant_member.go Refine scoped API key capabilities 2026-07-07 12:00:13 +08:00
tenant_sandbox_config_entity.go feat(sandbox): 沙箱后端改为按空间配置,并补齐 E2B 模板与检测诊断 2026-08-11 15:42:02 +08:00
tenant_sandbox_config_entity_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
tenant_sandbox_config_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
tenant_skill.go fix(skills): upgrade outdated sandbox skills without rolling back the catalog or taking them offline (#3406) 2026-09-18 20:27:43 +08:00
tenant_test.go fix: preserve MinerU automatic PDF parsing 2026-08-03 15:37:07 +08:00
token_usage_test.go feat(agent): report and attribute per-turn LLM token usage 2026-08-24 10:36:19 +08:00
tracing.go feat(tracing): migrate langfuse client to OTLP/OTel wire protocol 2026-07-17 16:30:45 +08:00
user.go feat(browser): polish connection settings, open tasks in a new window, and streamline prompts (#3193) 2026-09-11 21:28:58 +08:00
user_resource_favorite.go feat(favorites): implement user favorites and recents functionality 2026-05-18 17:28:58 +08:00
vectorstore.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
vectorstore_opensearch_test.go feat(retriever): activate OpenSearch k-NN driver (PR 3 of 3) 2026-05-29 16:32:27 +08:00
vectorstore_test.go feat(retriever): configure Tencent VectorDB replicas via env 2026-06-22 20:21:25 +08:00
web_search.go fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
web_search_filters.go fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
web_search_filters_test.go fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
web_search_provider.go feat(web-search): add Serply provider 2026-09-17 12:25:09 +08:00
web_search_provider_test.go feat(web-search): add Serply provider 2026-09-17 12:25:09 +08:00
web_search_test.go fix(web-search): normalize default tenant web search config at runtime 2026-04-30 15:28:35 +08:00
weknoracloud.go refactor(weknoracloud): update credential handling and configuration 2026-04-15 11:37:24 +08:00
wiki_page.go refactor(modelcontext): consolidate source_refs parsing, fixed data_analysis table, and identifier leak report (#3372) 2026-09-17 22:08:16 +08:00
wiki_page_test.go refactor(modelcontext): consolidate source_refs parsing, fixed data_analysis table, and identifier leak report (#3372) 2026-09-17 22:08:16 +08:00