WeKnora/internal/handler
lyingbug c3f077ed18
fix(models): align configuration and multimodal embedding workflows (#4080)
* fix(models): align configuration defaults and multimodal workflows

* feat(image): enhance image retrieval diagnostics and filtering

- Implemented `retrieveObservedImagePool` to observe engine calls and record image recall decisions without altering results.
- Added detailed diagnostics in `summarizeImagePool` and `summarizeImageCandidateCut` to track image filtering decisions and scores.
- Introduced tests for image pool diagnostics to ensure accurate filtering and decision recording.
- Updated existing functions to utilize new retrieval and summarization methods for improved observability and accuracy in image handling.

* feat(embeddings): enhance image handling with new formats and message support

- Introduced `EmbeddingImageFormat` to manage image input formats, allowing for both object and messages formats.
- Updated `EmbeddingsSettings` to include new fields for `ImageFormat`, `ImagePrompt`, and `TextAsMessages` to support multimodal embedding workflows.
- Enhanced `Client` methods to build request bodies for images in both formats, ensuring compatibility with vLLM's requirements.
- Added tests to validate the new image message handling and ensure correct request formatting for different scenarios.
- Updated documentation to reflect changes in image input handling and configuration options.

* feat(similarity): add embedding similarity mode for enhanced document scoring

- Introduced a new `similarity` option in `ModelDebugOptions` to score documents and an optional image against a query.
- Updated `ModelDebugDrawer.vue` to support a new input mode for similarity, allowing users to select similarity as an input type.
- Enhanced the UI to display similarity results, including ranked candidates based on cosine similarity.
- Added tests to validate the functionality of the new similarity mode in embedding models.
- Updated localization files to include new strings related to similarity mode in multiple languages.

* feat(embeddings): add Qwen3-VL-Embedding support for multimodal input

- Introduced Qwen3-VL-Embedding model with support for both text and image inputs.
- Updated model configuration files to include compatibility settings for the new embedding type.
- Enhanced tests to validate the functionality of Qwen3-VL-Embedding, ensuring correct handling of inputs and outputs.
- Updated documentation to reflect the new model's capabilities and usage instructions.

* test(models): wrap embedding assertions to satisfy lint

* test(opensearch): isolate HTTP transports between parallel tests
2026-10-10 11:06:00 +08:00
..
dto fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
session feat: judge image hits by their image in rerank and show them to vision models (#4013) 2026-10-09 13:01:48 +08:00
audit_log.go Remove redundant Wiki operation log 2026-07-28 11:18:33 +08:00
audit_log_test.go feat(knowledge): add scoped activity audit trail 2026-07-23 10:50:32 +08:00
auth.go feat(auth): 新增仅限邀请注册档位 (#3900) 2026-09-30 13:32:55 +08:00
auth_browser_preferences_test.go feat(browser): polish connection settings, open tasks in a new window, and streamline prompts (#3193) 2026-09-11 21:28:58 +08:00
auth_change_password_test.go feat: support complex password (#2929) 2026-09-01 11:50:10 +08:00
auth_desktop_security_test.go fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
auth_oidc_start_test.go feat(auth): add /auth/oidc/start endpoint for direct 302 OIDC login 2026-08-11 15:41:48 +08:00
auth_register_by_invite.go feat(auth): 新增仅限邀请注册档位 (#3900) 2026-09-30 13:32:55 +08:00
auth_register_by_invite_tenant_test.go feat(auth): 新增仅限邀请注册档位 (#3900) 2026-09-30 13:32:55 +08:00
auth_register_invite_only_test.go feat(auth): 新增仅限邀请注册档位 (#3900) 2026-09-30 13:32:55 +08:00
chunk.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
chunker_debug.go fix(chunking): 修复内联 HTML 表格与保护模式导致的超长 chunk (#3404) 2026-09-18 21:10:17 +08:00
chunker_debug_test.go fix(chunking): normalize line endings before splitting / 在分块前统一换行符 2026-08-05 19:24:45 +08:00
custom_agent.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
custom_agent_api_key_scope_test.go feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
custom_agent_avatar_test.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
custom_agent_reasoning_effort_test.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
custom_agent_sandbox_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
custom_agent_test.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
datasource.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
datasource_credentials.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
datasource_schedule_test.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
datasource_test.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
deployment_capabilities.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
deployment_capabilities_test.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
document_write_scope_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
embed_allowed_origins_test.go fix(embed): enforce host origin allowlists on framed pages 2026-09-14 18:03:57 +08:00
embed_channel.go fix(embed): warn at startup when no signing key is configured 2026-09-24 20:29:17 +08:00
embed_channel_exchange_test.go refactor(context): replace middleware.EmbedChannelContextKey with types.EmbedChannelContextKey 2026-08-13 18:00:02 +08:00
embed_flow_test.go fix(security): close the low-severity shared-space findings (#3416) 2026-09-18 22:58:46 +08:00
embed_resource_urls_test.go fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
embed_session_test.go fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
evaluation.go feat: 新增API Key认证支持,优化Swagger文档安全配置 2025-12-22 15:01:09 +08:00
faq.go feat(faq): filter entries by enabled status (#3157) 2026-09-10 14:42:59 +08:00
faq_enabled_filter_test.go feat(faq): filter entries by enabled status (#3157) 2026-09-10 14:42:59 +08:00
host_project_picker.go feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) 2026-09-22 19:35:03 +08:00
host_project_picker_test.go feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) 2026-09-22 19:35:03 +08:00
im.go feat(i18n): add Traditional Chinese (zh-TW) locale support (#3916) 2026-10-08 16:18:17 +08:00
im_locale_test.go feat(im): add per-channel reply locale (#3455) 2026-09-23 12:01:01 +08:00
im_platform_test.go feat(im,datasource): support Lark, Feishu's international edition 2026-07-17 11:05:54 +08:00
initialization.go fix(model): treat 400 as reachable in chat connection test again (#3929) 2026-10-08 14:02:35 +08:00
initialization_chat_check_test.go fix(model): treat 400 as reachable in chat connection test again (#3929) 2026-10-08 14:02:35 +08:00
initialization_config_redaction_test.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
initialization_extract_relations_test.go fix(chat_pipeline): treat non-JSON model refusals as skipped graph extraction (#3606) 2026-10-08 14:57:11 +08:00
initialization_fill_secrets_test.go fix(initialization): pass stored ExtraConfig to model connection test 2026-09-07 11:43:44 +08:00
initialization_model_merge_test.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
initialization_model_tenant_test.go fix(initialization): stamp the KB's tenant onto created models 2026-09-17 13:09:34 +08:00
initialization_owner_test.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
kb_access.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
kb_access_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
knowledge.go fix(knowledge): reject invalid JSON uploads before async parse (#3360) 2026-09-25 22:23:27 +08:00
knowledge_api_key_scope_test.go fix(api-keys): close KB scope gaps in handlers and CloneContext 2026-07-07 12:00:13 +08:00
knowledge_download.go feat(knowledge): add batch file downloads (#3300) 2026-09-16 15:40:10 +08:00
knowledge_download_test.go feat(knowledge): add batch file downloads (#3300) 2026-09-16 15:40:10 +08:00
knowledge_folder_test.go feat(knowledge): store folder path for folder uploads and expose folder tree API 2026-08-04 20:45:29 +08:00
knowledge_last_activity_test.go feat(knowledge): flag documents that stop making progress (#3557) 2026-09-22 21:34:08 +08:00
knowledge_move_gate_test.go fix: reject cross-store reuse_vectors knowledge move 2026-06-05 15:00:41 +08:00
knowledge_mutation_admission_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
knowledge_ownership_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
knowledge_preview_security_test.go fix(security): harden outbound requests and file previews 2026-07-02 19:15:46 +08:00
knowledge_sort_test.go feat: add configurable sorting to document and resource lists (#3318) 2026-09-23 12:00:21 +08:00
knowledge_spans_test.go feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535) 2026-09-25 22:22:58 +08:00
knowledge_tag_ids_test.go fix(knowledge): harden multi-tag data integrity and UX follow-ups 2026-06-26 10:52:52 +08:00
knowledge_transfer_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
knowledgebase.go fix(kb): let the update API set vlm_config (#3704) 2026-10-08 14:37:23 +08:00
knowledgebase_copy_preflight_test.go Simplify KB duplicate API and improve naming/docs coverage. 2026-07-08 20:55:54 +08:00
knowledgebase_hybrid_search_resource_urls_test.go feat: judge image hits by their image in rerank and show them to vision models (#4013) 2026-10-09 13:01:48 +08:00
knowledgebase_hybrid_search_test.go feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675) 2026-09-24 20:41:49 +08:00
knowledgebase_not_found_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
knowledgebase_pr3_response_test.go feat(knowledge-base): validate vector store bindings on create, copy, and delete 2026-05-18 15:58:46 +08:00
knowledgebase_pr5_list_test.go feat: expose KB ↔ vector store binding in list, editor, and detail UI 2026-05-22 17:40:10 +08:00
knowledgebase_request_test.go feat(knowledge-base): validate vector store bindings on create, copy, and delete 2026-05-18 15:58:46 +08:00
knowledgebase_vlm_update_test.go fix(kb): let the update API set vlm_config (#3704) 2026-10-08 14:37:23 +08:00
list_pagination.go fix(handler): validate knowledge search pagination 2026-08-24 14:16:35 +08:00
list_pagination_test.go fix(handler): validate knowledge search pagination 2026-08-24 14:16:35 +08:00
mcp_credentials.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
mcp_endpoint.go feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
mcp_metadata.go feat(mcp)!: persist tool catalogs and add guided configuration (#3118) 2026-09-09 13:34:56 +08:00
mcp_metadata_test.go feat(mcp)!: persist tool catalogs and add guided configuration (#3118) 2026-09-09 13:34:56 +08:00
mcp_oauth.go fix(mcp): coordinate OAuth token refresh and authorization polling 2026-07-23 15:35:24 +08:00
mcp_service.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
mcp_tool_approval_test.go feat(mcp): add per-tool enable toggle (#2989) 2026-09-05 17:19:20 +08:00
mcp_usage_instructions.go feat(i18n): add Traditional Chinese (zh-TW) locale support (#3916) 2026-10-08 16:18:17 +08:00
mcp_usage_instructions_test.go feat(mcp)!: streamline setup and use short routing handles 2026-09-09 16:11:52 +08:00
me_env_var.go feat(sandbox): inject per-user and workspace skill env vars at execution 2026-08-27 14:32:00 +08:00
me_env_var_test.go feat(sandbox): inject per-user and workspace skill env vars at execution 2026-08-27 14:32:00 +08:00
memory.go fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
memory_consistency_test.go fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
message.go fix(api): address resource_urls review follow-ups 2026-08-05 14:42:51 +08:00
message_resource_urls_test.go fix(api): close the resource_urls=public gaps found in review 2026-08-05 14:42:51 +08:00
message_session_not_found_test.go fix(chat): stabilize history pagination and keep message order 2026-05-20 21:03:39 +08:00
message_suggestion.go fix(sessions): 放行 embed 运行时读取自身会话,修复追问建议 500 与刷新丢对话 2026-07-30 17:51:34 +08:00
message_suggestion_test.go fix(sessions): 放行 embed 运行时读取自身会话,修复追问建议 500 与刷新丢对话 2026-07-30 17:51:34 +08:00
model.go fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
model_catalog.go fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
model_catalog_visibility_test.go fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
model_copy_test.go fix(frontend): 修复复制模型时模型名称被改写的问题 (#3830) 2026-09-29 19:32:08 +08:00
model_credentials.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
model_debug_images_test.go fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
model_debug_similarity.go fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
model_secret_extra_test.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
model_spec_flow_test.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
model_test.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
organization.go fix(security): close the low-severity shared-space findings (#3416) 2026-09-18 22:58:46 +08:00
organization_discovery_security_test.go fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
organization_invite_test.go fix(security): close the low-severity shared-space findings (#3416) 2026-09-18 22:58:46 +08:00
organization_share_path_test.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
rbac_lookups.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
rbac_lookups_test.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
sandbox_check.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
sandbox_check_test.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
sandbox_config.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
sandbox_config_test.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
sandbox_skill.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
sandbox_skill_env_test.go feat(sandbox): inject per-user and workspace skill env vars at execution 2026-08-27 14:32:00 +08:00
sandbox_skill_test.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
shared_agent_access.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
shared_agent_access_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
shared_agent_picker_test.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
skill_catalog.go refactor(skills): enhance skill upload size management and error handling 2026-09-02 15:08:05 +08:00
skill_catalog_test.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
skill_handler.go fix(docs): make `make docs` reproduce the committed Swagger again (#3751) 2026-10-08 01:26:08 +08:00
skill_handler_test.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
storage_allowlist.go fix(storage): align default provider fallback with STORAGE_ALLOW_LIST 2026-07-15 11:05:00 +08:00
storagebackend.go feat(storage): add storage backend management API and UI enhancements 2026-07-15 14:54:39 +08:00
system.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
system_admin_audit_test.go test(system): cover system.* audit constants + emitAdminAudit contract 2026-05-26 21:13:56 +08:00
system_model_catalog.go feat(models): add system admin model catalog console (#3662) 2026-09-24 16:31:45 +08:00
system_model_catalog_test.go feat(models): add system admin model catalog console (#3662) 2026-09-24 16:31:45 +08:00
system_parser_check_test.go fix(security): close init config and parser-check redaction gaps 2026-07-02 14:41:42 +08:00
system_password_reset_test.go feat(system): add admin password reset with session revocation 2026-07-13 20:11:00 +08:00
system_runtime_test.go feat(knowledge): flag documents that stop making progress (#3557) 2026-09-22 21:34:08 +08:00
system_storage_engine_test.go feat(storage): improve storage connectivity error handling and multi-instance support 2026-07-15 14:54:39 +08:00
system_storage_ssrf_test.go fix(security): close remaining GHSA gaps from security triage 2026-07-02 15:03:53 +08:00
system_user_create_test.go refactor(system): drop idempotent flag from create-user response body 2026-09-07 17:30:47 +08:00
tag.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
tag_delete_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
task_progress_auth.go fix(security): enforce KB access on wiki reads and related IDOR paths 2026-07-02 12:06:22 +08:00
task_progress_auth_test.go fix(security): enforce KB access on wiki reads and related IDOR paths 2026-07-02 12:06:22 +08:00
tenant.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
tenant_api_key_validation_test.go feat: support updating API key configuration 2026-08-12 11:43:03 +08:00
tenant_invitation.go fix(invitation): harden auto-accept follow-ups from code review 2026-08-10 15:47:29 +08:00
tenant_invitation_accept_by_token_test.go feat: 邀请链接在 invite_only 模式下改为登录后加入空间 2026-08-10 14:57:38 +08:00
tenant_invitation_auto_accept_test.go fix(invitation): harden auto-accept follow-ups from code review 2026-08-10 15:47:29 +08:00
tenant_invite_link.go fix: prevent API keys from assigning owner role 2026-07-23 16:04:42 +08:00
tenant_member.go fix(invitation): harden auto-accept follow-ups from code review 2026-08-10 15:47:29 +08:00
tenant_member_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
tenant_parser_ssrf_test.go fix(security): redact IM list credentials and harden MinerU SSRF 2026-07-02 17:58:20 +08:00
tenant_policy.go feat(auth): support tenantless provisioning and gated self-service workspace creation 2026-07-13 18:13:51 +08:00
tenant_secret_disclosure_test.go refactor(api-keys): remove API key reset endpoint and update documentation 2026-07-07 12:00:13 +08:00
tenant_self_service_policy_test.go feat: support complex password (#2929) 2026-09-01 11:50:10 +08:00
upload_limit.go refactor(skills): improve skill bundle size handling and error management 2026-09-02 15:08:05 +08:00
upload_limit_test.go refactor(skills): improve skill bundle size handling and error management 2026-09-02 15:08:05 +08:00
user_resource_favorite.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
vectorstore.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
vectorstore_env_test.go fix: use consistent qdrant env settings (#3960) 2026-10-07 23:59:56 +08:00
web_search.go docs(api): add and fix swag annotations on handlers 2026-05-12 13:16:58 +08:00
web_search_provider.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
web_search_provider_credentials.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
wechat_qrcode.go docs(api): add and fix swag annotations on handlers 2026-05-12 13:16:58 +08:00
weknoracloud.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
wiki_page.go fix(wiki): resolve titles for unloaded backlinks (#3931) 2026-10-09 10:50:52 +08:00
wiki_page_titles_test.go fix(wiki): resolve titles for unloaded backlinks (#3931) 2026-10-09 10:50:52 +08:00
wiki_rebuild_links_failure_test.go fix(wiki): 重建链接逐页写失败不再静默,半成品不再回 200 (#3884) 2026-10-08 01:25:53 +08:00
wiki_search_test.go feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535) 2026-09-25 22:22:58 +08:00