WeKnora/internal/types
lyingbug c3f077ed18
fix(models): align configuration and multimodal embedding workflows (#4080)
* fix(models): align configuration defaults and multimodal workflows

* feat(image): enhance image retrieval diagnostics and filtering

- Implemented `retrieveObservedImagePool` to observe engine calls and record image recall decisions without altering results.
- Added detailed diagnostics in `summarizeImagePool` and `summarizeImageCandidateCut` to track image filtering decisions and scores.
- Introduced tests for image pool diagnostics to ensure accurate filtering and decision recording.
- Updated existing functions to utilize new retrieval and summarization methods for improved observability and accuracy in image handling.

* feat(embeddings): enhance image handling with new formats and message support

- Introduced `EmbeddingImageFormat` to manage image input formats, allowing for both object and messages formats.
- Updated `EmbeddingsSettings` to include new fields for `ImageFormat`, `ImagePrompt`, and `TextAsMessages` to support multimodal embedding workflows.
- Enhanced `Client` methods to build request bodies for images in both formats, ensuring compatibility with vLLM's requirements.
- Added tests to validate the new image message handling and ensure correct request formatting for different scenarios.
- Updated documentation to reflect changes in image input handling and configuration options.

* feat(similarity): add embedding similarity mode for enhanced document scoring

- Introduced a new `similarity` option in `ModelDebugOptions` to score documents and an optional image against a query.
- Updated `ModelDebugDrawer.vue` to support a new input mode for similarity, allowing users to select similarity as an input type.
- Enhanced the UI to display similarity results, including ranked candidates based on cosine similarity.
- Added tests to validate the functionality of the new similarity mode in embedding models.
- Updated localization files to include new strings related to similarity mode in multiple languages.

* feat(embeddings): add Qwen3-VL-Embedding support for multimodal input

- Introduced Qwen3-VL-Embedding model with support for both text and image inputs.
- Updated model configuration files to include compatibility settings for the new embedding type.
- Enhanced tests to validate the functionality of Qwen3-VL-Embedding, ensuring correct handling of inputs and outputs.
- Updated documentation to reflect the new model's capabilities and usage instructions.

* test(models): wrap embedding assertions to satisfy lint

* test(opensearch): isolate HTTP transports between parallel tests
2026-10-10 11:06:00 +08:00
..
interfaces feat: judge image hits by their image in rerank and show them to vision models (#4013) 2026-10-09 13:01:48 +08:00
agent.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
agent_share_source.go fix: harden shared agent source selector validation 2026-07-30 12:28:23 +08:00
agent_share_source_test.go fix: harden shared agent source selector validation 2026-07-30 12:28:23 +08:00
agent_test.go feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
agent_type_preset.go feat: Introduce agent type presets for smart-reasoning mode 2026-04-22 21:18:20 +08:00
artifact_versions.go feat(agent): unify sandbox tools and drop host preloaded skills (#3033) 2026-09-05 16:45:59 +08:00
artifact_versions_test.go feat(agent): unify sandbox tools and drop host preloaded skills (#3033) 2026-09-05 16:45:59 +08:00
audit_log.go feat(system-admin): add endpoint for creating users (#2722) 2026-08-20 19:51:10 +08:00
audit_log_test.go feat(system-admin): add endpoint for creating users (#2722) 2026-08-20 19:51:10 +08:00
browser_search.go feat(browser): polish connection settings, open tasks in a new window, and streamline prompts (#3193) 2026-09-11 21:28:58 +08:00
builtin_agent_config.go fix(i18n): 修复内置智能体名称与描述不随界面语言切换的问题 (#2828) 2026-08-27 11:53:45 +08:00
builtin_agent_localization_test.go fix(i18n): 修复内置智能体名称与描述不随界面语言切换的问题 (#2828) 2026-08-27 11:53:45 +08:00
builtin_models_config.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
builtin_models_config_test.go feat(models): let system admins edit builtin model config 2026-07-14 15:56:02 +08:00
caller.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
chat.go fix(llm): treat cut-off streams as broken and tighten LLM retries 2026-09-25 18:22:14 +08:00
chat_history_config.go feat: implement chat history and retrieval settings 2026-03-09 11:46:56 +08:00
chat_manage.go feat: judge image hits by their image in rerank and show them to vision models (#4013) 2026-10-09 13:01:48 +08:00
chunk.go feat: index and recall images with multimodal embedding models (#3683) 2026-10-08 18:19:53 +08:00
chunk_contextheader_test.go test(types): regression test that ContextHeader is hidden from JSON 2026-05-06 17:17:07 +08:00
cleanup.go init commit 2025-08-05 15:08:07 +08:00
config_redaction.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
config_redaction_sandbox_test.go feat(sandbox): add a graphical desktop to session sandboxes (#3245) 2026-09-14 20:37:37 +08:00
config_redaction_test.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
const.go fix(wiki): 唯一写出口 UpdatePage 拦住丢表格行的机器重写(agent 整页写入不再静默改短页面) (#3793) 2026-10-09 10:55:28 +08:00
context_checkpoint.go feat(agent): size agent history by the window and calibrate token estimates (#3413) 2026-09-18 22:51:31 +08:00
context_clone.go fix(wiki): 唯一写出口 UpdatePage 拦住丢表格行的机器重写(agent 整页写入不再静默改短页面) (#3793) 2026-10-09 10:55:28 +08:00
context_clone_coverage_test.go refactor(context): replace middleware.EmbedChannelContextKey with types.EmbedChannelContextKey 2026-08-13 18:00:02 +08:00
context_helpers.go fix(wiki): 唯一写出口 UpdatePage 拦住丢表格行的机器重写(agent 整页写入不再静默改短页面) (#3793) 2026-10-09 10:55:28 +08:00
context_helpers_test.go fix: retry document summaries before fallback 2026-08-05 19:29:32 +08:00
custom_agent.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
custom_agent_reasoning_effort_test.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
custom_agent_test.go feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
dataset.go init commit 2025-08-05 15:08:07 +08:00
datasource.go feat(datasource): add Seafile data source connector (#3616) 2026-10-08 16:18:09 +08:00
datasource_test.go feat(feishu): sync docx via blocks API with per-type downdrill (#2087) 2026-07-27 16:04:56 +08:00
docparser.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
embed_channel.go feat(im): add per-channel reply locale (#3455) 2026-09-23 12:01:01 +08:00
embed_channel_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
embedding.go feat: index and recall images with multimodal embedding models (#3683) 2026-10-08 18:19:53 +08:00
errors.go init commit 2025-08-05 15:08:07 +08:00
evaluation.go fix: support custom jieba dictionary directory 2026-06-08 20:10:37 +08:00
event_bus.go refactor: Implement event-driven streaming and enhance session management 2025-11-07 22:41:41 +08:00
extract_graph.go fix(graph): preserve relationship direction and document provenance (#3950) 2026-10-08 00:48:51 +08:00
extract_graph_test.go fix(graph): preserve relationship direction and document provenance (#3950) 2026-10-08 00:48:51 +08:00
faq.go fix: remove GPL converter and bundle third-party license sources 2026-09-08 15:19:10 +08:00
faq_sync.go feat(knowledge): enhance FAQ export functionality 2026-07-23 10:50:32 +08:00
faq_sync_test.go feat(knowledge): enhance FAQ export functionality 2026-07-23 10:50:32 +08:00
faq_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
file_reference.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
file_reference_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
gallery_attr_source.go feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
gallery_builtin_source.go feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
gallery_policy.go feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
generated_question_revision_test.go fix(chunks): retain questions across content edits 2026-07-28 20:23:48 +08:00
generated_question_source_id_test.go fix(knowledge): bound generated question source IDs 2026-07-28 20:23:48 +08:00
graph.go feat: implement preview of user level memory graph 2026-02-25 14:29:54 +08:00
image_actions.go feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
image_attrs.go feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
image_attrs_gallery.go feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
image_attrs_test.go feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
image_gallery.go feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
indexing_strategy.go feat: Add comprehensive documentation for WeKnora knowledge base and wiki features 2026-04-22 21:18:10 +08:00
json.go test(types): add unit test for JSON unmarshalling and validation 2026-04-22 21:17:36 +08:00
json_map.go refactor(knowledge): replace flat stage table with langfuse-style span tree 2026-05-28 15:14:45 +08:00
json_test.go test(types): add unit test for JSON unmarshalling and validation 2026-04-22 21:17:36 +08:00
knowledge.go feat(datasource): add Seafile data source connector (#3616) 2026-10-08 16:18:09 +08:00
knowledge_custom_metadata_test.go feat(knowledge): support editable chunks and metadata 2026-07-28 20:23:48 +08:00
knowledge_folder.go fix(knowledge): keep long non-ASCII folder names valid UTF-8 2026-09-07 11:50:32 +08:00
knowledge_folder_test.go fix(knowledge): keep long non-ASCII folder names valid UTF-8 2026-09-07 11:50:32 +08:00
knowledge_process.go fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
knowledge_process_test.go feat(docreader): support forcing PDF scanned-page OCR parsing 2026-06-25 17:30:22 +08:00
knowledge_profile.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
knowledge_profile_test.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
knowledge_span.go fix(spans): expand knowledge span name column and truncate long wiki names 2026-07-13 10:35:44 +08:00
knowledgebase.go feat: index and recall images with multimodal embedding models (#3683) 2026-10-08 18:19:53 +08:00
knowledgebase_auto_tag_test.go fix(knowledge): harden auto-tagging and let it respect manual tags 2026-08-07 15:35:58 +08:00
knowledgebase_profile.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
knowledgebase_test.go feat: index and recall images with multimodal embedding models (#3683) 2026-10-08 18:19:53 +08:00
legacy_persisted_json_test.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
locale.go feat(i18n): add Traditional Chinese (zh-TW) locale support (#3916) 2026-10-08 16:18:17 +08:00
locale_test.go feat(i18n): add Traditional Chinese (zh-TW) locale support (#3916) 2026-10-08 16:18:17 +08:00
mcp.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
mcp_endpoint.go feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
mcp_endpoint_tools.go feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
mcp_endpoint_tools_test.go feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
mcp_metadata.go feat(mcp)!: persist tool catalogs and add guided configuration (#3118) 2026-09-09 13:34:56 +08:00
mcp_oauth.go fix(mcp): coordinate OAuth token refresh and authorization polling 2026-07-23 15:35:24 +08:00
mcp_test.go feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling 2026-05-17 15:27:52 +08:00
memory.go fix(memory): rank memories by the query, not by importance 2026-09-13 11:43:17 +08:00
memory_extraction.go fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
memory_test.go feat(browser): polish connection settings, open tasks in a new window, and streamline prompts (#3193) 2026-09-11 21:28:58 +08:00
message.go feat(artifacts): let users delete generated files (#3447) 2026-09-20 13:01:46 +08:00
message_artifact.go feat(artifacts): let users delete generated files (#3447) 2026-09-20 13:01:46 +08:00
message_attachment_test.go feat(im): process file attachments in QA 2026-08-12 16:08:51 +08:00
message_suggestion.go feat(agent): add question suggestions with after-answer follow-ups 2026-07-14 11:17:25 +08:00
message_test.go feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
model.go fix(frontend): 修复复制模型时模型名称被改写的问题 (#3830) 2026-09-29 19:32:08 +08:00
model_catalog_config.go feat(models): add system admin model catalog console (#3662) 2026-09-24 16:31:45 +08:00
organization.go fix(security): close the low-severity shared-space findings (#3416) 2026-09-18 22:58:46 +08:00
parser_engine_config_test.go fix(parser): default ppt/pptx to markitdown when no engine rules (#2882) 2026-08-31 14:58:15 +08:00
placeholder.go feat(chat): keep prompt prefixes stable and send provider cache markers 2026-09-03 00:45:52 +08:00
placeholder_time_test.go feat(chat): keep prompt prefixes stable and send provider cache markers 2026-09-03 00:45:52 +08:00
principal.go fix(mcp): resolve MCP endpoint sessions by their storage ID 2026-09-22 16:53:10 +08:00
principal_test.go test(api-keys): add unit tests for tenant API key validation 2026-07-07 12:00:13 +08:00
prompt_instructions.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
prompt_instructions_test.go Enhance knowledge base configuration by normalizing and validating custom instruction fields. Implemented length checks for prompt instructions and added tests for validation logic. Updated GraphSettings to retain custom instructions when extraction is disabled. 2026-07-13 12:16:19 +08:00
qa_request.go fix(embed): 内嵌网页只传图片不输入文字时不再返回 400 2026-09-30 15:26:51 +08:00
qa_request_test.go fix(chat): preserve steering continuity and composer focus 2026-09-09 20:15:23 +08:00
question_suggestion_test.go feat(agent): add question suggestions with after-answer follow-ups 2026-07-14 11:17:25 +08:00
rerank.go feat: judge image hits by their image in rerank and show them to vision models (#4013) 2026-10-09 13:01:48 +08:00
resource.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
resource_test.go feat(agent): preview skill artifacts and verify installs without running them (#2823) 2026-08-27 10:30:32 +08:00
retrieval_config.go fix(retrieval): fix recall correctness, score scales, cost and agent tools (#3687) 2026-09-25 10:48:18 +08:00
retriever.go feat: judge image hits by their image in rerank and show them to vision models (#4013) 2026-10-09 13:01:48 +08:00
runtime_task.go feat(system): add cursor pagination for runtime queue tasks 2026-07-14 16:53:07 +08:00
sandbox_network_policy.go fix(sandbox): always require inbound credentials 2026-09-03 13:34:40 +08:00
sandbox_network_policy_test.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
sandbox_tenant_context_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
search.go feat: judge image hits by their image in rerank and show them to vision models (#4013) 2026-10-09 13:01:48 +08:00
search_scope_test.go refactor(agent): introduce modelcontext registry and fix handle-codec bugs 2026-07-27 11:05:43 +08:00
secret.go fix(security): harden secret redaction follow-ups for PR #1884 2026-07-02 14:41:42 +08:00
secret_test.go fix(security): harden secret redaction follow-ups for PR #1884 2026-07-02 14:41:42 +08:00
session.go feat(chat): add session reasoning controls and polish model configuration 2026-09-23 17:12:19 +08:00
session_access_test.go feat(sandbox): expose tenant skill management over HTTP and in the console (#2789) 2026-08-25 11:28:12 +08:00
session_fork.go feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
session_fork_test.go feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
shared_agent_access.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
shared_agent_access_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
source_locator.go fix(citations): 修复原文定位、高亮范围与 DocReader 启动失败 (#3794) 2026-09-27 20:32:26 +08:00
storagebackend.go feat(storage): add multi-instance storage backends per workspace 2026-07-15 14:54:39 +08:00
storagebackend_test.go feat(storage): add multi-instance storage backends per workspace 2026-07-15 14:54:39 +08:00
sync_item_error_test.go fix(feishu): resilient large-wiki sync — retry, streaming resume, failure visibility 2026-07-21 16:14:22 +08:00
system_setting.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
tag.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
task.go fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
task_dead_letter.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
task_initiator_test.go feat(audit): 知识库活动记录对 API Key 操作额外显示 Key 名称 (#3405) 2026-09-18 19:39:34 +08:00
task_pending_op.go fix(queue): order ClaimBatch by fail_count so retries cannot starve new work (#3444) 2026-09-21 13:00:29 +08:00
task_queue_test.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
temporary_document.go fix: harden shared agent source permissions 2026-07-30 12:28:23 +08:00
tenant.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
tenant_api_key.go feat(audit): 知识库活动记录对 API Key 操作额外显示 Key 名称 (#3405) 2026-09-18 19:39:34 +08:00
tenant_api_key_scope_test.go Add API key member and space capabilities 2026-07-07 12:00:13 +08:00
tenant_api_key_test.go Refine scoped API key capabilities 2026-07-07 12:00:13 +08:00
tenant_env_vars.go fix(sandbox): make skill env capture fill blanks only 2026-08-27 14:32:00 +08:00
tenant_env_vars_test.go fix(sandbox): make skill env capture fill blanks only 2026-08-27 14:32:00 +08:00
tenant_invitation.go feat(rbac): add multi-use share-link invitations for invite_only mode (#1504) 2026-05-27 19:42:01 +08:00
tenant_member.go Refine scoped API key capabilities 2026-07-07 12:00:13 +08:00
tenant_sandbox_config_entity.go feat(sandbox): 沙箱后端改为按空间配置,并补齐 E2B 模板与检测诊断 2026-08-11 15:42:02 +08:00
tenant_sandbox_config_entity_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
tenant_sandbox_config_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
tenant_skill.go fix(skills): upgrade outdated sandbox skills without rolling back the catalog or taking them offline (#3406) 2026-09-18 20:27:43 +08:00
tenant_test.go fix: preserve MinerU automatic PDF parsing 2026-08-03 15:37:07 +08:00
token_usage_test.go feat(agent): report and attribute per-turn LLM token usage 2026-08-24 10:36:19 +08:00
tracing.go feat(tracing): migrate langfuse client to OTLP/OTel wire protocol 2026-07-17 16:30:45 +08:00
user.go feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
user_resource_favorite.go feat(favorites): implement user favorites and recents functionality 2026-05-18 17:28:58 +08:00
vectorstore.go fix: use consistent qdrant env settings (#3960) 2026-10-07 23:59:56 +08:00
vectorstore_opensearch_test.go feat(retriever): activate OpenSearch k-NN driver (PR 3 of 3) 2026-05-29 16:32:27 +08:00
vectorstore_test.go fix: use consistent qdrant env settings (#3960) 2026-10-07 23:59:56 +08:00
web_search.go fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
web_search_filters.go fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
web_search_filters_test.go fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
web_search_provider.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
web_search_provider_test.go feat(web-search): add Serply provider 2026-09-17 12:25:09 +08:00
web_search_test.go fix(web-search): normalize default tenant web search config at runtime 2026-04-30 15:28:35 +08:00
weknoracloud.go refactor(weknoracloud): update credential handling and configuration 2026-04-15 11:37:24 +08:00
wiki_page.go fix(wiki): resolve titles for unloaded backlinks (#3931) 2026-10-09 10:50:52 +08:00
wiki_page_test.go refactor(modelcontext): consolidate source_refs parsing, fixed data_analysis table, and identifier leak report (#3372) 2026-09-17 22:08:16 +08:00