WeKnora/internal
lyingbug c3f077ed18
fix(models): align configuration and multimodal embedding workflows (#4080)
* fix(models): align configuration defaults and multimodal workflows

* feat(image): enhance image retrieval diagnostics and filtering

- Implemented `retrieveObservedImagePool` to observe engine calls and record image recall decisions without altering results.
- Added detailed diagnostics in `summarizeImagePool` and `summarizeImageCandidateCut` to track image filtering decisions and scores.
- Introduced tests for image pool diagnostics to ensure accurate filtering and decision recording.
- Updated existing functions to utilize new retrieval and summarization methods for improved observability and accuracy in image handling.

* feat(embeddings): enhance image handling with new formats and message support

- Introduced `EmbeddingImageFormat` to manage image input formats, allowing for both object and messages formats.
- Updated `EmbeddingsSettings` to include new fields for `ImageFormat`, `ImagePrompt`, and `TextAsMessages` to support multimodal embedding workflows.
- Enhanced `Client` methods to build request bodies for images in both formats, ensuring compatibility with vLLM's requirements.
- Added tests to validate the new image message handling and ensure correct request formatting for different scenarios.
- Updated documentation to reflect changes in image input handling and configuration options.

* feat(similarity): add embedding similarity mode for enhanced document scoring

- Introduced a new `similarity` option in `ModelDebugOptions` to score documents and an optional image against a query.
- Updated `ModelDebugDrawer.vue` to support a new input mode for similarity, allowing users to select similarity as an input type.
- Enhanced the UI to display similarity results, including ranked candidates based on cosine similarity.
- Added tests to validate the functionality of the new similarity mode in embedding models.
- Updated localization files to include new strings related to similarity mode in multiple languages.

* feat(embeddings): add Qwen3-VL-Embedding support for multimodal input

- Introduced Qwen3-VL-Embedding model with support for both text and image inputs.
- Updated model configuration files to include compatibility settings for the new embedding type.
- Enhanced tests to validate the functionality of Qwen3-VL-Embedding, ensuring correct handling of inputs and outputs.
- Updated documentation to reflect the new model's capabilities and usage instructions.

* test(models): wrap embedding assertions to satisfy lint

* test(opensearch): isolate HTTP transports between parallel tests
2026-10-10 11:06:00 +08:00
..
agent fix(agent): align web_search count limit with the configured maximum (#4011) 2026-10-10 10:27:52 +08:00
application fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
assets feat(assets): add ASR test audio file and embed it in the application 2026-04-02 21:27:27 +08:00
browserskill fix(browser): refine connection and preview UI with extension version 2026-09-23 10:57:59 +08:00
common fix(markdown): sanitize invalid UTF-8 and NUL bytes 2026-08-31 12:24:35 +08:00
config feat(auth): 新增仅限邀请注册档位 (#3900) 2026-09-30 13:32:55 +08:00
container feat(datasource): add Seafile data source connector (#3616) 2026-10-08 16:18:09 +08:00
database revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
datasource feat(datasource/dingtalk): 摄取钉钉"上传文件"(docx/pptx/xlsx/pdf) (#3786) 2026-10-09 10:56:04 +08:00
embedpolicy fix(embed): enforce host origin allowlists on framed pages 2026-09-14 18:03:57 +08:00
errors feat(knowledge): flag documents that stop making progress (#3557) 2026-09-22 21:34:08 +08:00
event fix(agent): end the turn when the completion cap truncates an answer (#3449) 2026-09-20 13:17:50 +08:00
filetransport refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
handler fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
im fix(im): retry Yunzhijia attachment downloads (#3809) 2026-10-09 10:47:58 +08:00
infrastructure fix(websearch): return results for non-webpage Metaso scopes (#4010) 2026-10-10 10:25:40 +08:00
ipclass fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
localsandbox feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
logger fix(logger): keep LLM debug records inside their directory 2026-09-25 18:22:14 +08:00
mcp fix(mcp): report advertised directory failures in connection tests (#3905) 2026-10-09 12:01:27 +08:00
mcpserver fix(mcp): return accessible image URLs from existing tools (#3650) 2026-09-24 11:56:25 +08:00
middleware fix(middleware): stop buffering whole request bodies in the access log (#3927) 2026-10-08 01:29:05 +08:00
modelcontext fix(agent): align web_search count limit with the configured maximum (#4011) 2026-10-10 10:27:52 +08:00
models fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
ratelimit feat(embed): add backend embed channels with auth and rate limiting (#1657) 2026-06-12 23:10:39 +08:00
reranking fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
router fix(knowledge): claim cloned and released document images in the resource catalog (#3752) 2026-10-09 13:01:26 +08:00
runtime fix(server): run ResourceCleaner on signal shutdown instead of racing Fatalf exit (#3804) 2026-09-28 23:37:36 +08:00
sandbox fix(sandbox): omit env overrides when creating Aliyun E2B sandboxes from snapshots (#4054) 2026-10-09 14:24:54 +08:00
searchutil feat: judge image hits by their image in rerank and show them to vision models (#4013) 2026-10-09 13:01:48 +08:00
sourceloc fix(citations): 修复原文定位、高亮范围与 DocReader 启动失败 (#3794) 2026-09-27 20:32:26 +08:00
storageallowlist fix(storage): align default provider fallback with STORAGE_ALLOW_LIST 2026-07-15 11:05:00 +08:00
storageurl feat: judge image hits by their image in rerank and show them to vision models (#4013) 2026-10-09 13:01:48 +08:00
stream feat(session): in-place rewind with workspace checkpoint reset (#3469) 2026-09-20 21:01:00 +08:00
textconv fix: remove GPL converter and bundle third-party license sources 2026-09-08 15:19:10 +08:00
tracing/langfuse fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
types fix(models): align configuration and multimodal embedding workflows (#4080) 2026-10-10 11:06:00 +08:00
utils feat(datasource): add Seafile data source connector (#3616) 2026-10-08 16:18:09 +08:00