WeKnora/internal/types
ochan.kwon b291381cc2 feat: auto-describe MCP tool result images via VLM
When an MCP tool returns image content items, the agent engine now
automatically generates text descriptions using the configured VLM
model and appends them to the tool result message. This makes image
content accessible to LLMs that receive tool results as text.

Images are not passed directly to the LLM because Chat Completions API
does not reliably support images in tool role messages across providers
(tested: gpt-5.4 silently ignores them, Qwen3.5 processes them).

Changes:
- Add Images field to ToolResult for preserving base64 data URIs
- Add extractContentAndImages() with MIME whitelist, size/count limits
- Add ImageDescriberFunc injection to AgentEngine (no vlm package dep)
- Add describeImages() with graceful error handling and ctx cancellation
- Add VLM model injection in CreateAgentEngine (same pattern as
  SetAppConfig/SetSkillsManager)
- Redact image base64 from Data map to prevent log/SSE exposure
2026-03-25 21:38:15 +08:00
..
interfaces feat(knowledge): offload manual processing to async tasks 2026-03-18 21:56:34 +08:00
agent.go feat: auto-describe MCP tool result images via VLM 2026-03-25 21:38:15 +08:00
builtin_agent_config.go feat: enhance localization and configuration support 2026-03-17 22:23:24 +08:00
chat.go feat: Enhance chat message handling and agent completion events 2025-11-08 00:58:26 +08:00
chat_history_config.go feat: implement chat history and retrieval settings 2026-03-09 11:46:56 +08:00
chat_manage.go feat: enhance localization and configuration support 2026-03-17 22:23:24 +08:00
chunk.go feat: implement parent-child chunking strategy for enhanced context management 2026-03-05 18:14:14 +08:00
cleanup.go init commit 2025-08-05 15:08:07 +08:00
const.go feat: enhance localization and configuration support 2026-03-17 22:23:24 +08:00
context_helpers.go refactor: update session title generation and language context handling 2026-03-17 22:23:24 +08:00
custom_agent.go fix: Adjust temperature validation to allow zero as a valid setting and default only for negative values. 2026-03-20 18:21:39 +08:00
dataset.go init commit 2025-08-05 15:08:07 +08:00
docparser.go feat: implement parent-child chunking strategy for enhanced context management 2026-03-05 18:14:14 +08:00
embedding.go perf: 优化FAQ索引更新和删除性能 2026-01-14 17:30:04 +08:00
errors.go init commit 2025-08-05 15:08:07 +08:00
evaluation.go refactor: Update tenant ID handling across the application 2025-11-25 16:30:54 +08:00
event_bus.go refactor: Implement event-driven streaming and enhance session management 2025-11-07 22:41:41 +08:00
extract_graph.go feat(knowledge): offload manual processing to async tasks 2026-03-18 21:56:34 +08:00
faq.go feat(types): enhance FAQ metadata normalization and sanitization 2026-03-19 18:20:09 +08:00
faq_test.go feat(chunk): implement batch deletion for chunks to avoid MySQL placeholder limit 2026-03-19 18:20:09 +08:00
graph.go feat: implement preview of user level memory graph 2026-02-25 14:29:54 +08:00
json.go init commit 2025-08-05 15:08:07 +08:00
knowledge.go fix: handle empty metadata case in Knowledge struct 2026-03-02 21:21:49 +08:00
knowledgebase.go feat: update chunk size configurations for knowledge base processing 2026-03-05 18:14:14 +08:00
mcp.go fix: use service name for MCP tool naming with collision protection (#715) 2026-03-21 10:22:05 +08:00
memory.go feat: implement preview of user level memory graph 2026-02-25 14:29:54 +08:00
message.go feat: implement image upload and multimodal support 2026-03-16 02:26:47 +08:00
model.go feat(provider): add Novita AI as LLM provider 2026-03-24 22:10:42 +08:00
organization.go feat(api): integrate per-organization resource counts into organization listing 2026-02-09 11:40:25 +08:00
placeholder.go feat: enhance localization and configuration support 2026-03-17 22:23:24 +08:00
qa_request.go refactor: consolidate QA request handling and improve session service interface 2026-03-16 02:26:47 +08:00
retrieval_config.go feat: implement chat history and retrieval settings 2026-03-09 11:46:56 +08:00
retriever.go feat: support weaviate vectordb for knowledge retrieve 2026-03-09 10:36:16 +08:00
search.go feat: optimize hybrid search by grouping targets and reusing query embeddings 2026-03-16 02:26:47 +08:00
session.go feat: 添加思考模式配置支持 2025-12-31 15:36:20 +08:00
tag.go perf: batch tag reference counting 2026-01-20 18:19:44 +08:00
tenant.go feat: implement chat history and retrieval settings 2026-03-09 11:46:56 +08:00
user.go fix: improve auth error messages and unify username validation length 2026-03-15 23:59:50 +08:00
web_search.go feat: Update web search providers and enhance knowledge base management 2025-11-10 22:45:27 +08:00