WeKnora/internal/agent
ochan.kwon b291381cc2 feat: auto-describe MCP tool result images via VLM
When an MCP tool returns image content items, the agent engine now
automatically generates text descriptions using the configured VLM
model and appends them to the tool result message. This makes image
content accessible to LLMs that receive tool results as text.

Images are not passed directly to the LLM because Chat Completions API
does not reliably support images in tool role messages across providers
(tested: gpt-5.4 silently ignores them, Qwen3.5 processes them).

Changes:
- Add Images field to ToolResult for preserving base64 data URIs
- Add extractContentAndImages() with MIME whitelist, size/count limits
- Add ImageDescriberFunc injection to AgentEngine (no vlm package dep)
- Add describeImages() with graceful error handling and ctx cancellation
- Add VLM model injection in CreateAgentEngine (same pattern as
  SetAppConfig/SetSkillsManager)
- Redact image base64 from Data map to prevent log/SSE exposure
2026-03-25 21:38:15 +08:00
..
skills feat: adapt preloaded skills to smart-reasoning agent 2026-02-04 20:08:49 +08:00
tools feat: auto-describe MCP tool result images via VLM 2026-03-25 21:38:15 +08:00
const.go refactor: Remove agent_test command-line tool and associated files to streamline project structure and eliminate unused components 2025-12-01 17:24:57 +08:00
engine.go feat: auto-describe MCP tool result images via VLM 2026-03-25 21:38:15 +08:00
engine_image_test.go feat: auto-describe MCP tool result images via VLM 2026-03-25 21:38:15 +08:00
prompts.go feat: enhance localization and configuration support 2026-03-17 22:23:24 +08:00