WeKnora/internal
GUO Zhaogong 8d7298fb5d
fix(graph): feed OCR chunks to graph extraction instead of image-link placeholders (#3092)
* fix(graph): feed OCR chunks to graph extraction instead of image-link placeholders

KnowledgePostProcessService.Handle gathered "text-like" chunks through
chunkService.ListChunksByKnowledgeID, which is text-only by design, so the
image_ocr / image_caption chunks written by multimodal parsing were never
returned. For a scanned PDF the text chunks contain nothing but markdown
image links, so graph extraction ran on empty input: most calls failed
with "unexpected end of JSON input" and the rest echoed the few-shot
example from the extract config into the graph.

- Fetch the exact chunk types the post-process enriches via
  chunkRepo.ListChunksByKnowledgeIDAndTypes (text, image_ocr,
  image_caption); the service now receives interfaces.ChunkRepository
  through dig like knowledgeService already does
- Build a dedicated graphChunks list for graph extraction: skip
  image_caption chunks (they duplicate the OCR text) and chunks that
  contain only image links (chunkHasExtractableText)
- graphChunkCount, the enqueue loop and the log line use graphChunks;
  summary, wiki, auto-tag and question generation keep using textChunks
- Tests: fake ChunkRepository in the wiki-enqueue test, table test for
  chunkHasExtractableText

* fix(graph): skip figure OCR and harden empty-chunk detection

Reuse the CommonMark image scanner plus extractRealText so titles with
parentheses and <image> wrappers are not treated as prose. Only enqueue
OCR for graph extraction when the parent text chunk has no extractable
text, and skip link-only chunks for question generation as well.

* style(graph): wrap long post-process extract log line

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-08 19:23:13 +08:00
..
agent fix(mcp): require schema discovery before calls and render tool results (#3113) 2026-09-08 19:15:13 +08:00
application fix(graph): feed OCR chunks to graph extraction instead of image-link placeholders (#3092) 2026-09-08 19:23:13 +08:00
assets feat(assets): add ASR test audio file and embed it in the application 2026-04-02 21:27:27 +08:00
common fix(markdown): sanitize invalid UTF-8 and NUL bytes 2026-08-31 12:24:35 +08:00
config feat: support complex password (#2929) 2026-09-01 11:50:10 +08:00
container feat(sandbox): add an interactive terminal and files tab beside chat (#3110) 2026-09-08 17:43:15 +08:00
database feat(mcp): add per-tool enable toggle (#2989) 2026-09-05 17:19:20 +08:00
datasource feat(datasource): persist source created time; Feishu uses content edit time for source_updated_at (#3083) 2026-09-07 19:39:01 +08:00
errors feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
event feat(agent): keep long sandbox turns from truncating, overflowing, or rewriting files 2026-09-02 22:09:09 +08:00
filetransport refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
handler feat(sandbox): add an interactive terminal and files tab beside chat (#3110) 2026-09-08 17:43:15 +08:00
im fix(feishu): preserve images embedded in post messages 2026-09-05 16:48:16 +08:00
infrastructure fix(graph): feed OCR chunks to graph extraction instead of image-link placeholders (#3092) 2026-09-08 19:23:13 +08:00
ipclass fix: share IP classification between SSRF and sandbox URL guards 2026-09-01 15:35:14 +08:00
logger fix(sandbox): record product-level Langfuse spans and detach idle sweeps 2026-08-31 19:42:32 +08:00
mcp refactor(agent): discover and invoke MCP tools on demand 2026-09-08 12:01:36 +08:00
middleware feat(sandbox): add an interactive terminal and files tab beside chat (#3110) 2026-09-08 17:43:15 +08:00
modelcontext refactor(agent): discover and invoke MCP tools on demand 2026-09-08 12:01:36 +08:00
models fix(chat): send one completion-token field per provider (#3082) 2026-09-07 19:16:27 +08:00
ratelimit feat(embed): add backend embed channels with auth and rate limiting (#1657) 2026-06-12 23:10:39 +08:00
router feat(sandbox): add an interactive terminal and files tab beside chat (#3110) 2026-09-08 17:43:15 +08:00
runtime fix(redis): harden TLS tests and surface TLS config at startup 2026-07-06 23:04:52 +08:00
sandbox feat(sandbox): add an interactive terminal and files tab beside chat (#3110) 2026-09-08 17:43:15 +08:00
searchutil fix(chat): keep OCR/caption text of scanned-page image hits in merge (#3064) 2026-09-07 11:38:47 +08:00
storageallowlist fix(storage): align default provider fallback with STORAGE_ALLOW_LIST 2026-07-15 11:05:00 +08:00
storageurl refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
stream feat(redis): support TLS for Redis connections (#1930) 2026-07-06 23:02:02 +08:00
textconv fix: remove GPL converter and bundle third-party license sources 2026-09-08 15:19:10 +08:00
tracing/langfuse fix(observability): nest follow-up completions under the chat trace 2026-08-31 19:42:32 +08:00
types feat(sandbox): add an interactive terminal and files tab beside chat (#3110) 2026-09-08 17:43:15 +08:00
utils fix(security): reject traversal in local prefixes, task IDs, and wiki sort 2026-09-07 11:23:45 +08:00