WeKnora/internal/infrastructure/docparser
wizardchen 7aca1017db fix(docparser): address review feedback on PR #1404
Three follow-up fixes on top of the MinerU markdown preservation work:

- Stop applying normalizeMinerUMarkdown inside ResolveAndStore. The
  helper is already called by MinerUReader.Read, and ResolveAndStore is
  shared by every parser (docreader, session attachments, ...). Running
  the heading/image unescape regexes globally would silently rewrite
  content (including inside fenced code blocks) for non-MinerU sources.

- Recognize MinerU image references whose path contains spaces, e.g.
  "images/第 1 页.jpg". The previous regex used in
  extractImageRefsFromContent disallowed whitespace in the URL group,
  so such images were never matched and never persisted. Use a
  whitespace-tolerant pattern aligned with ResolveAndStore's own
  imgPattern.

- Deduplicate uploads when the same MinerU image is referenced under
  multiple path forms (e.g. "images/foo.png" vs "./images/foo.png").
  saveReferencedImage now caches by ref.Filename in addition to the
  raw ref path, so the second variant reuses the previously stored
  ServingURL instead of writing the same bytes to object storage
  again.

Tests added:
- TestProcessImagesMatchesPathsWithSpaces
- TestResolveAndStoreDedupsSameImageRefVariants
2026-05-21 11:42:56 +08:00
..
builtin_converter.go fix(docparser): preserve standalone image uploads from icon filter 2026-04-29 11:58:02 +08:00
engine_registry.go refactor(weknoracloud): update credential handling and configuration 2026-04-15 11:37:24 +08:00
grpc_parser.go fix(docreader/auth): harden gRPC TLS/Token rollout from #1359 2026-05-16 21:45:56 +08:00
helpers.go feat(weknoracloud): implement credential management and status checks 2026-04-14 17:36:01 +08:00
http_parser.go feat(weknoracloud): implement credential management and status checks 2026-04-14 17:36:01 +08:00
image_resolver.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
image_resolver_relative_html_test.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
image_resolver_test.go feat: Add pending tasks and active status to WikiStats 2026-04-22 21:17:31 +08:00
json_converter.go feat(docparser): enhance sortedKeys function for numeric string sorting 2026-03-25 22:08:29 +08:00
json_converter_test.go feat(i18n): add JSON file type support across multiple locales 2026-03-25 22:08:29 +08:00
mineru_cloud_converter.go feat(weknoracloud): implement credential management and status checks 2026-04-14 17:36:01 +08:00
mineru_converter.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
mineru_converter_test.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
resolve_remote_images_test.go fix: improve tenant access token logging and update test cases 2026-04-22 21:17:30 +08:00
weknoracloud_http_reader.go feat(weknoracloud): implement credential management and status checks 2026-04-14 17:36:01 +08:00