WeKnora/internal/infrastructure/docparser
wizardchen e22a8367ef feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU
MinerU 4.0 removed the synchronous /file_parse endpoint that the mineru
engine relied on, so a self-hosted 4.0 server answered every parse with
404. The reader now probes GET /v1/health before each parse: a 200 selects
the V1 flow, 404/405 keeps the legacy /file_parse path (AMD and domestic
accelerators must stay on mineru<4), and a 503 error envelope surfaces the
server's model-preload failure instead of falling back.

The V1 flow uploads via /v1/uploads (sha256 dedup, API key only sent to
same-origin upload URLs), creates a zip-only parse job, polls with
exponential backoff (2s to 30s), cancels the job on timeout or caller
cancellation, and reads markdown.md plus images/ from the zip artifact.
The zip extraction is shared with the MinerU Cloud reader and now also
resolves <img src> references and percent-encoded paths.

New tenant settings: mineru_server_api_key (redacted in responses,
preserved on update) and mineru_tier (flash/basic/standard/advanced, empty
for the server default). mineru_parse_method maps to V1 ocr_mode. Backend,
vLLM URL, formula/table toggles and language only apply to MinerU 3.x and
are grouped as legacy options in the settings page. The connection check
also verifies the API key on V1 servers, since /v1/health is
unauthenticated.
2026-09-24 20:40:52 +08:00
..
anydoc fix(docker): 默认在 app 镜像中链接 anydoc 解析引擎 2026-08-16 13:31:51 +08:00
anydoc_reader.go feat(docparser): 引入 anydoc 引擎,在 Go 进程内解析 office 文档 2026-08-16 11:37:27 +08:00
anydoc_reader_test.go feat(docparser): 引入 anydoc 引擎,在 Go 进程内解析 office 文档 2026-08-16 11:37:27 +08:00
builtin_converter.go fix(docparser): preserve standalone image uploads from icon filter 2026-04-29 11:58:02 +08:00
doc.go feat(docparser): 引入 anydoc 引擎,在 Go 进程内解析 office 文档 2026-08-16 11:37:27 +08:00
engine_registry.go feat(docparser): 引入 anydoc 引擎,在 Go 进程内解析 office 文档 2026-08-16 11:37:27 +08:00
engine_registry_reader_test.go fix(parser): default ppt/pptx to markitdown when no engine rules (#2882) 2026-08-31 14:58:15 +08:00
engine_registry_test.go fix(parser): keep PDFs on builtin instead of defaulting to anydoc 2026-09-21 16:28:02 +08:00
engines.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
grpc_parser.go refactor(parser): reorganize Markdown parser and enhance gRPC document reading 2026-06-03 12:29:13 +08:00
helpers.go feat(weknoracloud): implement credential management and status checks 2026-04-14 17:36:01 +08:00
html_table_normalizer.go fix(chunking): 修复内联 HTML 表格与保护模式导致的超长 chunk (#3404) 2026-09-18 21:10:17 +08:00
html_table_normalizer_test.go fix(chunking): 修复内联 HTML 表格与保护模式导致的超长 chunk (#3404) 2026-09-18 21:10:17 +08:00
http_parser.go fix(security): harden outbound URL SSRF protection (#2626) 2026-08-10 13:24:14 +08:00
image_resolver.go fix(docparser): resolve remote images referenced by HTML img tags (#2355) 2026-07-28 21:08:31 +08:00
image_resolver_relative_html_test.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
image_resolver_test.go fix: preserve MHTML image resolution and Markdown structure (#1743) 2026-06-22 15:53:54 +08:00
json_converter.go refactor: remove unused private Go helpers 2026-09-17 15:51:51 +08:00
json_converter_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
markdown_image_scanner.go fix(graph): feed OCR chunks to graph extraction instead of image-link placeholders (#3092) 2026-09-08 19:23:13 +08:00
markdown_image_scanner_test.go fix(graph): feed OCR chunks to graph extraction instead of image-link placeholders (#3092) 2026-09-08 19:23:13 +08:00
mineru_cloud_converter.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
mineru_converter.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
mineru_converter_ssrf_test.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
mineru_converter_test.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
mineru_v1_client.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
mineru_v1_client_test.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
paddleocr_vl_cloud_converter.go fix(chunking): 修复内联 HTML 表格与保护模式导致的超长 chunk (#3404) 2026-09-18 21:10:17 +08:00
paddleocr_vl_converter.go fix(docparser): make PaddleOCR-VL request timeout configurable (#3537) 2026-09-23 11:58:59 +08:00
paddleocr_vl_converter_test.go fix(docparser): make PaddleOCR-VL request timeout configurable (#3537) 2026-09-23 11:58:59 +08:00
resolve_remote_html_images_test.go fix(docparser): resolve remote images referenced by HTML img tags (#2355) 2026-07-28 21:08:31 +08:00
resolve_remote_images_test.go fix: deep-copy stored files and images when cloning a knowledge base 2026-06-03 14:45:59 +08:00
weknoracloud_http_reader.go fix(security): harden outbound URL SSRF protection (#2626) 2026-08-10 13:24:14 +08:00