WeKnora/internal/infrastructure/docparser
linus-liu-web 1826f46b12
fix(docparser): 云解析超时改为跟随既有配置,不再硬编码 (#3891)
* fix(docparser): 云解析轮询上限接入配置,去掉两处硬编码超时

PaddleOCR-VL Cloud 与 WeKnoraCloud 两个 reader 的整批轮询上限仍是写死的:
paddleocr_vl_cloud_converter.go 的 paddleOCRVLCloudTimeout(600s)直接决定了
pollJob 的 deadline,reader 结构体没有任何 timeout 字段;weknoracloud_http_reader.go
的 pollTimeout 固定 20 分钟。

同族的另外两个 reader 早就可以配置:MinerU Cloud 走
WEKNORA_MINERU_CLOUD_TIMEOUT(mineru_cloud_converter.go),自建 PaddleOCR-VL 走
WEKNORA_PADDLEOCR_VL_TIMEOUT(paddleocr_vl_converter.go)。后果是运维把
WEKNORA_DOCREADER_CALL_TIMEOUT(默认 30m)调大也无效:WeKnoraCloud 的 20 分钟
比它还短,硬编码上限先触发,大体积扫描件固定在第 10/20 分钟失败。

改动:
- PaddleOCRVLCloudReader 增加 timeout / pollInterval 字段(与 MinerUCloudReader
  同形),构造函数用 requestTimeoutFromEnv 读 WEKNORA_PADDLEOCR_VL_CLOUD_TIMEOUT
  (默认仍是 600s),pollJob 用 c.timeout / c.pollInterval;超时文案对齐 MinerU
  的 "task timed out after %s (%d polls)"(旧的 "after %d polls" 是 27b0c84b 已经
  替 MinerU 换掉的写法)。
- WeKnoraCloudSignedDocumentReader 的 pollTimeout 改读
  WEKNORA_WEKNORACLOUD_TIMEOUT(默认仍是 20m)。
- .env.example 补两条说明,位置紧跟 WEKNORA_MINERU_CLOUD_TIMEOUT。

新增用例:两个 reader 的 env→超时值映射(含空/非法/0/负数回退默认)、超时真的会
结束一个厂商持续报 running 的任务(假服务器与假 Transport,带看门狗,回归时 5 秒
内失败而不是轮询 600 秒)。只回退接线(保留字段)时四条用例在 10 秒内全部变红:
PaddleOCR-VL Cloud 的 200ms 配置被忽略、WeKnoraCloud 读回 20m 而非配置值;
把两个源文件整体回退则用例引用的新配置面不存在,包测试无法编译。

* docs(docparser): document PaddleOCR-VL Cloud / WeKnoraCloud poll timeouts

Add the two new env vars to both website-docs timeout tables and pass them
through docker-compose.yml next to WEKNORA_MINERU_CLOUD_TIMEOUT.

* fix(docparser): WeKnoraCloud 轮询上限在调用方已有 deadline 时也生效

入库和临时文档调用 pollTaskResult 时 ctx 都带有 DocReader 调用超时,
原先的 !ok 条件让 WEKNORA_WEKNORACLOUD_TIMEOUT 在生产路径上被跳过。
改为始终叠加 WithTimeout,context 自动取内外较早的 deadline;
回归测试补上父 ctx 带更长 deadline 的场景。

---------

Co-authored-by: linus-liu-web <11640009+linus-liu-web@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-08 14:02:42 +08:00
..
anydoc fix(docparser): anydoc 内嵌 EMF/WMF 图片的扩展名与 Rust 侧对齐 (#3938) 2026-10-08 01:21:57 +08:00
anydoc_reader.go feat(docparser): upgrade AnyDoc to 0.2.4 and parse assets once 2026-09-30 15:25:01 +08:00
anydoc_reader_linked_test.go feat(docparser): upgrade AnyDoc to 0.2.4 and parse assets once 2026-09-30 15:25:01 +08:00
anydoc_reader_test.go feat(docparser): 引入 anydoc 引擎,在 Go 进程内解析 office 文档 2026-08-16 11:37:27 +08:00
anydoc_vector_image_test.go fix(docparser): anydoc 内嵌 EMF/WMF 图片的扩展名与 Rust 侧对齐 (#3938) 2026-10-08 01:21:57 +08:00
builtin_converter.go fix(docparser): preserve standalone image uploads from icon filter 2026-04-29 11:58:02 +08:00
cloud_poll_timeout_test.go fix(docparser): 云解析超时改为跟随既有配置,不再硬编码 (#3891) 2026-10-08 14:02:42 +08:00
doc.go feat(docparser): 引入 anydoc 引擎,在 Go 进程内解析 office 文档 2026-08-16 11:37:27 +08:00
engine_registry.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
engine_registry_reader_test.go fix(parser): default ppt/pptx to markitdown when no engine rules (#2882) 2026-08-31 14:58:15 +08:00
engine_registry_test.go fix(parser): keep PDFs on builtin instead of defaulting to anydoc 2026-09-21 16:28:02 +08:00
engines.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
grpc_parser.go fix(milvus): validate connections and preserve gRPC resolver defaults (#3654) 2026-10-08 00:48:45 +08:00
grpc_parser_resolver_test.go fix(milvus): validate connections and preserve gRPC resolver defaults (#3654) 2026-10-08 00:48:45 +08:00
helpers.go fix(docparser): make the MinerU parse timeouts configurable 2026-09-25 18:38:25 +08:00
html_table_normalizer.go fix(chunking): 修复内联 HTML 表格与保护模式导致的超长 chunk (#3404) 2026-09-18 21:10:17 +08:00
html_table_normalizer_test.go fix(chunking): 修复内联 HTML 表格与保护模式导致的超长 chunk (#3404) 2026-09-18 21:10:17 +08:00
http_parser.go feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
image_resolver.go feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
image_resolver_relative_html_test.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
image_resolver_test.go fix: preserve MHTML image resolution and Markdown structure (#1743) 2026-06-22 15:53:54 +08:00
json_converter.go fix(knowledge): reject invalid JSON uploads before async parse (#3360) 2026-09-25 22:23:27 +08:00
json_converter_test.go fix(knowledge): reject invalid JSON uploads before async parse (#3360) 2026-09-25 22:23:27 +08:00
markdown_image_scanner.go fix(graph): feed OCR chunks to graph extraction instead of image-link placeholders (#3092) 2026-09-08 19:23:13 +08:00
markdown_image_scanner_test.go fix(graph): feed OCR chunks to graph extraction instead of image-link placeholders (#3092) 2026-09-08 19:23:13 +08:00
mineru_cloud_converter.go fix(docparser): make the MinerU parse timeouts configurable 2026-09-25 18:38:25 +08:00
mineru_converter.go fix(docparser): make the MinerU parse timeouts configurable 2026-09-25 18:38:25 +08:00
mineru_converter_ssrf_test.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
mineru_converter_test.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
mineru_layout.go fix(citations): 修复原文定位、高亮范围与 DocReader 启动失败 (#3794) 2026-09-27 20:32:26 +08:00
mineru_layout_precision_test.go fix(citations): 修复原文定位、高亮范围与 DocReader 启动失败 (#3794) 2026-09-27 20:32:26 +08:00
mineru_timeout_test.go fix(docparser): make the MinerU parse timeouts configurable 2026-09-25 18:38:25 +08:00
mineru_v1_client.go feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
mineru_v1_client_test.go feat(docparser): support MinerU 4.0 V1 API for self-hosted MinerU 2026-09-24 20:40:52 +08:00
paddleocr_vl_cloud_converter.go fix(docparser): 云解析超时改为跟随既有配置,不再硬编码 (#3891) 2026-10-08 14:02:42 +08:00
paddleocr_vl_converter.go fix(docparser): make the MinerU parse timeouts configurable 2026-09-25 18:38:25 +08:00
paddleocr_vl_converter_test.go fix(docparser): make PaddleOCR-VL request timeout configurable (#3537) 2026-09-23 11:58:59 +08:00
paddleocr_vl_layout.go feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
resolve_remote_html_images_test.go fix(docparser): resolve remote images referenced by HTML img tags (#2355) 2026-07-28 21:08:31 +08:00
resolve_remote_images_test.go fix: deep-copy stored files and images when cloning a knowledge base 2026-06-03 14:45:59 +08:00
source_blocks.go feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
source_blocks_test.go feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
weknoracloud_http_reader.go fix(docparser): 云解析超时改为跟随既有配置,不再硬编码 (#3891) 2026-10-08 14:02:42 +08:00