WeKnora/internal/infrastructure
ordinary being 1fe804bd5c
fix(docparser): bound MinerU result packages and their expansion (#4396)
A MinerU result package arrives over the network from a third-party parse API and
was buffered whole before anything about it was validated. The Cloud converter
downloaded it with an unbounded io.ReadAll, even though the V1 client already caps
the same artifact at mineruV1MaxZipBytes - the two implementations of one feature
disagreed about the limit.

Decompression was unbounded on every path. readZipEntry and readZipEntryBytes
called io.ReadAll on a decompressed archive entry, and extractMarkdownZip keeps
every extracted image in memory until the caller hands the slice on, so a package
of many small entries expanded without ever tripping a per-entry check. The
declared UncompressedSize64 is attacker or vendor controlled and may understate
the expansion, so the header is checked and the read is capped as well.

Cap the download at the V1 ceiling, each entry at maxZipEntryBytes, and the
accumulated images at maxZipImagesTotalBytes. The image budget is a parameter on
extractMarkdownZipBounded so a test can exercise it without materialising the
production figure, mirroring the jsonLimit field added in #3985.
2026-10-16 19:37:27 +08:00
..
chunker fix(chunker): respect Markdown fences during heading detection 2026-10-08 21:06:59 +08:00
docparser fix(docparser): bound MinerU result packages and their expansion (#4396) 2026-10-16 19:37:27 +08:00
web_fetch feat(security): add a whitelist-only outbound mode that refuses before DNS (#3393) 2026-09-23 12:02:06 +08:00
web_search fix(web-search): bound provider response bodies instead of buffering them whole (#4370) 2026-10-16 17:16:39 +08:00