WeKnora/docreader/tests
Captain 1dd3fe867e
fix(docreader): drop data validation ranges openpyxl cannot parse instead of failing the sheet (#3599) (#3608)
* fix(docreader): drop data validation ranges openpyxl cannot parse instead of failing the sheet (#3599)

A Feishu sheet/bitable exported as .xlsx could fail to ingest with
"TypeError: expected <class 'openpyxl.worksheet.cell_range.MultiCellRange'>".
openpyxl reads a data validation's `sqref`, and a scenario list's, through a
converter that turns the real range-parse ValueError into that message and
aborts the whole workbook — in load_workbook (used by fill_merged_cells_xlsx)
and in pandas alike.

Measured on openpyxl 3.1.5, the version docreader locks: a whole column
(`C:C`), a whole row (`1:1`), a comma-separated list, `#REF!` and an
out-of-bounds range each raise it. Conditional formatting goes through the
same converter, but openpyxl already catches it there and drops the rule
with a warning, so it is left alone.

strip_unreadable_ranges_xlsx sits next to repair_xlsx_bytes and follows the
same contract: it returns rewritten bytes only when some range would fail
openpyxl's own MultiCellRange constructor — the exact step that raises — and
removes just those elements, logging how many. Every other workbook passes
through byte for byte. Neither element changes a cell's value, so dropping
one costs text extraction nothing, while keeping it cost the document.

* fix(docreader): strip unreadable ranges on the MarkItDown xlsx path too

MarkItDown reads spreadsheets through pandas/openpyxl, so a whole-column
data validation still failed that engine with FileConversionException
after the builtin engine was fixed (#3599). Run the same
strip_unreadable_ranges_xlsx pre-step before handing .xlsx to MarkItDown.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-25 22:29:27 +08:00
..
fixtures fix(docreader): preserve vertically merged table cells (#2657) 2026-08-12 10:59:49 +08:00
test_config.py test(docreader): align pdf JPEG quality default assertion 2026-06-17 17:18:57 +08:00
test_docx_merge.py fix(docreader): preserve vertically merged table cells (#2657) 2026-08-12 10:59:49 +08:00
test_docx_tables.py fix(docreader): keep DOCX table content in parsed text (#2898) 2026-08-31 17:56:17 +08:00
test_endecode.py fix(docreader): stop a non-ASCII base64 payload from failing the whole parse 2026-09-22 19:37:21 +08:00
test_epub_parser.py feat: add EPUB and MHTML document support 2026-06-16 11:25:27 +08:00
test_excel_parser.py fix(docreader): drop data validation ranges openpyxl cannot parse instead of failing the sheet (#3599) (#3608) 2026-09-25 22:29:27 +08:00
test_html_parser.py fix(knowledge): support local HTML uploads 2026-07-29 20:11:17 +08:00
test_markdown_table_util.py fix: stop markdown table regex catastrophic backtracking (#2770) 2026-08-24 18:08:07 +08:00
test_mhtml_parser.py fix(docreader): handle Header object when stripping mhtml header (#2902) 2026-08-31 17:14:58 +08:00
test_opendataloader_parser.py fix(security): harden outbound URL SSRF protection (#2626) 2026-08-10 13:24:14 +08:00
test_parser_concurrency.py fix(docreader): throttle heavy parser concurrency 2026-05-07 17:36:09 +08:00
test_parser_routing.py fix(parser): default ppt/pptx to markitdown when no engine rules (#2882) 2026-08-31 14:58:15 +08:00
test_pdf_embedded_images.py fix(docreader): keep pdfium bitmap alive while copying SMask pixels 2026-08-24 14:41:54 +08:00
test_pdf_router.py feat(docreader): support forcing PDF scanned-page OCR parsing 2026-06-25 17:30:22 +08:00
test_ppt_convert.py ci: add app frontend and docreader checks 2026-07-30 14:13:15 +08:00
test_sandbox_executor.py fix(docreader): kill timed-out converter process groups (#2990) 2026-09-23 12:02:53 +08:00
test_source_locator.py feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
test_ssrf.py feat(security): add a whitelist-only outbound mode that refuses before DNS (#3393) 2026-09-23 12:02:06 +08:00
test_ssrf_proxy.py fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
test_tempfile_context.py fix: remove temp file when TempFileContext write fails inside __enter__ (#3495) 2026-09-21 14:21:07 +08:00
test_web_parser.py fix(docreader): fail URL scrape instead of indexing error text (#2883) 2026-08-31 14:12:11 +08:00
test_xlsx_styles_repair.py fix(docreader): sanitize non-conforming styles.xml fills before openpyxl loads (#3637) (#3638) 2026-09-25 22:23:57 +08:00
test_xmind_parser.py feat(docreader): add XMind outline parsing (#2713) 2026-08-20 19:49:48 +08:00