WeKnora/docreader/parser
Captain 1dd3fe867e
fix(docreader): drop data validation ranges openpyxl cannot parse instead of failing the sheet (#3599) (#3608)
* fix(docreader): drop data validation ranges openpyxl cannot parse instead of failing the sheet (#3599)

A Feishu sheet/bitable exported as .xlsx could fail to ingest with
"TypeError: expected <class 'openpyxl.worksheet.cell_range.MultiCellRange'>".
openpyxl reads a data validation's `sqref`, and a scenario list's, through a
converter that turns the real range-parse ValueError into that message and
aborts the whole workbook — in load_workbook (used by fill_merged_cells_xlsx)
and in pandas alike.

Measured on openpyxl 3.1.5, the version docreader locks: a whole column
(`C:C`), a whole row (`1:1`), a comma-separated list, `#REF!` and an
out-of-bounds range each raise it. Conditional formatting goes through the
same converter, but openpyxl already catches it there and drops the rule
with a warning, so it is left alone.

strip_unreadable_ranges_xlsx sits next to repair_xlsx_bytes and follows the
same contract: it returns rewritten bytes only when some range would fail
openpyxl's own MultiCellRange constructor — the exact step that raises — and
removes just those elements, logging how many. Every other workbook passes
through byte for byte. Neither element changes a cell's value, so dropping
one costs text extraction nothing, while keeping it cost the document.

* fix(docreader): strip unreadable ranges on the MarkItDown xlsx path too

MarkItDown reads spreadsheets through pandas/openpyxl, so a whole-column
data validation still failed that engine with FileConversionException
after the builtin engine was fixed (#3599). Run the same
strip_unreadable_ranges_xlsx pre-step before handing .xlsx to MarkItDown.

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-09-25 22:29:27 +08:00
..
__init__.py fix(knowledge): support local HTML uploads 2026-07-29 20:11:17 +08:00
base_parser.py feat: introduce WeKnora Lite edition with lightweight configuration and deployment 2026-03-02 21:21:49 +08:00
chain_parser.py feat(metadata): enhance document processing to include metadata extraction and handling 2026-04-01 15:49:47 +08:00
concurrency.py fix(docreader): throttle heavy parser concurrency 2026-05-07 17:36:09 +08:00
doc_parser.py fix(docreader): kill timed-out converter process groups (#2990) 2026-09-23 12:02:53 +08:00
docx2_parser.py feat: 新增文档模型类,调整配置与解析逻辑,优化日志及导入 2025-11-18 22:37:01 +08:00
docx_merge.py fix(docreader): preserve vertically merged table cells (#2657) 2026-08-12 10:59:49 +08:00
docx_parser.py fix(docreader): keep DOCX table content in parsed text (#2898) 2026-08-31 17:56:17 +08:00
epub_parser.py feat: add EPUB and MHTML document support 2026-06-16 11:25:27 +08:00
excel_convert.py feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements 2026-06-03 12:29:13 +08:00
excel_parser.py fix(docreader): drop data validation ranges openpyxl cannot parse instead of failing the sheet (#3599) (#3608) 2026-09-25 22:29:27 +08:00
html_parser.py fix(knowledge): support local HTML uploads 2026-07-29 20:11:17 +08:00
image_parser.py feat: introduce WeKnora Lite edition with lightweight configuration and deployment 2026-03-02 21:21:49 +08:00
markdown_parser.py fix: stop markdown table regex catastrophic backtracking (#2770) 2026-08-24 18:08:07 +08:00
markitdown_parser.py fix(docreader): drop data validation ranges openpyxl cannot parse instead of failing the sheet (#3599) (#3608) 2026-09-25 22:29:27 +08:00
mhtml_parser.py fix(docreader): handle Header object when stripping mhtml header (#2902) 2026-08-31 17:14:58 +08:00
opendataloader_parser.py fix(security): harden outbound URL SSRF protection (#2626) 2026-08-10 13:24:14 +08:00
parser.py fix(docreader): detect legacy doc payloads 2026-07-11 12:21:56 +08:00
pdf_parser.py feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
ppt_convert.py feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements 2026-06-03 12:29:13 +08:00
pptx_media.py feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements 2026-06-03 12:29:13 +08:00
registry.py fix(parser): default ppt/pptx to markitdown when no engine rules (#2882) 2026-08-31 14:58:15 +08:00
source_locator.py feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
web_parser.py fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
xlsx_merge.py fix(docreader): avoid chart serialization during xlsx preprocessing 2026-08-12 13:26:19 +08:00
xlsx_repair.py fix(docreader): drop data validation ranges openpyxl cannot parse instead of failing the sheet (#3599) (#3608) 2026-09-25 22:29:27 +08:00
xmind_parser.py feat(docreader): add XMind outline parsing (#2713) 2026-08-20 19:49:48 +08:00