WeKnora/docreader/parser
sufan721 3842068d22
fix: stop markdown table regex catastrophic backtracking (#2770)
* fix: stop markdown table regex catastrophic backtracking

The line_pattern in MarkdownTableUtil used overlapping character classes
([\t ]* and [^|\r\n]*) so a `|`-prefixed line missing its trailing `|`
forced the engine to explore an exponential number of backtracking paths,
stalling the single-threaded docreader gRPC event loop (100% CPU, health
check timeout).

Replace whole-text regex substitution with an O(n) line-based pass that
only treats rows beginning and ending with `|` as tables, and preserves
the previous align_pattern semantics via _is_alignment_row.

Fixes #2768

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: keep lone pipe rows and normalize CRLF in table formatter

A single "|" was rewritten into an empty table row and then dropped.
Unify CR/LF before the line scan so exported Markdown tables format
without mixed endings.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-24 18:08:07 +08:00
..
__init__.py fix(knowledge): support local HTML uploads 2026-07-29 20:11:17 +08:00
base_parser.py feat: introduce WeKnora Lite edition with lightweight configuration and deployment 2026-03-02 21:21:49 +08:00
chain_parser.py feat(metadata): enhance document processing to include metadata extraction and handling 2026-04-01 15:49:47 +08:00
concurrency.py fix(docreader): throttle heavy parser concurrency 2026-05-07 17:36:09 +08:00
doc_parser.py fix(doc_parser): enhance DOC to DOCX conversion reliability 2026-06-01 20:50:02 +08:00
docx2_parser.py feat: 新增文档模型类,调整配置与解析逻辑,优化日志及导入 2025-11-18 22:37:01 +08:00
docx_merge.py fix(docreader): preserve vertically merged table cells (#2657) 2026-08-12 10:59:49 +08:00
docx_parser.py fix(docreader): remove default 100-page limit for DOCX parsing 2026-04-28 21:50:15 +08:00
epub_parser.py feat: add EPUB and MHTML document support 2026-06-16 11:25:27 +08:00
excel_convert.py feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements 2026-06-03 12:29:13 +08:00
excel_parser.py fix(docreader): align XLS header override behavior with XLSX 2026-07-28 22:00:36 +08:00
html_parser.py fix(knowledge): support local HTML uploads 2026-07-29 20:11:17 +08:00
image_parser.py feat: introduce WeKnora Lite edition with lightweight configuration and deployment 2026-03-02 21:21:49 +08:00
markdown_parser.py fix: stop markdown table regex catastrophic backtracking (#2770) 2026-08-24 18:08:07 +08:00
markitdown_parser.py fix(docreader): preserve vertically merged table cells (#2657) 2026-08-12 10:59:49 +08:00
mhtml_parser.py fix(knowledge): support local HTML uploads 2026-07-29 20:11:17 +08:00
opendataloader_parser.py fix(security): harden outbound URL SSRF protection (#2626) 2026-08-10 13:24:14 +08:00
parser.py fix(docreader): detect legacy doc payloads 2026-07-11 12:21:56 +08:00
pdf_parser.py fix(docreader): keep pdfium bitmap alive while copying SMask pixels 2026-08-24 14:41:54 +08:00
ppt_convert.py feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements 2026-06-03 12:29:13 +08:00
pptx_media.py feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements 2026-06-03 12:29:13 +08:00
registry.py feat(docreader): add XMind outline parsing (#2713) 2026-08-20 19:49:48 +08:00
web_parser.py fix(security): block SSRF via redirects in knowledge URL import 2026-07-02 12:27:48 +08:00
xlsx_merge.py fix(docreader): avoid chart serialization during xlsx preprocessing 2026-08-12 13:26:19 +08:00
xlsx_repair.py feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements 2026-06-03 12:29:13 +08:00
xmind_parser.py feat(docreader): add XMind outline parsing (#2713) 2026-08-20 19:49:48 +08:00