HTML and MHTML uploads turn tables into GFM through markdownify, which
writes a pipe in a cell's text as is. That pipe ends the cell, so
<td>a|b</td> became two cells and every later cell in the row moved one
column right. Pipes in table cell text are now escaped before the
conversion, as `\|`.
The escape then has to survive MarkdownTableUtil, which normalizes the
converted tables and split rows on every pipe: `| a \| b | c |` came out
as a three-column row with the escape turned into `\ |`. It now splits
on unescaped pipes only; a pipe after an odd run of backslashes is cell
text, as in the Go chunker since #3979.
markdownify renders an emphasis, code or link element whose text is only
whitespace as "", and the whitespace goes with it. HTML and MHTML
uploads therefore lost word boundaries: `further<strong> </strong>
reference` became `furtherreference`, and `<b>First</b><b> </b><b>Last</b>`
became `**First****Last**`. Editors produce that markup whenever a single
space between two words carries different formatting.
Before conversion, unwrap such elements so their whitespace stays as plain
text. Only elements with no child elements are touched, innermost first,
so a linked image keeps its link and nested wrappers come off completely.
* fix(docreader): handle Header object when stripping mhtml header
* fix(docreader): decode 8-bit MHTML headers without crashing (#2901)
compat32 wraps non-ASCII 8-bit headers as Header; str() turns them into
U+FFFD and HTML Content-Location still crashed on .lower()/urljoin.
Refs: https://github.com/Tencent/WeKnora/issues/2901
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
English: Accept .html and .htm knowledge uploads across frontend, Go, and DocReader. Parse static HTML with charset-aware decoding, preserve document structure and local links, and keep scripts/network execution disabled. Add regression coverage for validation, engine registration, encoding, and parser behavior.
中文:打通知识库本地 .html 与 .htm 文件从前端、Go 服务到 DocReader 的完整上传链路。使用可识别字符编码的静态解析,保留文档结构和本地链接,不执行脚本或发起网络请求,并补充文件校验、引擎注册、编码及解析行为的回归测试。