Commit Graph

1 Commits (main)

Author SHA1 Message Date
VinhChu ae185dd1a9 fix(docreader): stop a non-ASCII base64 payload from failing the whole parse
encode_image documents errors="ignore" as "return empty bytes on decoding
errors", but it only catches binascii.Error. base64.b64decode raises that for
bad padding or a bad alphabet; for a str that is not pure ASCII it fails in
_bytes_from_decode_data first and raises a plain ValueError, which escapes.

The one caller that passes errors="ignore" is the markdown base64-image
extractor, whose payload group is `([^\)]+)` — everything up to the closing
paren, not the base64 alphabet. A CJK image title therefore lands inside the
payload:

    ![x](data:image/png;base64,iVBORw0KGgo= "图片")
                                            ^^^^ captured as base64

MarkdownImageBase64.parse_into_text does not catch, PipelineParser does not
catch, and BaseParser.parse does not catch, so the ValueError reaches the gRPC
handler and the whole document comes back as
ReadResponse(error="string argument should contain only ASCII characters").
One image title fails the entire upload, and the message names nothing the
user can act on.

The same title written in ASCII decodes fine, because b64decode with
validate=False discards the quotes and spaces. The failure only shows up for
titles in the languages this project is mostly used in.

Catching ValueError covers both cases — binascii.Error subclasses it — so the
undecodable image is skipped and logged, which is what the extractor's own
`if not image_byte` branch already expects to happen.

Not addressed here: the pattern swallowing the title is a separate bug, and
narrowing that group changes which markdown the extractor matches. Filed as
its own change rather than folded in.
2026-09-22 19:37:21 +08:00