Commit Graph

14 Commits (37837c00a92d976efe2373c74bd3c232bea8a2aa)

Author SHA1 Message Date
Ochan Kwon c9a342d965
fix(docparser): resolve remote images referenced by HTML img tags (#2355)
Remote http(s) images written as <img src="..."> reached no resolver.
Markdown images went through ResolveRemoteImages, HTML data-URIs and HTML
relative paths each had their own pass, but nothing claimed the remaining
combination, so those images were dropped without a log line and the
multimodal stage was skipped for documents that rely on HTML for layout.

ResolveRemoteImages now scans HTML img tags with a quoted src as a second
pass and rewrites only the src attribute value, leaving the tag intact
because the surrounding markup is why these documents use HTML at all.
The per-image work (whitelist, SSRF, download, icon filter, upload) moves
into one helper so the two syntaxes cannot drift apart on policy. Each
pass carries its own image budget; sharing one would let HTML images eat
into the number of Markdown images a document already gets resolved.

An attribute value is trimmed, HTML-unescaped and its scheme lowercased
before use, because HTML permits padding, spells a query separator &amp;
and tolerates any scheme case, while every fetcher downstream compares the
scheme byte-for-byte. Markdown targets get none of that normalization, so
no Markdown reference resolves differently than it does today. A
whitelisted host keeps serving the image from where it already lives, and
its span is rewritten with the normalized URL rather than the raw attribute
text: later stages both locate the image by searching the document for its
serving URL and fetch that same string, so the two have to agree.

The src attribute must now be preceded by whitespace, so a hyphenated
attribute name is no longer mistaken for it. Among HTML tags this cuts two
ways: one carrying both a hyphenated *-src and a real src now resolves the
real src instead of the placeholder, while one carrying only a hyphenated
*-src is no longer resolved at all. Recovering the latter needs an
attribute parser rather than a regex, and resolving a lazy-loading
placeholder while the real image goes unvisited is the worse of the two
failures. Both halves are pinned by tests.

Image-derived text also has to survive into the answer, which is why this
touches internal/searchutil. The chat enricher and the image_info filters
matched Markdown links only, so an HTML-only passage looked image-free:
every entry was dropped and the model received bare markup with the
caption and OCR text stripped, even though the analysis had run. Both now
recognise HTML img tags through one shared regex, so a tag the parser
stores can always be matched back. Both syntaxes are located against the
original passage and spliced in a single pass; scanning one regex over the
other's output let a metadata block that itself quotes an image reference
be rescanned and duplicated. The chat variant still enriches only images
present in the passage, so orphan entries stay out.

Known limits:

  - An unquoted src and a srcset-only tag are not matched, nor is a tag
    whose attribute values hold a bare > or a quoted src= of their own.
  - imgHTMLDataURI still matches src without a boundary, so a data-src
    holding a base64 placeholder is consumed as if it were the src and the
    whole tag replaced. Worth the same fix, but it changes what an existing
    lazy-loading document resolves, so it belongs on its own.
  - A reference shown as a documentation example is rewritten like any
    other. Recognising code spans needs a Markdown parser; a hand-rolled
    scanner risks the far worse failure of treating a document's real
    screenshots as examples.
  - Rewriting a whitelisted src writes the unescaped URL back into an HTML
    attribute, so a query parameter whose name spells an entity renders
    differently than the author wrote it. Keeping the encoded form would
    mean unescaping on every matcher instead.
  - Image text reaches the model only when the whole reference falls inside
    the retrieved window; an img tag is several times longer than a
    Markdown link and straddles a chunk boundary more often.
  - A reference repeated in one document is fetched once per occurrence,
    which is what gives each chunk its own analysed copy.
  - Nothing bounds how long one document's fetches may take in total.
  - Parent-context pruning removes only Markdown images.

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-07-28 21:08:31 +08:00
wizardchen 6a439a4c3b fix(chunks): keep edited retrieval content consistent 2026-07-28 20:23:48 +08:00
wizardchen 951f95903b fix(chat): use Markdown for LLM image context and normalize legacy XML
Chat and agent tool context now emits answer-ready Markdown images instead
of internal <image> XML so copied content renders safely. The frontend also
normalizes legacy XML blocks in persisted conversations during streaming.
2026-07-15 18:10:31 +08:00
wizardchen 2906c6656e feat(backend): improve parent-child chunk merge and imageinfo matching
Scope image info to matched text children during parent resolution and
add range/URL filtering utilities with unit tests.
2026-06-16 01:09:07 +08:00
wizardchen f92302cc64 fix(knowledge): stitch chunks by text overlap, not by position
Reconstructing full document content from chunks relied on position math
(offset = content length - (EndAt - lastEndAt)) that assumes
len([]rune(Content)) == EndAt - StartAt. Two upstream behaviors break that
invariant and caused the merged "全文" view to drop or duplicate text:

- The parent-child chunker prepends a synthetic table header to split
  tables; that header is zero-width in position space (start == end), so
  Content is longer than EndAt - StartAt.
- Chunk Content can retain HTML entities (&#34;, &gt; ...), inflating the
  rune count relative to the source span.

Replace position-based trimming with text-overlap matching: find where the
accumulated text's suffix reappears at the head of the next chunk and join
there. Positions are only used to size the search window. This naturally
skips the prepended header and is immune to entity length drift.

Extract the logic into searchutil.AppendWithOverlap / MergeTextChunks and
reuse it across the three backend stitching sites (wiki_ingest
reconstructContent, graph mergeChunkContents, chat_pipeline
mergeOverlappingChunks). Mirror the same algorithm in the frontend
doc-content merge. Also fix the markdown hr/heading styles (thin solid
divider, primary heading color).
2026-05-30 17:53:47 +08:00
wizardchen c0e4a1d2f1 fix(summary): preserve image caption/OCR text in document summaries
Documents whose only payload is an embedded image (e.g. a docx with a
single picture) intermittently produced the refusal line "No textual
content was extractable from this document." even though the vision
model had successfully extracted a caption.

Three coordinated fixes:

- Clarify the summary prompt that text inside `<image_caption>` and
  `<image_ocr>` is first-class extracted content, not an image
  reference, so the model only triggers the empty-content branch when
  the body is genuinely textless.
- For image-dominated documents (real text < 200 runes after stripping
  image markup) include OCR alongside captions so screenshots and
  scanned figures contribute their actual content; text-heavy
  documents continue to use caption-only enrichment to avoid OCR
  noise from incidental figures.
- Add `EnrichContentCaptionAndOCR` which embeds caption + OCR text
  inline next to the original Markdown image link, deliberately
  omitting the `<image url=...>` and `<image_original>` wrapper
  blocks. Those wrappers carry only opaque export hashes that consume
  tokens and have been observed to retrigger the LLM's "image
  reference with no extracted text" heuristic.
2026-05-22 17:25:39 +08:00
wizardchen 4f55994266 feat(prompt): enhance question rewriting and intent classification in prompt templates
- Updated the prompt template to clarify the task of understanding and rewriting user questions, improving the instructions for coreference resolution and ellipsis completion.
- Renamed sections for better clarity, changing "Rewriting Goals" to "Query Understanding."
- Added critical notes for intent classification regarding image and document attachments, ensuring accurate processing based on the presence of attachments.
- Improved examples to reflect the updated intent classification logic, enhancing the overall user experience in query handling.
2026-04-16 23:15:02 +08:00
wizardchen 53e8e17df4 feat(logger): implement LLM debug logging functionality
- Added a new logger for LLM calls, enabling detailed logging of model interactions, including request and response data.
- Introduced configuration options for enabling and specifying the log directory for LLM debug logs.
- Implemented cleanup for old log files to manage disk space effectively.
- Wrapped existing chat implementations to log calls when debug logging is enabled, enhancing traceability for model interactions.
2026-04-16 23:15:02 +08:00
wizardchen a5ca9a2784 feat(web-search): refactor web search provider
- Updated relevant files to include provider registration, implementation, and metadata.
- Enhanced frontend components to support provider management and configuration.
- Added localization for new provider settings and messages.
- Implemented backend repository methods for CRUD operations on web search providers.
2026-03-31 20:22:34 +08:00
wizardchen 47397464ef refactor(chat_pipeline): enhance merging logic and add partial overlap removal
- Updated the merging process in the chat pipeline to include a new step for re-merging overlapping ranges introduced by context expansion.
- Implemented a new function, removePartialOverlaps, to eliminate chunks that are largely contained within higher-scored chunks, improving deduplication accuracy.
- Enhanced normalization and overlap checking utilities to support the new deduplication logic.
2026-03-25 22:08:29 +08:00
wizardchen b333499078 feat: implement parent-child chunking strategy for enhanced context management
- Added support for parent-child chunking in the knowledge base, allowing for a two-level chunking strategy that improves context retrieval.
- Introduced new configuration options for enabling parent-child chunking, including parent and child chunk sizes.
- Updated relevant components and services to handle parent chunk data, ensuring proper linking and retrieval of child chunks.
- Enhanced user interface elements to reflect the new chunking options and provide better user guidance.

These changes significantly improve the knowledge extraction process by providing richer context through hierarchical chunking.
2026-03-05 18:14:14 +08:00
CodeMarker 5fda6bc266 Optimize Jaccard function for set comparisons 2026-02-05 20:05:55 +08:00
wizardchen 6b4d17ec70 feat: Implement parallel search functionality combining chunk and entity searches, enhancing retrieval efficiency and result accuracy 2025-12-03 17:45:21 +08:00
wizardchen 85b1c4bc49 refactor: Introduce searchutil package for web search result conversion and keyword score normalization
- Migrate web search result conversion logic to a new searchutil package to avoid code duplication and improve maintainability.
- Implement robust keyword score normalization using percentile bounds in the searchutil package, enhancing score handling across different contexts.
- Update relevant functions in the application to utilize the new searchutil methods, streamlining the codebase.
2025-12-01 18:27:12 +08:00