AppendWithOverlap deduplicates chunker-prepended table headers by
matching a suffix of acc against a head window of next. When chunks are
strictly contiguous (positionOverlap == 0) there is no overlap to
deduplicate, but the loop still ran. If a sentence at the tail of acc
reappeared as real content inside next's head window (same sentence
written multiple times in the document), the head of next was mistaken
for a prepended header and deleted — silently losing up to hundreds of
bytes from the merged result.
This affects three callers:
- chat_pipeline.mergeOverlappingChunks: RAG search results returned
to the LLM with truncated content
- wiki_ingest.reconstructContent: distillation text fed to the LLM
is truncated, so wiki search returns 0 hits for present content
- frontend doc-content.vue port: the "merged" document view drops
the same segment
Fix: skip text matching when positionOverlap <= 0 and concatenate
verbatim. The chunker's prepended-header case only occurs inside the
overlap region (positionOverlap > 0), which still runs the loop.
Add a regression test covering a compliance-doc scenario where a
logging-requirement sentence recurs in a later Compliance section.