WeKnora/internal/infrastructure/docparser
ochan.kwon e9980c6011 fix: deep-copy stored files and images when cloning a knowledge base
Cloning a knowledge base previously copied only the storage path strings
(knowledge.FilePath and chunk.ImageInfo.URL), so the source and the clone
shared the same physical objects in the storage backend. Once the original
file and extracted images are deleted on source removal, the clone is left
with dangling references and its document and images become unreadable —
data loss that occurs even for same-store clones.

Add a CopyFile primitive to the FileService interface and implement it in
every backend: server-side CopyObject on the object stores
(s3/obs/cos/oss/tos/ks3/minio), io.Copy on local, and a no-op on dummy.
Destinations use the knowledge-owned layout and reuse the existing
path/object-key guards; a sentinel ErrCrossBackendCopy is returned when the
source scheme does not match the backend.

Use CopyFile to deep-copy the document file in cloneKnowledge and the
extracted images in CloneChunk and cloneFAQKnowledgeBase via a shared
cloneChunkImageInfo helper that deduplicates identical image URLs per clone
and rewrites them to the new objects. Copied objects are cleaned up
best-effort if a clone fails partway through. A clone-time preflight rejects
cloning into a target bound to a different storage backend when the tenant
pins providers via StorageEngineConfig.

Adds unit tests for local CopyFile (independent copy survives source
deletion, traversal rejection, cross-backend rejection), cloneChunkImageInfo
(empty/multi/dedup/parse-failure/OriginalURL handling), and the storage
provider preflight.
2026-06-03 14:45:59 +08:00
..
builtin_converter.go fix(docparser): preserve standalone image uploads from icon filter 2026-04-29 11:58:02 +08:00
engine_registry.go feat(parser): add OpenDataLoader, PaddleOCR-VL engines, and parser improvements 2026-06-03 12:29:13 +08:00
grpc_parser.go refactor(parser): reorganize Markdown parser and enhance gRPC document reading 2026-06-03 12:29:13 +08:00
helpers.go feat(weknoracloud): implement credential management and status checks 2026-04-14 17:36:01 +08:00
http_parser.go feat(weknoracloud): implement credential management and status checks 2026-04-14 17:36:01 +08:00
image_resolver.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
image_resolver_relative_html_test.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
image_resolver_test.go fix: deep-copy stored files and images when cloning a knowledge base 2026-06-03 14:45:59 +08:00
json_converter.go feat(docparser): enhance sortedKeys function for numeric string sorting 2026-03-25 22:08:29 +08:00
json_converter_test.go feat(i18n): add JSON file type support across multiple locales 2026-03-25 22:08:29 +08:00
mineru_cloud_converter.go refactor(parser): reorganize Markdown parser and enhance gRPC document reading 2026-06-03 12:29:13 +08:00
mineru_converter.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
mineru_converter_test.go fix(docparser): address review feedback on PR #1404 2026-05-21 11:42:56 +08:00
paddleocr_vl_cloud_converter.go refactor(parser): reorganize Markdown parser and enhance gRPC document reading 2026-06-03 12:29:13 +08:00
paddleocr_vl_converter.go refactor(parser): reorganize Markdown parser and enhance gRPC document reading 2026-06-03 12:29:13 +08:00
resolve_remote_images_test.go fix: deep-copy stored files and images when cloning a knowledge base 2026-06-03 14:45:59 +08:00
weknoracloud_http_reader.go feat(weknoracloud): implement credential management and status checks 2026-04-14 17:36:01 +08:00