WeKnora/internal
niannian 4364e61afa
feat(datasource/yuque): preserve TOC folder hierarchy for synced documents (#3476)
* feat(datasource/yuque): preserve TOC folder hierarchy for synced documents

The Yuque connector emitted a bare "<title>.md" file name, and folder_path
is derived from FileName alone (SplitKnowledgeRelativePath ->
NormalizeKnowledgeFolderPath). With no "/" in the name the derived folder is
always empty, so every synced document landed at the knowledge base root no
matter where the book's TOC places it.

Read the book's TOC once per sync, fold it into a per-document path by walking
parent_uuid upward, and prepend the result to FileName. The GitLab connector
already works this way; this supplies the missing input rather than changing
the FileName -> folder_path machinery.

Three optional settings, all defaulting to the previous behaviour so the
emitted FileName stays byte-for-byte identical unless folder_mode is set to
"toc". A TOC failure degrades to flat names rather than failing the sync, and
the toc_only filter is skipped entirely when no TOC was fetched.

* feat(datasource/yuque): make toc_only an admission filter, drop include_book_title

Review follow-ups for the TOC folder hierarchy change.

toc_only used to run before the document was recorded for deletion detection,
so a document that was present in the source but not admitted came back as
IsDeleted — and with sync_deletions on, the sync service then deleted it from
the knowledge base. That is the wrong semantics for a key named after
filtering, and on Yuque it is unusually damaging: the web UI does not show
documents created through the API that were never attached to the TOC, so a
knowledge base is often the only place such a document can be reached at all.
A filtered document is now recorded as seen but not ingested, so enabling the
filter removes nothing. It is deliberately kept out of the cursor so that
admitting it later — by attaching it to the TOC, or by switching the filter
off — ingests it instead of dismissing it as unchanged.

include_book_title is removed. Leaving the book segment out silently merges
same-named top-level groups from different books, and the "which book"
dimension has no other carrier: the automatic tag granularity is one tag per
data source, not one per book. Under folder_mode=toc the book segment is now
unconditional; a source still carrying the key keeps syncing with the value
ignored.

Also supplies the UI the feature was missing — a Yuque-only folder_mode select
in the sync-strategy step, with newly created sources opting into the TOC
layout and existing ones left untouched — and documents both settings in the
Yuque section of the datasource guide.

* feat(datasource/yuque): expose the TOC admission filter in the datasource editor

UI follow-ups from review.

The folder-layout control now uses the pill-button group that the neighbouring
sync-mode and conflict-strategy settings already use, instead of a select — a
two-way choice was the only thing on that step rendered as a dropdown.

toc_only is exposed beside it, disabled unless the TOC layout is selected: the
connector only fetches the table of contents under folder_mode=toc, so the flag
does nothing in the flat layout, and the control says so. Its label names the
documents it admits as the ones visible in the Yuque table of contents, which
is what separates them from documents created through the API that were never
attached to one.

Drops the include_book_title paragraph from the datasource guide. That key was
never in a released build, so describing it as removed documents a history no
reader has; the tolerance test for unknown settings keys stays, without the
retired-key framing.

* docs(datasource): note Yuque folder-mode sync limitations

---------

Co-authored-by: niannian <10008660+RichardChengshaojin@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
2026-09-25 22:33:48 +08:00
..
agent feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535) 2026-09-25 22:22:58 +08:00
application fix(knowledge): reject invalid JSON uploads before async parse (#3360) 2026-09-25 22:23:27 +08:00
assets feat(assets): add ASR test audio file and embed it in the application 2026-04-02 21:27:27 +08:00
browserskill fix(browser): refine connection and preview UI with extension version 2026-09-23 10:57:59 +08:00
common fix(markdown): sanitize invalid UTF-8 and NUL bytes 2026-08-31 12:24:35 +08:00
config docs: 迁移有效文档至 website-docs 并清理旧文档 (#3580) 2026-09-22 23:25:40 +08:00
container fix(container): propagate host project lookup errors (#3611) 2026-09-25 18:51:54 +08:00
database feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
datasource feat(datasource/yuque): preserve TOC folder hierarchy for synced documents (#3476) 2026-09-25 22:33:48 +08:00
embedpolicy fix(embed): enforce host origin allowlists on framed pages 2026-09-14 18:03:57 +08:00
errors feat(knowledge): flag documents that stop making progress (#3557) 2026-09-22 21:34:08 +08:00
event fix(agent): end the turn when the completion cap truncates an answer (#3449) 2026-09-20 13:17:50 +08:00
filetransport refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
handler fix(knowledge): reject invalid JSON uploads before async parse (#3360) 2026-09-25 22:23:27 +08:00
im fix(im): handle unresolved Feishu resource images (#2761) 2026-09-25 18:48:12 +08:00
infrastructure fix(knowledge): reject invalid JSON uploads before async parse (#3360) 2026-09-25 22:23:27 +08:00
ipclass fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
localsandbox feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
logger fix(logger): keep LLM debug records inside their directory 2026-09-25 18:22:14 +08:00
mcp fix(mcp): close MCP connections on shutdown without blocking exit 2026-09-25 18:22:14 +08:00
mcpserver fix(mcp): return accessible image URLs from existing tools (#3650) 2026-09-24 11:56:25 +08:00
middleware fix(knowledge): stop pipeline errors from stranding rows in processing 2026-09-22 20:33:18 +08:00
modelcontext fix(retrieval): fix recall correctness, score scales, cost and agent tools (#3687) 2026-09-25 10:48:18 +08:00
models fix(models): stop advertising graded thinking levels the wire format drops (#3552) 2026-09-25 22:23:08 +08:00
ratelimit feat(embed): add backend embed channels with auth and rate limiting (#1657) 2026-06-12 23:10:39 +08:00
reranking fix(retrieval): fix recall correctness, score scales, cost and agent tools (#3687) 2026-09-25 10:48:18 +08:00
router feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535) 2026-09-25 22:22:58 +08:00
runtime fix(embed): warn at startup when no signing key is configured 2026-09-24 20:29:17 +08:00
sandbox feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
searchutil feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535) 2026-09-25 22:22:58 +08:00
sourceloc feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
storageallowlist fix(storage): align default provider fallback with STORAGE_ALLOW_LIST 2026-07-15 11:05:00 +08:00
storageurl fix(mcp): return accessible image URLs from existing tools (#3650) 2026-09-24 11:56:25 +08:00
stream feat(session): in-place rewind with workspace checkpoint reset (#3469) 2026-09-20 21:01:00 +08:00
textconv fix: remove GPL converter and bundle third-party license sources 2026-09-08 15:19:10 +08:00
tracing/langfuse fix(milvus): preserve keyword retrieval scores 2026-09-17 11:59:40 +08:00
types feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535) 2026-09-25 22:22:58 +08:00
utils feat(security): add a whitelist-only outbound mode that refuses before DNS (#3393) 2026-09-23 12:02:06 +08:00