* fix(assistant): polish interaction cards and stabilize composer context
Localize operation labels across all six UI languages and expose answer copying in the assistant panel. Refine question and page-link cards with a single open/return action, and keep the current-page context row at a stable height.
* feat(assistant): add common data source, chunk and knowledge base operations
* feat(assistant): support parsing configuration and batch FAQ updates
* feat(assistant): add MCP tool directory query and sync operations
* feat(assistant): manage web search providers and data source sync settings
* feat(assistant): add model configuration copy operation
* feat(assistant): add workspace profile members and system information operations
* feat(assistant): add system administrator queries and runtime diagnostics
* fix(frontend): polish assistant and resource card UI
Refine assistant composer layout, context controls and reply actions, including narrow-window alignment. Unify knowledge base, agent and shared-space cards with consistent icons, hover feedback, favorites, menus and compact localized statistics.
* fix(assistant): preserve concurrent settings and exclude upstream error secrets
* ci(build): add manual docker build workflow
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(datasource): deep-adapt Feishu/Lark sync with folder mirror, block parsing and embedded objects
- folder mirror: wiki/drive connectors emit path-qualified FileName so
knowledge lands in KB folders mirroring the Feishu tree; shortcut
subtrees deduped; sanitized directory names; legacy datasources
converge via one-shot resync marker (incremental sync upgrades to
full once, then clears)
- parse mode: replace FEISHU_DOCX_PARSE_MODE env with per-datasource
settings.parse_mode (blocks default, export escape hatch); frontend
selector + 5 locales; zero-migration read-time fallback
- block-level GFM: extend docx block coverage (callout/grid/iframe/
board/mindnote/agenda/synced blocks), rich text elements (links,
inline code, KaTeX equations, mentions), real ordered-list numbering,
code language mapping, merged-cell fill, truncation note
- embedded objects: images (gated off multimodal toggle) and boards
(download_as_image, 403-degraded) stored as knowledge with
weknora-img://<n> markers resolved in the document-processing
pipeline via knowledge FilePath; document attachments whitelisted
(pdf/md/xls(x)/csv/doc(x)/ppt(x)) as standalone knowledge in the
parent folder with bidirectional parent/child metadata and filename
list in the body; videos skipped with placeholder + source link
- golden baseline explicitly re-recorded; docs (.env.example,
docker-compose, docs/*, website-docs) synced
* fix(datasource): address Feishu deep-adaptation smoke-test findings
- folder mapping: drop the space/root-folder name prefix — the selected
sync root maps to the knowledge-base root, subdirectories stay relative
- docx blocks: render view(33) blocks silently as containers (children
only), removing the stray '[飞书块: 视图]' placeholder before file lists
- sheet inline tables: fetch merge ranges via sheets/v3 sheets/query and
fill the top-left value across the merged region (closed-interval
interpretation, flagged for real-environment verification)
- knowledge list: hide embedded image/board rows (metadata
embedded_image=true) from the document list and folder-tree counts;
rows remain for VLM indexing and marker resolution; search untouched
- frontend: hydrate protected images (resource:// etc.) in the document
preview so embedded Feishu images/boards actually render
* fix(datasource): resolve embedded image markers at ingest time
The weknora-img://<n> markers were only resolved during document
processing, so chunks carried real resource:// URLs while the stored
markdown file and source_content kept raw markers — file preview
showed blank images. Resolve them in ingestItem before
CreateKnowledgeFromFile (children are emitted first, so image rows
with FilePath already exist); markers with missing children are kept
for the ProcessDocument fallback to degrade. Stored file,
source_content and chunks now agree. Existing datasources converge on
the next full re-sync (update=delete+recreate).
* feat(datasource): resolve Feishu mention users to real names at ingest
docx mention_user elements only carry an open_id, so they rendered as
a generic '@成员'. Resolve the display name via
GET /contact/v3/users/:id (user_id_type=open_id) while generating the
markdown during FetchDocxWithBlocks — before storage, per user
decision; the frontend performs no lookups. A per-sync closure caches
positive and negative lookups; permission errors and empty names
silently degrade to '@成员' (debug log only, ingest never blocked).
Requires one of the contact read scopes (e.g.
contact:user.base:readonly) on the Feishu app.
* refactor(datasource): polish round — defect fixes and structural cleanup
- attachment placeholder patching now targets the unique
'- weknora-att://<token>' marker line rendered per attachment instead of
matching by display name, so an identical earlier bullet is never
replaced by mistake (regression-tested)
- markdown link targets with spaces or parens are percent-escaped
(iframe / mention_doc / inline links), keeping them parseable
- drive folder-tree walking consolidated into core.WalkDriveTree,
deleting the 174-line copy in drive/paths.go
- parse mode threading made explicit: DocxFetchInput.ParseMode replaces
the private client field (connector entry resolves via
ParseFeishuConfig and passes it through ops)
- maxFeishuDownloadBytes restored to a const; the oversize-degrade test
now declares an over-cap Content-Length and downloadRawBytes rejects
early on the header
* refactor(datasource): align Feishu image handling with the platform image pipeline
Replace the image-as-separate-knowledge-row channel with the platform's
orthodox flow: the connector downloads image/board bytes at fetch time and
embeds them in the parent markdown as base64 data URIs; the document
processing image resolver (image_resolver.ResolveAndStore, the same stage
used for parsed PDFs/Word) stores them on the tenant file service, swaps in
persistent provider:// URLs and produces OCR/caption child chunks when a VLM
is configured.
Removed with the marker pipeline:
- weknora-img:// ingest resolution in ingestItem
- the ProcessDocument Step-1.9 marker hook
- the document-list/folder-tree filter for embedded_image rows
- the feishu-specific runtime parse_mode backfill; legacy data sources are
now stamped by migration 000096 (sqlite 000017) which writes the one-shot
resync_required marker — the generic full-upgrade + clear consumption stays
Attachment (PDF/Word/md) sub-items are unchanged.
* style: gofmt test files
* fix(datasource): resolve Drive sub-selection kind from the root listing
A single-file Drive selection (rootFolderToken:fileToken) was probed with
folder APIs before falling back: folder meta returned 91202 and the folder
list 1061002 on every sync, logging API errors for a working configuration.
listDriveFilesForResource now walks the root subtree first and reads the
selected node's kind off the listing: a file filters straight out of the
already-fetched results (zero error calls), a sub-folder still gets its
dedicated walk, and only a token missing from the listing (partial listing /
permission edge) keeps the legacy probe + fallback.
Also pins sub-folder selection semantics with a dedicated test.
* fix(datasource): address review findings across the Feishu deep-adaptation branch
Four parallel reviewer slices audited the branch; this applies the accepted
findings.
Content fidelity (blocks rendering):
- todo blocks parse style.done — completed items render "- [x]" instead of
always unchecked
- code fences size past the longest backtick run in the content, so a line
of ``` inside code can no longer close the fence early
- ordered-list "auto" numbering only resets on a sibling of the last item;
child blocks rendered flat between items no longer renumber the run
Image pipeline:
- inline base64 images are capped at the document-processing resolver's
10 MB budget; larger images degrade to a placeholder instead of leaving a
permanent raw-base64 blob in stored markdown
- image markers carry a per-document nonce so user-typed look-alike text
can never be patched in place of a real marker
- attachment marker patching anchors on the token's line end, closing the
token-prefix collision
- SupportedImageExt recovers image/svg+xml (board exports), which the
platform resolver already stores natively
Sync correctness:
- clearResyncMarker re-reads the row before writing: a long upgraded sync
no longer risks reverting concurrent user edits with a stale full-row
update
- wiki EditTime keys on max(obj_edit_time, node_edit_time): pure
moves/renames re-file the document on the next incremental sync
- file-typed wiki/drive base names are sanitised; a "/" in a title can no
longer fake an extra folder level
Docs/tests/comments:
- datasources docs rewritten to the v3 image contract (no image rows,
no image_map, no parse_mode backfill claim)
- DataSourceEditorDialog drops settings.parse_mode when backtracking to a
non-Feishu connector type
- stale comments corrected (types.go parse-mode transport, repository
folder-counts, wiki List partial-error branch, test headers); regression
tests added for todo done, fence sizing, ordered numbering, svg sniff
* fix(datasource): repair markdown conversion defects found in live smoke test
Live re-sync of the all-block-type Feishu test document surfaced three
converter defects:
- Code blocks: the closing fence reused the opening fence string, so a
tagged block ended with ```java — CommonMark requires a bare closing
fence, and every renderer left the block unterminated. The closing
fence is now backticks only.
- Feishu heading7-9 emitted 7-9 hash marks; GFM stops at H6, so those
lines rendered as literal '#######' text. They now degrade to bold.
- The docx API percent-encodes link targets (http%3A%2F%2F...); escapeURL
now reverses that encoding before markdown-escaping, so stored links
point at real URLs instead of percent-encoded strings.
Verified against the live document via a throwaway API probe: the
'i|g|k' table cells questioned during review are the document's own
content (the sheets copy says i|j|k; the two docx tables genuinely
contain 'g'), not a converter bug.
* fix(frontend): render GFM task lists in document preview
Task list items (- [x] / - [ ]) rendered as plain bullets: marked emits
<input type="checkbox"> for them, but the preview DOMPurify config's
ALLOWED_TAGS allowlist has no 'input', so the element was stripped whole.
The preview renderer now overrides checkbox to emit a styled
task-checkbox span (class/attr both allowlisted) and the preview styles
draw the box with a check mark, hiding the list bullet via :has().
Same gap exists in the chat markdown surface; left untouched there.
* fix(datasource): crop blank canvas borders from whiteboard exports
The board download_as_image API renders the WHOLE board canvas, not the
content bounding box — editors routinely leave the canvas far larger than
the drawing, so board/mindnote/diagram exports surfaced with large blank
areas below the content. The API has no crop parameter, so the connector
now trims near-uniform blank borders after download (bottom-right pixel
seeds the background color, 14/255 per-channel tolerance absorbs JPEG
noise, 8px margin kept). PNG and JPEG are re-encoded; anything
undecodable, fully blank, or already tight passes through unchanged, and
oversized images (>64MP) skip the scan to bound memory.
* fix(datasource): require corner majority before trimming board exports
The blank-border trim seeded the background color from the bottom-right
pixel alone. A large solid shape reaching that corner would seed the
background with its own fill color, and fully-filled rows could then be
trimmed as blank — cutting real content. The background color is now the
majority of the four corner pixels (>=3 must agree); on a split vote
content reaches the corners and the image passes through untrimmed.
Content visible against the background is untouched either way: a row or
column survives if even one pixel differs from the background by more
than the 14/255 JPEG-noise tolerance.
* fix(datasource): widen board trim margin and lower JPEG quality to 80
The 8px margin read as a tight crop — a 32px white frame now visually
bounds the content region. JPEG re-encode quality drops 90 -> 80 to keep
the re-encoded size closer to the original export.
* feat(datasource): timeline widget Markdown degradation + table column alignment
Two content-fidelity gaps from the live smoke test:
- 文档小组件 (add_ons, block_type 40) carry their full data inline in
record; there is no server-side render to download. The timeline
component (component_type_id blk_6358a421bca0001c...) now degrades to
a Markdown list (- **time · title**:text), honoring contentShow and
omitting empty segments; unknown component types and malformed records
keep the previous placeholder note.
- docx text blocks carry style.align (1 left / 2 center / 3 right,
absent unless the author set it). Native-table rendering now collects
per-column alignment from the first explicit cell and emits GFM colon
separators (:---: / ---:). Embedded sheets/bitable keep the plain
separator — their value APIs expose no alignment.
* feat(datasource): show document titles for 云文档 mention links
The docx renderer emitted mention_doc references as [url](url) because
the payload carries no title. Feishu doc URLs now emit a
[weknora-docname://token](url) placeholder and FetchDocxWithBlocks
resolves the titles in one drive metadata batch call
(POST /drive/v1/metas/batch_query, docx/doc/sheet/bitable/wiki/...),
patching the link text to [文档名](url). Refs whose metadata cannot be
fetched — no permission (970003), deleted/mismatched (970005), API
absent — degrade to the previous URL-as-text form with a warn log.
This is display-only: referenced docs are not pulled as sub-documents
(deferred design lives in the task PRD).
* docs(datasource): document mention rendering enhancement and drive rename limitation
- Add optional mention scopes (contact:user.base:readonly,
drive:drive.metadata:readonly, board:whiteboard:node:read) with
degrade behavior to the Feishu permission tables
- Describe mention_user real-name lookup, mention_doc title backfill
and timeline widget placeholders in the blocks rendering flow
- Document the Drive known limitation: folder renames / file moves do
not bump modified_time, so incremental sync keeps the old KB folder
structure until a full sync; Wiki sources are unaffected
- Clarify the directory-mapping wording in website-docs to distinguish
Wiki (rename-aware) from Drive (rename-blind) sources
* docs(datasource): add blocks vs export tradeoff summary to website-docs
* docs(datasource): document docx block-type coverage for blocks mode
- Add a full block-type support table (block_type 1-52 grouped) to the
datasource dev doc: text/heading/list/task/quote structures, code
fences with language aliases, containers (callout/quote/grid/view),
native tables with merged-cell splitting (top-left value fill) and
column alignment, embedded sheet/bitable converted to in-document
Markdown tables, image/board inline pipeline, attachment whitelist,
iframe links, timeline add-on degradation and the placeholder set
- Summarize the same coverage in the drive user guide, the wiki
quick-start doc and website-docs; fix the stale claim that board
blocks degrade to placeholders (they ride the image pipeline now)
* style: resolve diff-scoped lint findings across feishu packages
- Wrap over-length signatures and calls (lll, max 120)
- Add doc comments to exported Block* payload types (revive)
- Rename unused closure parameters to _ (revive)
- Check w.Write / resp.Body.Close error returns (errcheck)
- Apply De Morgan rewrite in docname token validation (staticcheck)
- gofumpt-format touched files
golangci-lint run --new-from-rev=main ./... now reports 0 issues.
* ci: drop fork-only docker-build workflow
The manual docker-build workflow lives on the fork's main and is not
part of the upstream contribution; it entered the branch through the
fork-main merge.
* fix(migrations): renumber feishu resync migrations to avoid version collision with upstream
Upstream added 000017_dingtalk_stream_only (sqlite) and 000096_dingtalk_stream_only
(versioned) before our merge, colliding with our 000017/000096 feishu resync
migrations. golang-migrate rejects duplicate versions. Renumber to the next free
slots (sqlite 000027, versioned 000108) and update comments + test constant.
* fix(migrations): renumber feishu resync migrations ahead of upstream 000027/000108
Upstream main now also claims sqlite 000027 and versioned 000108
(session_sandbox_config_tenant), colliding with our just-renumbered
resync migrations. Move to 000028 / 000109.
* fix(feishu): address review — deprecated env fallback and upgrade notes
- Keep FEISHU_DOCX_PARSE_MODE as a one-release deprecated fallback when a
data source has no per-source parse_mode (deprecation warning logged);
annotate it in .env.example / docker-compose.yml instead of removing.
- Point stale migration-number comments at the final numbers
(versioned 000110 / sqlite 000030) in datasource_service.go and tests.
- website-docs: add Feishu upgrade notes (env deprecation, default mode
flip to blocks, one-shot full resync with embedding/VLM recompute);
refresh 24-feishu-drive.md parse-mode section to the per-source
setting and new blocks semantics.
* docs(feishu): document necessary and optional connector permissions
Restore the permission matrix lost with the docs/ removal: necessary
scopes (wiki/drive/export/docx) with missing-permission symptoms and
optional enhancement scopes (embedded sheets/bitable, board export,
mention user/doc rendering) with their degrade behavior, in
10-datasource.md's Feishu section; make 24-feishu-drive.md list the
three necessary drive scopes and link to the full table.
* Revert "docs(feishu): document necessary and optional connector permissions"
This reverts commit 3be19cb1b656b3596f2cb6afb2cda8c2a1d12be1.
* docs(feishu): permission matrix by connector, parse mode and necessity
Necessary-scope tables split by connector (feishu/lark wiki vs
feishu_drive/lark_drive) with blocks/export applicability columns and
missing-permission symptoms; optional enhancement scopes with their
degrade behavior; scope names cross-checked against the Feishu open
platform API docs (files/folder list, medias download, export tasks,
metas batch_query). Full matrix lives in 10-datasource.md; the drive
guide mirrors its own table and links back.
* test(feishu): adapt mention_user_test to upstream retryTestServer signature
* feat(feishu): mirror the sync root (wiki space / drive folder) as a top-level KB folder
So a knowledge base linking several feishu sources can tell them apart at a
glance: syncing folder 测试1 yields 测试1/... and syncing wiki space
飞书知识库同步测试 yields 飞书知识库同步测试/.... Root-name resolution
failure falls back to the previous root-less behaviour without failing the
sync; existing sources converge through the one-shot resync marker.
* refactor(feishu): cache wiki space folder prefix per run, refresh stale comments
Review follow-ups for d012e593f: cache the sanitized space prefix per spaceID
for the whole sync run (K selected spaces now cost one ListWikiSpaces paging
pass instead of K), fix the misleading warn wording, and realign the
WalkDriveTree / driveFolderName doc comments with root-mirror semantics.
* docs(feishu): drop stale and duplicated comments from the deep-adaptation
- board.go: trimBoardImage no longer claims a bottom-right seed (the code
votes the background over the four corners)
- shared.go: rewrite the FetchDocxWithBlocks note (images/boards inline via
base64 now; removes a stale-marker splice artefact)
- markdown.go: drop an internal task-number leftover
- datasource_service.go: collapse the ProcessSync/processSyncStreaming notes
duplicated verbatim from resyncRequired's doc comment, keep the call-site
why (marker cleared only on success)
- datasource_resync_test.go: drop the obsolete 'backfill' wording
* feat(i18n): 添加解析模式及其提示信息到繁體中文語言包
* fix(frontend): use radius token for task checkbox; fix simplified chars in zh-TW parse mode
CI guard fixes: the PR-added task-checkbox style pushed the radius-literal
count to 161 (> baseline 160); the new zh-TW keys missed the locale parity
audit (user's dc6fdcff9) — also normalise the simplified 文档 leftovers.
* fix(feishu): preserve sync scope, nested content, and legacy settings
Cover shortcut selection, ancestor path changes, nested containers and table boards with regression tests. Avoid redundant Drive traversal, repair resync migrations for legacy settings, and preserve inherited parse modes. Bound board image dimensions before decoding.
* docs(feishu): clarify shortcut scope and incremental path tracking
* Fix Feishu sync reconciliation and embedded media regressions
* Bound Feishu table rendering by actual cell data
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
A blob that a commit emptied is still a blob, and ingestion reads an item with
neither content nor URL as nothing to sync: applyFetchedItem counts it as
skipped and the sync reports success. The body indexed from an earlier commit
stays fully searchable, and no later sync revisits it — a full sync still sees
the path in the tree, and an incremental sync only walks the diffs of commits
newer than the recorded head, which the emptying commit already moved past.
Emit the file's path as the content in that case so ingestion replaces the stale
body, matching what the Notion (b9be7d0d), Confluence and IMA (b71b31c8)
connectors already do. The check is on length alone: a blob holding only
whitespace still carries content the repository has.
* fix(feishu): 修复同步请求限流与共享重试等待
* fix(feishu): 完善知识库限流冷却和导出轮询
Bound export polling by poll count so shared pacing and cooldowns retain the
same export ticket without causing false 60-second timeouts. Preserve the
feishu_timeout failure classification when the poll budget is exhausted.
Cap cooldowns, isolate reactive waits by API path, and use a configurable
10-second default when no valid retry hint is available. Refresh expired
tokens before rejoining the request gate to preserve pacing and new cooldowns.
Skip JSON decoding when the legacy rate-limit code is absent, derive the
prefilter from the shared protocol constant, and update configuration docs.
Isolate shared gates in tests and cover cooldowns, cancellation, API grouping,
concurrent token refresh, and timeout classification.
Validation: Feishu core tests pass; diff-scoped Feishu lint reports 0 issues;
gofmt and git diff --check pass. No live Feishu sync was run for this commit.
* fix(feishu): grant request gate slots in arrival order and improve rate-limit logs
- Reserve the next slot before sleeping so concurrent callers are paced in
FIFO order instead of racing after a shared wake-up.
- Restore the error log for non-rate-limited HTTP 400 download failures.
- Include the API gate name in rate-limit warnings.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(datasource/dingtalk): tighten Validate sampling edges
Keep probing later documents in the same workspace, hold listing budget
for sibling folders that are already queued, fail closed when any root
page cannot be listed, and say when the probe cap stops a conclusive walk.
Fixes#4021
Co-authored-by: fight for you <wangrui19971019@users.noreply.github.com>
* fix(datasource/dingtalk): stop Validate walk once probe cap is spent
- Break the workspace walk as soon as maxValidateProbes is spent, so a
later unlistable folder can no longer turn failed probes into acceptance.
- Treat a skipped folder tail as covered only by a probe made after the
skip, not by any earlier probe in the same workspace.
- Keep the document name on probe context errors.
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: fight for you <wangrui19971019@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(datasource): omit last_sync_cursor from the data source list response
The settings page polls GET /datasource every few seconds during a sync,
and connector cursors can grow to megabytes. No list consumer reads the
cursor, so NewDataSourceResponses now blanks it; detail, create and
update responses still return it through NewDataSourceResponse.
* feat(datasource): add Seafile connector with fingerprint-based incremental sync
Sync folders and files from one Seafile library per data source. The
connector walks the selected scope one directory level at a time,
downloads files through the link + fileserver flow without credentials,
and reconciles deletions against a per-file oid/mtime/size fingerprint
cursor that is checkpointed every 50 mutations.
- resource ids are "<repo_id>:<path>"; ancestors collapse, one library
- per-file failures become coded error items and keep the path pending
- directory failures and Retry-After > 60s abort without deletions
- forced full runs keep the old cursor as a deletion baseline
- API token never reaches cursors, metadata, logs or error text
- share the supported import extension set through internal/utils
* feat(datasource): add Seafile frontend connector and docs
Wire the Seafile data source into the editor dialog, channel labels
and the five locales, and document how to connect a Seafile 10.0.1
library.
- connector definition with base_url + API token fields and icon
- picker guard: one library per data source, at least one selection
- "library" resource type and icon in the lazy tree
- channel label in the knowledge base filter, document views and chat
- i18n for labels, guide steps and the eight seafile_* sync error codes
- website-docs integration guide, data source and extension point doc
updates, and Seafile in the README data source lists
---------
Co-authored-by: zaqqmm <q-zhang10@qq.com>
A partial sync recorded each doc's new content_updated_at before fetching
its detail. When a detail request failed after client retries, the cursor
still held the new version, so every later incremental sync treated the
doc as unchanged and never re-fetched it (mixed success/failure batches
persist the cursor; only all-failed batches are aborted).
Rewind the cursor entry to the previously synced version (existing docs)
or drop it (new docs) on detail failure so recovery re-syncs the doc.
Deletion detection is unaffected: currentDocs is set at listing time.
Fixes#3690
Co-authored-by: LuckVd <LuckVd@users.noreply.github.com>
* fix(server): signal shutdown raced Fatalf exit before ResourceCleaner ran
The signal goroutine closed the listener before calling server.Shutdown,
so Serve returned the raw accept error ('use of closed network connection')
instead of ErrServerClosed. The invoke error path then hit logger.Fatalf
(os.Exit(1)) in the same millisecond, killing the process before the
shutdown goroutine reached resourceCleaner.Cleanup. Every registered
cleaner — BrowserSkill daemon, sandbox reapers, IMService, AntsPool,
Langfuse — was skipped on every SIGTERM/SIGINT, leaking the shared bsk
daemon process and its temp home on each restart.
Two changes:
- Drop the manual listener.Close(): Shutdown closes all listeners as its
first step, releasing the port just as fast without racing Serve.
- Give resourceCleaner.Cleanup its own fresh timeout context so a slow
connection drain cannot leave Cleanup with an already-expired context,
which made it skip every entry without running any.
* fix(server): keep signal cleanup inside one grace period
Desktop quit still closed the listener before Shutdown and returned
before ResourceCleaner finished, so Fatalf could skip every hook.
Share ShutdownTimeout between drain and cleanup, run BrowserSkill
first, and do not drop later hooks when the deadline expires.
* fix(server): satisfy errcheck and line-length lint
---------
Co-authored-by: vicfei <vicfei@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Reverts the eleven plugin system PRs merged on 2026-09-26 so they can be
reviewed before landing:
* a829d7cc9 agent tools served by plugins (#3733)
* 4f5d475c3 Jira example plugin (#3730)
* 01222aac8 dynamic form options, OAuth connections and resource limits (#3729)
* cf502ba46 event delivery and webhooks (#3728)
* 996696057 sandboxed plugin pages (#3727)
* da4ef68d9 standalone plugin hosts (#3725)
* 728007ada remote runtime, Python SDK (#3724)
* ff49f9491 Host API and document parsers (#3721)
* 4328ab26a code plugins (#3720)
* ed63e35cc declarative plugins (#3716)
* c91a8d11f plugin system foundation (#3711)
The tree is back to 47773b5bd, migrations included: no environment ran
the plugin migrations (versioned 000115-000119, sqlite 000034-000038).
* feat(pluginsdk): extension protocol v1, Go SDK and conformance suite
* feat(plugin): run code plugins in an embedded plugin host
* feat(plugin): remote adapters for web search and data sources
* fix(plugin): show plugin errors by their message
* feat(frontend): plugin connectors and web search providers in the UI
* docs(pluginsdk): how to write, package, test and install a plugin
* fix(plugin): gofmt two files and read openapi.yaml line-ending agnostic
* fix(plugin): mark a host plugin ready before Activate returns
* feat(plugin): add plugin manifest, registry and builtin catalog
* feat(plugin): add config schemas with unified secret handling
* feat(frontend): render config schemas with SchemaForm
* feat(datasource): serve connector credential forms from the backend
* feat(im): describe IM platforms and credentials from the backend
* feat(frontend): render model vendor extra fields from the config schema
* feat(frontend): build the settings modal from an extension registry
* refactor(agent): register builtin tools and their chat views
* feat(plugin): let workspaces enable and disable plugins
* feat(plugin): add the runtime driver interface and gather builtins
* feat(datasource/yuque): preserve TOC folder hierarchy for synced documents
The Yuque connector emitted a bare "<title>.md" file name, and folder_path
is derived from FileName alone (SplitKnowledgeRelativePath ->
NormalizeKnowledgeFolderPath). With no "/" in the name the derived folder is
always empty, so every synced document landed at the knowledge base root no
matter where the book's TOC places it.
Read the book's TOC once per sync, fold it into a per-document path by walking
parent_uuid upward, and prepend the result to FileName. The GitLab connector
already works this way; this supplies the missing input rather than changing
the FileName -> folder_path machinery.
Three optional settings, all defaulting to the previous behaviour so the
emitted FileName stays byte-for-byte identical unless folder_mode is set to
"toc". A TOC failure degrades to flat names rather than failing the sync, and
the toc_only filter is skipped entirely when no TOC was fetched.
* feat(datasource/yuque): make toc_only an admission filter, drop include_book_title
Review follow-ups for the TOC folder hierarchy change.
toc_only used to run before the document was recorded for deletion detection,
so a document that was present in the source but not admitted came back as
IsDeleted — and with sync_deletions on, the sync service then deleted it from
the knowledge base. That is the wrong semantics for a key named after
filtering, and on Yuque it is unusually damaging: the web UI does not show
documents created through the API that were never attached to the TOC, so a
knowledge base is often the only place such a document can be reached at all.
A filtered document is now recorded as seen but not ingested, so enabling the
filter removes nothing. It is deliberately kept out of the cursor so that
admitting it later — by attaching it to the TOC, or by switching the filter
off — ingests it instead of dismissing it as unchanged.
include_book_title is removed. Leaving the book segment out silently merges
same-named top-level groups from different books, and the "which book"
dimension has no other carrier: the automatic tag granularity is one tag per
data source, not one per book. Under folder_mode=toc the book segment is now
unconditional; a source still carrying the key keeps syncing with the value
ignored.
Also supplies the UI the feature was missing — a Yuque-only folder_mode select
in the sync-strategy step, with newly created sources opting into the TOC
layout and existing ones left untouched — and documents both settings in the
Yuque section of the datasource guide.
* feat(datasource/yuque): expose the TOC admission filter in the datasource editor
UI follow-ups from review.
The folder-layout control now uses the pill-button group that the neighbouring
sync-mode and conflict-strategy settings already use, instead of a select — a
two-way choice was the only thing on that step rendered as a dropdown.
toc_only is exposed beside it, disabled unless the TOC layout is selected: the
connector only fetches the table of contents under folder_mode=toc, so the flag
does nothing in the flat layout, and the control says so. Its label names the
documents it admits as the ones visible in the Yuque table of contents, which
is what separates them from documents created through the API that were never
attached to one.
Drops the include_book_title paragraph from the datasource guide. That key was
never in a released build, so describing it as removed documents a history no
reader has; the tolerance test for unknown settings keys stays, without the
retired-key framing.
* docs(datasource): note Yuque folder-mode sync limitations
---------
Co-authored-by: niannian <10008660+RichardChengshaojin@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* feat(confluence): support page subtree selection
* feat(confluence): add private image asset resolver
* fix(confluence): preserve pages during partial syncs
* fix(confluence): cap total inlined image bytes per page
Up to 30 images of 10 MB each could be inlined as base64, and every
downstream conversion copies the page again, so one extreme page could
peak above 1 GB. Cap the data URIs a page holds, in the download cache
and the rewritten HTML, at 50 MB; images past the budget keep their
original src like any other skipped image.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* fix(datasource): preserve schedule state on invalid updates
* docs(datasource): document PUT /datasource/:id as a full replace
The update now always persists sync_schedule, so a partial body clears
the schedule. Show a full-object example and state that omitted fields
are reset rather than preserved.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
On a Server/DC instance with the #3596 behaviour every listing ends
early, so skipping deletion reconciliation for such runs meant pages
deleted in Confluence were never removed from the knowledge base.
When a listing is cut short, look up each previously synced page it did
not return with GET /rest/api/content/{id}?expand=space and delete it
only when Confluence confirms it is gone from the space: a 404, a
non-current status, or a different space. Pages that are still there,
that fail to load (including 403), or that exceed the 200-probe budget
per run stay tracked and are checked again next time.
Some Server/Data Center builds keep returning _links.next from
/rest/api/space/{key}/content/page after the last page, with empty
results (#3596). The connector followed it until the 10000-hop cap, so a
sync ran for hours, imported nothing and then failed.
Stop after three consecutive empty pages that still link onward. A
single empty page is not taken as the end, since permission filtering
can empty one window mid-listing. A listing cut short this way is
reported as incomplete: its pages are imported, pages it did not reach
stay in the cursor, and deletion reconciliation is skipped for that run
so an early stop cannot mirror-delete documents.
* feat(datasource): add Confluence sync
* fix(confluence): resume full sync and list Server pages without CQL
CQL search silently truncates large spaces and can delete pages that
dropped out of the index. Full-sync checkpoints now keep a deletion
baseline so Asynq retries continue instead of restarting.
* fix(confluence): satisfy package lint for pre-push against main
* fix(confluence): parse Server page lists and keep version expand
The default Server/DC edition unmarshaled /content/page as a nested
envelope, so sync imported zero pages. Pagination next links also drop
expand=version,space, which made later pages versionless and hid edits.
* fix(confluence): classify errors and fail closed on empty listings
Raw Confluence error bodies were copied into sync-log UI, and pagination
next links could leave the configured context path via `..`. Empty
listings against a non-empty baseline now abort instead of deleting.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>