* fix(qdrant): 复制索引改用服务端返回的游标,重复游标报错并给分页加跳数上限
CopyIndices 用"上一页最后一个点的 id"当游标(repository.go:987)。Qdrant 的
scroll offset 是**包含**语义 —— proto 里写的是 "Start with this ID",实测下一页
正是从该 id 本身开始返回,所以每翻一页都会把上一页的最后一个点再读一次、再写一份
到目标库。服务端本来就把下一页该用的游标放在 ScrollResponse.next_page_offset
("Use this offset for the next query")里,而客户端只在 ScrollAndOffset() 中返回它,
本函数调用的 Scroll() 恰好把它丢掉了;终止条件也只能退化成"这一页比 64 短"。
在真实 Qdrant 1.19.1 上实测:150 个源 chunk 复制出 152 个目标点,chunk-63 与
chunk-126 各被复制两次(每个页边界一个),totalCopied 同步虚高。源库只要超过一页
(>64 个 chunk)就会命中,不需要服务端有任何异常行为。
本次修改:
- 改用 q.client.ScrollAndOffset,游标直接取服务端返回的 next_page_offset;终止
条件改为"服务端没有下一页"(next_page_offset == nil),不再用短页推断读完。
- 记录用过的游标,重复即返回 "copy indices made no progress at cursor ...",
检测点在整页处理之前,重复的那一页不会再多写一份。
- 分页跳数上限 maxCopyPaginationHops(10000 页);触顶打印 Warn 并返回错误,
而不是把这一轮复制当成成功。
- 修复后同一份数据在真实 Qdrant 上测得 150 个目标点、0 重复。
回归用例用假 gRPC 服务端(拦截 unary RPC,对齐 repository_test.go 的
newInterceptedQdrantClient)复刻真实服务端的游标语义:150 个源 chunk 必须各复制
一次(修复前 152 个点、2 个重复);游标恒定不动的源端必须报 no progress
(修复前 5 秒写入 3127040 个点仍未返回);一直推进但永不结束的源端在第 10000 页
被拦下。
* fix(qdrant): drop the CopyIndices page cap
10000 pages of 64 points capped a copy at 640k points and failed the
whole clone beyond that. The scroll walks every point of the source
knowledge base (a single-knowledge clone included), so a cap derived from
the chunk map would cut off small clones out of large bases too.
next_page_offset only moves forward, so the repeated-cursor check already
stops a walk that cannot finish, and the task context bounds the rest.
---------
Co-authored-by: linus-liu-web <11640009+linus-liu-web@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(data-analysis): skip style-only worksheets before DuckDB loading
* chore: retrigger CI for XLSX sheet fix
* fix(data-analysis): keep all sheets when the worksheet preflight fails
The style-only sheet preflight is best-effort: if it cannot read the
workbook parts, log a warning and load the sheets DuckDB enumerated
instead of failing the whole workbook. Context cancellation still
aborts the load.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
Logger() runs ahead of Auth and of every route handler, and read the entire
request body with io.ReadAll before any per-route http.MaxBytesReader was
installed. Bound the read to the log budget and hand the unread remainder back
to the handler through io.MultiReader.
An exec that collided with the idle sweeper's delete (#3942 mechanism A)
surfaced as a non-zero exit with an empty stderr, which the filesystem op
wrappers classified as InvalidRequest — outside CanReplaceRemoteBinding's
NotFound/Terminal allow-list, so the rebinding self-heal never ran and the
failure reached the user (empty answer, zero steps). The op wrappers now
re-inspect on failure and reclassify as NotFound when the Engine confirms
the container is gone, making the sweeper's 'deleting needs no coordination
with the binding store' premise hold for the mid-flight window too. Only a
confirmed 404 reclassifies; transport errors keep the original kind.
The shared message now carries the exit code — with an empty stderr it was
the one missing clue during the original diagnosis (#3942).
Co-authored-by: vicfei <vicfei@users.noreply.github.com>
* fix(skills): skip vendored dep trees in install verification script enumeration
sortedScriptPaths fed every .py/.js/.sh in the bundle into one shell 'for
f in ...' command. A skill that vendors a scientific venv (pandas/scipy/
matplotlib etc.) carries thousands of .py files under .venv/, and the
generated command exceeds the exec single-argument limit (MAX_ARG_STRLEN,
128KiB): /bin/sh fails with 'argument list too long' (exit 255) before
any check runs, so no dependency-heavy skill can ever install.
Vendored trees are installed dependencies, not scripts the model can
name; their health is already covered by the dependency verification
passes. Skip .venv/ and node_modules/ prefixes in sortedScriptPaths,
which fixes the tree check, the per-language parse passes and the
runtime prerequisite pass at once.
Verified by installing a 4862-file / 105MB quant skill (pandas+numpy+
scipy+statsmodels+matplotlib venv, real interpreter binary instead of
symlinks since zips cannot carry them): install failed before the fix,
reaches ready after it.
* fix(skills): skip vendored trees at any depth, not only at the bundle root
Bundles are author-zipped, so tools/.venv/ or sub/package/node_modules/
are as legal as root-level trees and enumerated just the same. The
filter now matches vendored trees under any prefix; nested placements
are covered by a regression test.
---------
Co-authored-by: vicfei <vicfei@users.noreply.github.com>
Running `make docs` on main no longer reproduces the committed docs/*:
every definition loses its package-qualified name (types.* instead of
github_com_Tencent_WeKnora_internal_types.*), enums disappear, ErrorCode
becomes a plain integer, and docs/swagger_contract_test.go fails
(TestModelDeleteUsageContract).
Cause: internal/localsandbox is not a dependency of cmd/server (its
importers are behind the `desktop` build tag), but swag's directory walk
reaches it. Its `const X = core.X` aliases make swag load
internal/localsandbox/core again through go/loader, and that second load
re-registers every already-deduplicated type under its short name without
enum values. Excluding the directory removes the second load.
That load was also hiding a second problem: skill_handler.go references
errors.AppError in @Failure without importing internal/errors, so with the
exclude swag stops with "cannot find type definition: errors.AppError".
A blank import lets swag resolve it.
Co-authored-by: Victor Maznyuk <victor@maznyuk.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sweep A only selects pending/processing/finalizing, so a row that reaches a
terminal state while its attempt is still open is never revisited: its spans
stay "running" and the document timeline shows a permanent 进行中. Found in
production with four root spans stuck for 11 and 27 days.
- housekeeping: add Sweep D (reapTerminalSpans) closing spans still open on
terminal rows, carrying the row own error_message so the timeline still
names the real cause. A self-healing backstop for every failure path that
forgets to close its attempt, including rows stranded by older releases.
- span tracker: OpenAttempt now cancels spans left open by earlier attempts
(ATTEMPT_SUPERSEDED). A reparse used to open its new root alongside the
previous attempt dead root, which is what made historical runs look
like they had been reactivated.
Co-authored-by: asmh1989 <minhua.sun@velavigo.com>
processChunks marked the knowledge failed and returned nil when BatchIndex
failed, so ProcessDocument acked the task and the configured document-task
retries never ran. A short embedding or vector-store outage left the row
permanently failed; unchanged data-source syncs did not reprocess it.
While attempts remain, still clean up the partial chunks and vectors but
keep the row processing and return the error to the queue. The final
attempt (or a caller without a retry loop) records the failure as before.
* fix(im/wecom): preserve image in mixed messages with text captions
convertMixedMessage discarded the image whenever a mixed (text+image)
message contained any text, so the most common shape — a question plus
a screenshot — reached the QA pipeline as text-only and the model never
saw the image, despite the IM integration docs promising images are
handled as QA attachments.
Keep the first image as the message attachment and carry the text as
its caption: the IM service already supports captioned images
(prepareIMAttachments downloads the file while fileMessageQAContent
preserves msg.Content as the query), so the adapter just has to emit
MessageTypeImage with Content set instead of MessageTypeText.
Text-only, image-only, and empty mixed messages keep their previous
behavior. Multiple images still collapse to the first one; supporting
several attachments requires a multi-attachment IncomingMessage and is
left out of this minimal fix.
* fix(im/wecom): wrap over-length line in mixed_test to satisfy lll lint
CallTool kept only text and image content from a tool result. A tool
can also return audio, a link to a resource, or a resource embedded in
the result, and GitHub's MCP server returns files that way:
get_file_contents answers with a short message and the file as an
embedded resource. The agent only ever saw the message.
Convert audio, embedded resources (text or blob) and resource links,
recording the resource URI. The agent now follows the reference to a
text resource with its text, and shows links and binary resources as
references. Base64 data of any kind, not only images, is redacted from
the logged content items.
On a session-based protocol version, a server answers a request for a
session it no longer knows, after a restart or when the session expired,
with 404, and the spec has the client start a new session. mcp-go
reports it as transport.ErrSessionTerminated. checkErrorAndDisconnectIfNeeded
only looked for two error messages some servers send, so the client
stayed connected and GetOrCreateClient kept handing out the dead
session: every call to the service failed until WeKnora restarted.
Disconnect on ErrSessionTerminated too, so the next use reconnects.
StartAuthorization stored only the client ID from dynamic client
registration. When the authorization server registers a confidential
client and returns a client secret, the secret was lost: the callback
rebuilds the OAuth handler from the stored client, so the code exchange,
and every later refresh, went out without it and the server rejected
them as invalid_client.
Store the secret along with the ID. The model already has the column,
encrypts it on save, and both handlers read it back.
A partial sync recorded each doc's new content_updated_at before fetching
its detail. When a detail request failed after client retries, the cursor
still held the new version, so every later incremental sync treated the
doc as unchanged and never re-fetched it (mixed success/failure batches
persist the cursor; only all-failed batches are aborted).
Rewind the cursor entry to the previously synced version (existing docs)
or drop it (new docs) on detail failure so recovery re-syncs the doc.
Deletion detection is unaffected: currentDocs is set at listing time.
Fixes#3690
Co-authored-by: LuckVd <LuckVd@users.noreply.github.com>
The header tracker compares a row's column count with the active table
header's to tell a new table from more rows of the same one.
tableRowColumnCount split rows on every `|`, so a cell holding an escaped
pipe (`\|`, which GFM writers such as anydoc emit for a pipe in a cell)
counted as one more column. The tracker took that row for the start of a
new table and dropped the header, so the chunks after it no longer began
with the column names.
Split rows on unescaped pipes only. A pipe after an odd run of
backslashes is text inside the cell, as in GFM.
newS3Client forced path-style for every non-AWS endpoint, so a COS
endpoint such as https://cos.ap-shanghai.myqcloud.com sent requests to
cos.<region>.myqcloud.com/<bucket>/<key>. COS rejects path-style for
buckets created from 2024-01-01, so STORAGE_TYPE=s3 could not reach new
COS buckets, and force_path_style=false had no way to turn it off.
Treat *.myqcloud.com like AWS and use virtual-hosted style. An explicit
force_path_style still wins, and other S3-compatible endpoints keep
path-style.
Fixes#3134
Signed-off-by: Ankit Jha <ankit.jha@tradomate.one>
The global `ARG TARGETPLATFORM=linux/amd64` in Dockerfile.sandbox overrode
the per-platform value BuildKit injects, so the multi-platform CI build ran
every platform on the amd64 rootfs and only relabeled the arm64 config.
The published arm64 entries of main, latest and main-desktop are byte-for-byte
the amd64 layers. The stage-level `ARG TARGETARCH=amd64` had the same effect,
so the cube "amd64 only" guard could never fire.
- Declare TARGETPLATFORM without a default and fall back to linux/amd64 in
FROM only when it is empty (legacy builder).
- Fail the runtime stage when the rootfs architecture differs from
TARGETARCH.
- cubesandbox-base:2026.16 publishes linux/arm64 as well, and Cube runs on
arm64 bare-metal KVM (PVM stays x86_64-only). Follow TARGETPLATFORM for
cube-base, replace the amd64-only guard with an envd-vs-rootfs check, and
build cube / desktop-cube for linux/amd64,linux/arm64 in CI and
build_images.sh.