* feat(plugin): Host API with plugin tokens and a key-value store
* feat(plugin): document parsers from code plugins
* docs(pluginsdk): parsers and the Host API
* feat(pluginsdk): extension protocol v1, Go SDK and conformance suite
* feat(plugin): run code plugins in an embedded plugin host
* feat(plugin): remote adapters for web search and data sources
* fix(plugin): show plugin errors by their message
* feat(frontend): plugin connectors and web search providers in the UI
* docs(pluginsdk): how to write, package, test and install a plugin
* fix(plugin): gofmt two files and read openapi.yaml line-ending agnostic
* fix(plugin): mark a host plugin ready before Activate returns
* feat(plugin): install declarative plugin packages across nodes
* feat(plugin): serve installed plugins' MCP servers in each workspace
* feat(plugin): let installed plugins contribute skills and model vendors
* feat(plugin): register plugin skills into the workspace skill catalog
* feat(frontend): manage installed plugins and their configuration
* test(weaviate): read the class state before the probe barrier
* fix(plugin): keep MCP tool directories reachable behind the plugin wrapper
* test(plugin): wrap a long line
* feat(plugin): add plugin manifest, registry and builtin catalog
* feat(plugin): add config schemas with unified secret handling
* feat(frontend): render config schemas with SchemaForm
* feat(datasource): serve connector credential forms from the backend
* feat(im): describe IM platforms and credentials from the backend
* feat(frontend): render model vendor extra fields from the config schema
* feat(frontend): build the settings modal from an extension registry
* refactor(agent): register builtin tools and their chat views
* feat(plugin): let workspaces enable and disable plugins
* feat(plugin): add the runtime driver interface and gather builtins
useCitationPopover watches its cache-scope getter, so Vue evaluates the
sessionId getter synchronously during setup. botmsg.vue called
useChatCitationPopover above defineProps, hitting the TDZ on `props` and
logging "Cannot access 'props' before initialization" whenever an
existing chat session rendered assistant messages.
Move the call below defineProps and add a test that compiles every SFC
calling a citation popover composable and runs its setup against the
real popover chain, so a caller reading props too early fails in CI.
* feat(datasource/yuque): preserve TOC folder hierarchy for synced documents
The Yuque connector emitted a bare "<title>.md" file name, and folder_path
is derived from FileName alone (SplitKnowledgeRelativePath ->
NormalizeKnowledgeFolderPath). With no "/" in the name the derived folder is
always empty, so every synced document landed at the knowledge base root no
matter where the book's TOC places it.
Read the book's TOC once per sync, fold it into a per-document path by walking
parent_uuid upward, and prepend the result to FileName. The GitLab connector
already works this way; this supplies the missing input rather than changing
the FileName -> folder_path machinery.
Three optional settings, all defaulting to the previous behaviour so the
emitted FileName stays byte-for-byte identical unless folder_mode is set to
"toc". A TOC failure degrades to flat names rather than failing the sync, and
the toc_only filter is skipped entirely when no TOC was fetched.
* feat(datasource/yuque): make toc_only an admission filter, drop include_book_title
Review follow-ups for the TOC folder hierarchy change.
toc_only used to run before the document was recorded for deletion detection,
so a document that was present in the source but not admitted came back as
IsDeleted — and with sync_deletions on, the sync service then deleted it from
the knowledge base. That is the wrong semantics for a key named after
filtering, and on Yuque it is unusually damaging: the web UI does not show
documents created through the API that were never attached to the TOC, so a
knowledge base is often the only place such a document can be reached at all.
A filtered document is now recorded as seen but not ingested, so enabling the
filter removes nothing. It is deliberately kept out of the cursor so that
admitting it later — by attaching it to the TOC, or by switching the filter
off — ingests it instead of dismissing it as unchanged.
include_book_title is removed. Leaving the book segment out silently merges
same-named top-level groups from different books, and the "which book"
dimension has no other carrier: the automatic tag granularity is one tag per
data source, not one per book. Under folder_mode=toc the book segment is now
unconditional; a source still carrying the key keeps syncing with the value
ignored.
Also supplies the UI the feature was missing — a Yuque-only folder_mode select
in the sync-strategy step, with newly created sources opting into the TOC
layout and existing ones left untouched — and documents both settings in the
Yuque section of the datasource guide.
* feat(datasource/yuque): expose the TOC admission filter in the datasource editor
UI follow-ups from review.
The folder-layout control now uses the pill-button group that the neighbouring
sync-mode and conflict-strategy settings already use, instead of a select — a
two-way choice was the only thing on that step rendered as a dropdown.
toc_only is exposed beside it, disabled unless the TOC layout is selected: the
connector only fetches the table of contents under folder_mode=toc, so the flag
does nothing in the flat layout, and the control says so. Its label names the
documents it admits as the ones visible in the Yuque table of contents, which
is what separates them from documents created through the API that were never
attached to one.
Drops the include_book_title paragraph from the datasource guide. That key was
never in a released build, so describing it as removed documents a history no
reader has; the tolerance test for unknown settings keys stays, without the
retired-key framing.
* docs(datasource): note Yuque folder-mode sync limitations
---------
Co-authored-by: niannian <10008660+RichardChengshaojin@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* fix(docreader): drop data validation ranges openpyxl cannot parse instead of failing the sheet (#3599)
A Feishu sheet/bitable exported as .xlsx could fail to ingest with
"TypeError: expected <class 'openpyxl.worksheet.cell_range.MultiCellRange'>".
openpyxl reads a data validation's `sqref`, and a scenario list's, through a
converter that turns the real range-parse ValueError into that message and
aborts the whole workbook — in load_workbook (used by fill_merged_cells_xlsx)
and in pandas alike.
Measured on openpyxl 3.1.5, the version docreader locks: a whole column
(`C:C`), a whole row (`1:1`), a comma-separated list, `#REF!` and an
out-of-bounds range each raise it. Conditional formatting goes through the
same converter, but openpyxl already catches it there and drops the rule
with a warning, so it is left alone.
strip_unreadable_ranges_xlsx sits next to repair_xlsx_bytes and follows the
same contract: it returns rewritten bytes only when some range would fail
openpyxl's own MultiCellRange constructor — the exact step that raises — and
removes just those elements, logging how many. Every other workbook passes
through byte for byte. Neither element changes a cell's value, so dropping
one costs text extraction nothing, while keeping it cost the document.
* fix(docreader): strip unreadable ranges on the MarkItDown xlsx path too
MarkItDown reads spreadsheets through pandas/openpyxl, so a whole-column
data validation still failed that engine with FileConversionException
after the builtin engine was fixed (#3599). Run the same
strip_unreadable_ranges_xlsx pre-step before handing .xlsx to MarkItDown.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(docreader): sanitize non-conforming styles.xml fills before openpyxl loads
openpyxl raises TypeError('expected ... Fill') on <fill> elements with no
patternFill/gradientFill child — cloud-doc exporters, LabView and Acumatica
ERP emit these, upstream treats the workbooks as non-conformant, and both
parse engines died on files Excel opens fine (#3637).
Extend the zip-level repair module with sanitize_xlsx_styles: scan
xl/styles.xml, replace each malformed fill in place with a no-op
patternFill (index-preserving, so cellXf fillId references stay valid),
rewrite the zip only when something needed fixing. The builtin chain
retries the merge-cell prefill once on sanitized bytes; the markitdown
chain sanitizes xlsx/xlsm before MarkItDown sees them. Fixes#3637.
* fix(docreader): log sanitized fill count and never raise from the sanitizer
Fold review feedback into the styles sanitizer: a structured log line on
every repair (count included, doubling as the recurrence meter for #3637,
mirroring the sharedStrings repair log) and a never-raise wrapper so an
internal sanitizer failure passes the original bytes through instead of
trading the caller's error for its own.
* fix(docreader): read the workbook stylesheet by its exact part name
The styles sanitizer picked its part with a suffix match over a set of
zip member names, so a workbook that also carries
xl/richData/richStyles.xml (Excel 365 in-cell images / data types) got
the wrong part on some PYTHONHASHSEED values and skipped the repair.
Read xl/styles.xml, the one stylesheet openpyxl loads.
---------
Co-authored-by: vicfei <vicfei@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* feat(confluence): support page subtree selection
* feat(confluence): add private image asset resolver
* fix(confluence): preserve pages during partial syncs
* fix(confluence): cap total inlined image bytes per page
Up to 30 images of 10 MB each could be inlined as base64, and every
downstream conversion copies the page again, so one extreme page could
peak above 1 GB. Cap the data URIs a page holds, in the download cache
and the rewritten HTML, at 50 MB; images past the budget keep their
original src like any other skipped image.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* fix(client): accumulate multiline SSE data
* fix(client): skip empty SSE data frames and clear lint findings
Accumulating data lines turned a bare "data:" frame into an empty JSON
payload that aborted the stream; the old parser skipped it. Skip frames
whose joined payload is empty. Also use strings.TrimPrefix in
appendSSEDataLine and check Fprint results / drop unused params in the
new streaming tests.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* fix(knowledge): reject invalid JSON uploads before async parse
Validate .json payloads at HTTP upload/replace time with the same gate used
by the simple JSON reader, so JSONC or truncated files fail with HTTP 400
instead of after document-process retries. CreateKnowledgeFromFile itself
skips the gate so datasource sync and IM still create a knowledge row that
fails in async parse.
Signed-off-by: xiaojunxiang <xiaojunxiang@kingsoft.com>
* fix(knowledge): localize JSON upload validation messages
The upload queue shows the backend message verbatim and the rest of the
upload handler speaks Chinese. Map ValidateJSONContent's errors (now
sentinels with unchanged text, still shared with the async JSON reader)
to Chinese messages at the upload/replace boundary.
---------
Signed-off-by: xiaojunxiang <xiaojunxiang@kingsoft.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* fix(datasource): preserve schedule state on invalid updates
* docs(datasource): document PUT /datasource/:id as a full replace
The update now always persists sync_schedule, so a partial body clears
the schedule. Show a full-object example and state that omitted fields
are reset rather than preserved.
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(models): stop advertising graded thinking levels the wire format drops
A generic-provider row (relays, self-hosted runtimes — the default for
uncatalogued models) resolves an empty ThinkingLevelMap, whose missing-key
semantics report every graded rung as supported. Capabilities() then hands
minimal/low/medium/high to the frontend, which renders a selector whose
choice applyThinking silently discards: the chat-template-kwargs branch
sends only the enable_thinking boolean and no reasoning_effort is computed.
Same defect class as the two NVIDIA NIM entries fixed in #3489, but there
is no catalog entry to repair here.
Collapse Capabilities() to the levels the wire can actually express for a
boolean-only switch (off/auto) unless the settings opt into
supports_reasoning_effort. The vendor map itself is untouched, so vendors
that do carry effort keep their ladders.
* fix(models): carry reasoning_effort through the chat-template switch when opted in
The Capabilities() guard kept a vendor's ladder when it opts into
supports_reasoning_effort, but applyThinking's chat-template-kwargs branch
wrote only the enable_thinking boolean and dropped the already-computed
effort — so the opt-in combination still sent byte-identical bodies for
every graded rung (#3551's defect, alive in exactly the shape the review
flagged). Mirror the enable_thinking dialect: write reasoning_effort on
top of the switch when one was computed. A golden case pins the opt-in
combination; the no-opt-in guard case keeps asserting the level is
dropped.
* style(models): wrap long Resolve call in thinking-levels test
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* feat(wiki): add cross-KB wiki search POST /wiki-search
Give external callers a knowledge-search counterpart for wiki pages: one
SQL IN query ranked by title/slug/summary/content, slim hits with
match_snippet, and Viewer retrieve auth including shared KBs.
* fix(wiki): satisfy golangci-lint on cross-KB wiki search
Check c.Error returns, wrap lines over 120 columns, and use wikiDialect
instead of the embedded Dialector selector.
* fix(wiki): address review on cross-KB search helpers
Share KB viewer auth and snippet extraction, keep ID limits in the
service layer, and escape SQLite LIKE wildcards.
* fix(lint): wrap knowledge span fixtures over 120 columns
Go Lint still diffs from the original PR base, so later main tests
in knowledge_spans_test.go were reported as new line-length issues.
* fix(wiki): authorize KB access before checking wiki enablement in cross-KB search
Checking IsWikiEnabled before AuthorizeKBAccess let a caller tell apart a
foreign KB with wiki disabled (400) from one it cannot see (404), leaking
the KB's existence and settings. Run the access check first.
---------
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: wizardchen <wizardchen@tencent.com>
Co-authored-by: lyingbug <xiayula2014@gmail.com>
* refactor(frontend): drop TTL resource caches and remove duplicate startup requests
Space-level resource stores (chatResources / editorResources / organization)
cached lists for 60s. Pages that own the data bypassed the cache with force,
while side consumers (chat input bar, selectors) read stale values and relied
on a web of indirect refresh signals. Replace the TTL with in-flight
deduplication plus explicit invalidation:
- add stores/resourceCache.ts: createCachedResource (one request in flight
per resource, force queues behind it, invalidate/markLoaded discard stale
responses) and createKeyedSnapshotCache for per-id details
- rebuild chatResources / editorResources on those primitives; organization
store drops its loadedAt timestamps and uses the request coordinator only
- isFresh -> isLoaded (only "has a snapshot" is meaningful now)
Fix concrete duplicates found on the way:
- /auth/me was fetched twice on every cold load (main.ts + UserMenu); auth
store now dedupes in-flight calls and exposes ensureAuthMe, router hydrate
reuses refreshFromAuthMe instead of a second copy of the persist logic
- /system/info was fetched by menu.vue directly next to editorResources
- /platform/settings rendered Settings.vue as the route component while the
platform layout already mounts it, so two modals were visible and every
settings panel request ran twice; the route now renders an empty view
- components that called listKnowledgeBases / listAgents / getParserEngines /
getPromptTemplates directly now go through the stores
- Input-field no longer force-refreshes models and web-search providers on
"settings closed" / "left settings route"; WebSearchSettings writes back to
the shared snapshot and useTenantModelReadiness derives from the store
Measured with a mock backend: cold load 19 -> 18 requests (/auth/me once),
settings model panel 3 -> 2 (one modal), route changes now refetch lists
(creatChat 3 -> 9, back to KB list 0 -> 4) instead of serving 60s-old data.
* fix(frontend): fence stale responses in the resource caches and /auth/me refresh
Review follow-ups on the TTL removal:
- createKeyedSnapshotCache tracks a per-key revision; invalidate() and a
force read both advance it, so a response that lands after invalidation
or behind a newer request is handed to its caller but never written back
into the snapshot. Previously a KB detail read still in flight during
save could re-pin the pre-save detail until the next explicit invalidate.
- ensureKnowledgeBases / ensureAgents no longer reject when the shared
resource fetch fails; the own list is already applied and the previous
shared snapshot is kept. Callers migrated onto the store (manual editor,
KB selector, FAQ manager, API integration page) had independent failure
handling before. McpServerIntegrationSettings shows empty lists on
failure instead of the last snapshot.
- /auth/me refresh is sequenced via a new createSequencedRefresh helper:
refreshFromAuthMe() always resolves on a request issued after the call
(queued behind an in-flight one), ensureAuthMe() shares in-flight, and
logout() invalidates so a late response cannot re-apply the old session.
Tests: keyed cache invalidate-during-flight and force-overlap; sequencer
refresh ordering, invalidate after logout, and recovery after a failed
in-flight request.
* fix(frontend): refetch models after publishing a catalog change
invalidate() no longer triggers a refetch and the chat input stopped
reloading models when settings close, so the reasoning-level selector kept
stale capabilities after a catalog publish. Force-refresh the snapshot.
The platform shell always renders <Settings /> and opens it when the path is
/platform/settings, but the settings route also rendered Settings.vue into
the router outlet. Loading that URL mounted two independent modals: both
fetched their data, and a change saved in one left the other stale.
Keep the route for bookmarks, redirects and the close behaviour, but give it
a component that renders nothing.
- 07-agent: the retrieved-image output requirement and
internal/agent/image_requirement.go were removed in #3193; the
Observe step no longer appends that message.
- 15-evaluation: internal/handler/dataset.go never existed; dataset
loading lives in internal/application/service/dataset.go.
- 03-document-parsing: the proto generation script is
docreader/scripts/generate_proto.sh (see docker/Dockerfile.docreader).
* fix(im): handle unresolved Feishu resource images
* fix(im): degrade unresolved Feishu images to plain text
Feishu cards reject non-http(s) markdown links as well as invalid image
keys, so [alt](resource://…) can still fail the whole card update.
Match every ![]() target and keep only http(s) as a fallback link.
* test(im): cover Feishu summary stripping of resource images
---------
Co-authored-by: wizardchen <wizardchen@tencent.com>
The indexer saves embeddings in batches of 40, up to five at a time,
and every BatchSave calls ensureCollection first. On the first write at
a new dimension each of them can find the class missing and try to
create it. Weaviate lets one create win and refuses the rest with 422:
"class name X already exists" when the loser is filtered before the
Raft apply, or "updating schema: TYPE_ADD_CLASS: class already exists"
when it loses inside the apply. ensureCollection returned either as
"failed to create collection", and the errgroup then failed the whole
document although the class was fine. A replica racing this one takes
the same path.
After a 422, look the class up again and carry on if it exists now,
which does not depend on how the refusal is worded. The class is
created with all of its properties in one call, so there is nothing
left to set up. Other failures are returned as before.
This is the Weaviate side of the race #2855 reports for Qdrant.
Self-hosted MinerU capped every parse at a hardcoded 1000s, which fired
before WEKNORA_DOCREADER_CALL_TIMEOUT and WEKNORA_DOCUMENT_PROCESS_TIMEOUT
could matter, so large PDFs always failed at 16m40s (#3091). MinerU Cloud
had the same problem with a hardcoded 600s polling deadline.
- WEKNORA_MINERU_TIMEOUT (default 1000s) bounds a self-hosted parse: the
whole V1 job, or the legacy /file_parse request.
- WEKNORA_MINERU_CLOUD_TIMEOUT (default 600s) bounds how long a MinerU
Cloud batch is polled.
- Both share the env parsing added for WEKNORA_PADDLEOCR_VL_TIMEOUT in
#3537, now in helpers.go: invalid or non-positive values log a warning
and fall back to the default.
MCPManager.Shutdown was never registered with the resource cleaner, so
remote sessions were left to time out. CloseAll also disconnected clients
one by one while holding the lock; it now detaches them, disconnects them
concurrently and gives up after a bound.
The debug file name came straight from the client's X-Request-ID, so a
"../" in it chose where the record was written. Names are now reduced to
a plain identifier alphabet, and the records, which hold full prompts, are
written 0600 in a 0700 directory.
A stream body that ran out without a finish reason (a proxy cutting the
connection) was closed as a clean stop on the Completions, Responses and
Gemini paths, so a half-written answer was stored as complete. It is now
marked incomplete, as the Anthropic path already did, and both the chat
pipeline and the agent treat an incomplete answer as a stream error.
Retry classification checks typed HTTP/transport errors first and matches
status codes as whole numbers, so "max_tokens 5000" no longer reads as a
500. The retry backoff returns as soon as the turn is cancelled.
Parallel tool calls run on errgroup goroutines, and errgroup does not carry
a panic back to Wait, so one panicking tool (MCP client, sandbox, SDK) took
down the whole process. The registry now turns a panic into a failed tool
result, and the parallel path guards its own bookkeeping as a backstop.
Point all four READMEs at the new 1:52 intro video rendered from
misc/intro-video, and update the captions: it has no narration and
uses English on-screen text.
Reparse:
- Open the new attempt only after override validation and cleanup pass.
A rejected reparse used to leave the previous run's subtasks treating
themselves as superseded, so the row stayed in finalizing for good.
- Cancel the previous run's queued tasks when it is still in flight.
ProcessDocument, processChunks, post-process, image tasks, wiki ops and
the dead-letter callback now skip work that belongs to a superseded
attempt.
- Summary tasks write only their own columns. A full-row save after the
LLM call could put finalizing over the new run's processing row.
Errors that were acked with the row still in flight:
- A failed post-process enqueue is retried in place and then returned,
so the task retries instead of leaving the row in processing.
- Transient tenant/knowledge reads and the processing write now retry.
- An image whose orphan check fails on its final attempt is still
counted. Empty manual content fails the row.
Counters:
- FinalizeSubtask runs its decrement and promote in one transaction.
Slot releases retry, and the post-process shortfall release keeps
going past a failed release.
Wiki:
- A synthesis model that cannot be built releases the KB's pending
documents on the final attempt instead of retrying forever.
- Panics in map/reduce workers become per-document failures.
- Housekeeping protects a row held by a wiki op for at most 48h.
Lite mode:
- SyncTaskExecutor honours options set on asynq.NewTask (MaxRetry,
Timeout, Deadline, ProcessIn/At), stops on SkipRetry, and runs the
same final-failure callback as the dead-letter middleware.
- The multimodal fan-in counter is kept in process, so post-process
waits for every image.
Also:
- Housekeeping recovers summaries left pending on finished rows. Cancel
and stall recovery close unfinished summaries.
- Housekeeping sweeps no longer overlap.
- Data source URL ingest writes only the metadata column.
- The document list drops cards the batch endpoint no longer returns.
* feat(chat): open cited documents at the cited passage
Clicking a citation used to show only the parsed chunk. It now opens the
original file in the references panel, jumps to the cited place and
highlights it, narrowed to the sentence the citation supports.
Parsing records where each chunk sits in the original file
(chunks.source_locators): PDF page and paragraph boxes from the docreader,
MinerU and PaddleOCR-VL layouts; Word paragraphs, slides, sheet rows and
EPUB chapters aligned in Go against any parser engine's Markdown; text
offsets for Markdown/TXT; segment times for audio. Pipeline text rewrites
are followed with a line-diff remapper. Scanned PDF pages get their OCR
text as quotes at search time so a sentence can pick its page.
The frontend renders PDFs with pdf.js and every other previewable type
with the existing preview, then highlights with the CSS Custom Highlight
API in a highlighter yellow that stays visible against the brand green.
* fix(docreader): serialize source blocks outside main so tests import it
test_docx_tables stubs docreader.parser in sys.modules, so importing
docreader.main from a later test failed in the full unittest discover run.
The wire conversion now lives in docreader.source_wire, which needs only
the proto.
* fix(chat): tighten source locators from review
- Remap rewritten-line-for-line stretches by line so scanned PDF pages
no longer drift onto the wrong page.
- Keep the source preview mounted for citations in the file already open.
- Wait for in-flight PDF page renders before looking for the text layer.
- Count image rune offsets in one pass; stream xlsx rows up to maxUnits.
- Start text locators after a UTF-8 BOM, as viewers decode without it.
- Only re-query image children for scanned-page results.
- Seek audio citations without autoplay and drop stale seek listeners.
- Fix the SQLite migration's cross-reference to versioned/000113.
* fix(migrations): renumber chunk source locators after main's chunk_images
main took versioned 000113 and sqlite 000032 for chunk_images, so the
source_locators column moves to versioned 000114 and sqlite 000033.
Rename the tab to 画廊 and give the knowledge-base views a proper switch.
文档 / Wiki / 图谱 / 画廊 read as another breadcrumb level; they are now a
segmented control with an icon and a tooltip per view, and the breadcrumb
wraps instead of squeezing its links on narrow widths.
The gallery toolbar now has the documents tab's shape: what is listed on
the left, search, a 筛选 popup with an active-count badge and a 排序 popup
on the right. The pinned rail, the 全显示/自定义 status words, the filter
on/off switch and the apply-to-all row are gone. Per-value verdicts read
不限 / 隐藏 / 始终显示 with a line explaining that 始终显示 only overrides a
hide, which is what AttrRules actually does. The search scope becomes
checkboxes: ticking every field is "all" mode, and the last field cannot
be unticked. The grid drops the extra 16px padding that doubled the page
gutter, and cards match the document cards.
The viewer moves into GalleryViewer.vue as a full-screen lightbox and
fixes what the old one got wrong: keys never fired (the listener sat on
a div that is never focused), prev/next wrapped within the page instead
of crossing into the next one, and "open source document" had a string
but no button. It adds wheel/double-click zoom, drag to pan, 1:1,
rotation, download, open in new tab, copy caption/OCR, a filmstrip,
image size and status, and a collapsible info panel.
* feat(image): observe image attributes and let the policy decide OCR
Describe each image with the VLM as before, but ask it to also observe two
attributes about the image in the same round — how much body text it carries
(contain.text) and whether it presents data visually (contain.data_visual) —
then let a small pure function decide whether that image is worth a second,
separate OCR round.
Per-class OCR was the first shape of this idea; a class (chart / photo / logo)
is only an indirect proxy for 'is there text worth reading here', and the class
table has to grow whenever a new kind of image shows up. The decision actually
needs the observed attributes, so it reads them directly, and the attribute set
is a backend registry row rather than a new class.
The observation and the caption share one VLM call; OCR stays a separate call
(prompt isolation, see #3291). An attribute the model does not answer for is
left absent rather than filled with a conservative default, so the trace never
claims an observation that did not happen, and an incomplete observation falls
back to the on_unobserved policy (OCR still runs). The registry also carries the
panel's display text, so a new attribute needs no frontend change.
Off by default: image_attrs_enabled=false keeps today's behaviour (one caption
and one OCR per image). KB config: image_processing_config.image_attrs_enabled /
image_actions; per-upload override via process_config. Registry:
GET /api/v1/image-attrs/schema.
* feat(gallery): add image gallery tab with schema-driven filters
Adds a 4th knowledge-base tab parallel to Documents/Wiki/Graph that
browses every image asset in a KB.
Backend:
- GET /api/v1/knowledge-bases/:id/images lists image assets projected
from chunks.image_info, with keyword (caption/OCR), multi-value
attribute (attr_<name>=<value>), is_enabled and sort filters applied
in memory, plus pagination.
- Resolves the source knowledge item title (SourceName) via a batched
GetKnowledgeBatch so the viewer can show a human-readable source.
Frontend:
- ImageGallery.vue: thumbnail grid + search/sort + schema-driven filter
panel (from GET /image-attrs/schema) + single-image viewer with
prev/next, attributes/caption/OCR/source.
- Wired as the gallery tab in KnowledgeBase.vue; i18n keys in all 5 locales.
* feat(gallery): drive filters/search/sort from a configurable attribute contract
The gallery UI is no longer hardcoded: it renders from a self-describing
contract served by GET /knowledge-bases/:id/gallery-config (live attribute
sources, every attribute's definition with usage flags merged across the
config tiers, and the caller's search activation state).
- Attribute sources form an open registry (types.RegisterGalleryAttrSource);
a builtin source models caption/ocr_text/timestamps/enabled so the gallery
has zero special cases.
- Usage resolution: user > KB > system > source, field by field; the system
tier lives in system_settings (gallery.policy, JSON, no rebuild), the KB
tier slot is wired in the engine but not stored yet.
- Search activation is the user's own record: mode (all|custom) plus
per-field on/off in users.preferences.gallery; source declarations never
set defaults (unrecorded = off in custom mode).
- image_info is read through a gallery-local struct, removing the compile
dependency on the attribute-pipeline types; the gallery builds and runs
with or without that feature deployed.
- ListImages validates search_in / sort_by / attr_filters against the
contract; ineligible fields are dropped, not served.
* fix(frontend): inline the shared graph-drawer stylesheet (@import instead of <style src>)
plugin-vue 6.0.6 keys unscoped src-style descriptors by the imported file
path, so AgentStreamDisplay.vue and WikiBrowser.vue — both importing
components/css/wiki-graph-drawer.less — overwrite each other during the
build and the second one crashes with "reading 'scoped'". Adding the
gallery changed the module order enough to make the collision hit every
time. Inlining the same stylesheet keeps the emitted CSS identical and
removes the shared descriptor key.
* feat(gallery): render attributes from short labels with descriptions in tooltips
A source could only say one thing about a value, so the filter panel had to put
a full sentence inside a checkbox. Attributes and their values now carry two
pieces of text: a short label for the space-constrained controls (checkbox,
search field, sort option) and a description for wherever there is room
(tooltip, detail panel).
The contract's value list becomes an array of {value, label, description}, and
the display helpers resolve each field separately: the gallery stamps the short
name on a control and moves the sentence into a tooltip. Translation overlays
are looked up per field, so a source that ships no gallery-specific text still
picks up the attribute pipeline's own wording.
* fix(gallery): let locale text win over the contract's wording
The filter panel read the backend's own label before asking the locale,
so a translated value never got a chance: the contract ships its own
default-language wording, and it shadowed the overlay for every locale.
Built keys were also off by a segment — imageAttr.<name> has to be
imageAttr.<name>.label, and the gallery's own keys live under
knowledgeEditor.wikiBrowser.gallery.attr — so builtin and system
attributes fell through to English on a Chinese UI.
* feat(gallery): top bar layout, pinned panels and per-value verdicts
The toolbar moves onto its own row above the grid, so the controls no
longer eat into the space the images are there for: search box, the two
scope switches, sort field and direction, and the result count all sit in
one flex row. The settings behind them are no longer inline either — each
opens as a popover under its control and can be pinned to a right-hand
rail, where the two panels take turns as tabs instead of both claiming
width at once.
Filtering gains a third state per attribute value. Where a value could
only be picked or not picked, it can now be "off" (hide those images),
"on" (show them whatever else says) or neutral (leave them alone), which
is what lets one attribute's rules talk about an image another attribute
tried to hide without the pair of them cancelling each other out: a
forced display outranks a forced hide, and anything no rule speaks about
stays visible.
The switch that used to only exist inside the panel now has a toolbar
home as well. "筛选:全显示" genuinely drops the attribute constraint
instead of imposing it against a panel the user cannot see.
Filtering used to empty the list the moment it was switched on. Any
image whose attribute was never observed — every image ingested before
the attribute pipeline existed — read as a mismatch for every filter,
which is not narrowing a list but losing it. Absence now stays silence:
an attribute no image carries places no constraint on it, until the
pipeline catches up and fills the value in.
Co-authored-by: WorkBuddy
* feat(gallery): expose the image attribute pipeline as a gallery source
The gallery's contract resolved to a single source, so the filter panel
could only offer "Enabled" while the attributes the observe-and-describe
pipeline fills in — contain.text, contain.data_visual — never reached the
UI. The registry is a gallery attribute source like any other: every
attribute it declares is offered to the gallery, filterable by default,
with the config tiers able to override the usage flags at runtime.
The registration lives in the attribute pipeline's own commit; this branch
carried an earlier, squapshot of that work which predated the split of an
attribute value into a short label and a description, so the registry's
value shape and this file have been brought in line with it. Both PRs now
hold byte-identical copies of the source hook, which is what lets either
one land first and the other merge cleanly afterwards.
Co-authored-by: WorkBuddy
* fix(gallery): viewer race, pin icon, panel anchoring and scope wording
Opening the viewer rendered the img element with the previous or empty
source while the proxy fetch was still producing a blob, and the error
that empty source fired locked the viewer in its failure state even
after the real image arrived. The resolved source now lands before the
flag is consulted, and a token keeps a slow fetch of one image from
overwriting a faster one that navigated past it.
The pin button drew nothing: TDesign has no "pushpin" glyph, so the
button rendered as an empty shell. It now uses pin / pin-filled, keeps
its place beside the tab bar, and a pinned rail assigns its columns
explicitly — the panel precedes the main section in the DOM, so leaving
the grid to source order handed the wide column to the panel and
squeezed the images into the rail.
The floating panel no longer appears at a hardcoded offset: it anchors
just below the arrow that summoned it, the arrow flips up while its
panel is out, a second press retracts, and any click elsewhere in the
toolbar closes it. When a rail is pinned the arrows turn to the
requested tab instead of spawning a second surface.
The scope words (全部 / 全显示 / 自定义) become plain status text rather
than a second dropdown — they name the current mode, and only the arrow
beside them opens the panel that changes it.
Co-authored-by: WorkBuddy
* fix(web): align the OCR-on-unobserved switch with the other setting rows
The switch sat bare inside its setting-row while every other switch on
the page is wrapped in setting-control, which is what reserves the right
column and centers the control vertically. With the row laid out from
the top, the bare switch hugged the row's upper edge and crowded the
description text — visibly out of place next to its neighbors. Wrapping
it in setting-control gives it the same right-aligned, centered slot as
the rest.
Co-authored-by: WorkBuddy
* fix(web): move the OCR-on-unobserved row out of the vertical schema row
The row lived inside setting-row-vertical, whose descendant rule stretches
every setting-info and setting-control beneath it to 100% width. Two
full-width flex children side by side cannot fit, so the control overflowed
the container and the switch landed past the right edge — visibly further
out than the switch on the row above. The row and its note now sit beside
the vertical block instead of inside it, under their own condition, so the
switch shares the ordinary row layout and its right edge lines up with the
"image attribute observation" switch above.
Co-authored-by: WorkBuddy
* fix(gallery): applying a verdict now activates the filter scope
Setting an off/on verdict while the panel's master switch was off
silently ignored it: buildParams only attaches attr_rules in custom
scope, and setVerdict never flipped the scope. A verdict the user just
expressed now switches the scope to custom; clearing the last meaningful
verdict returns to all, since custom with no rules constrains nothing.
* fix(gallery): search checkboxes stay visible and idle panels read as dimmed
The search panel hid its field checkboxes while the master switch was
on, so the only way to reach them was to flip the switch first. They now
sit next to the switch like the filter panel's verdicts do: dimmed in
all-field mode (where they carry no weight) yet clickable, and touching
one drops the mode to custom. The filter panel's verdict rows dim the
same way while the filter switch is off.
* fix(gallery): bulk verdict row, neutral default mark, panel polish
The clear-filters button never matched what the panel is for, so it is
now a bulk verdict row: one set of -/off/on buttons that applies the
chosen position to every value of every value-typed attribute, with the
default mark turned into a language-neutral dash. The panel head drops
its second divider (the tab bar already draws one) and the tabs follow
the toolbar's order: search left, filter right.
* fix(gallery): drop the all-fields caption and center the order arrow
The caption above the search checkboxes showed and hid with the master
switch, shoving the checkboxes down on every flip; it said nothing the
dimmed state doesn't, so it is gone. The order button's arrow is an
inline svg, which sits on the text baseline and reads as riding high;
it now centers against the label.
* fix(gallery): center the order arrow for real
vertical-align never applied: the icon is a flex item inside the
button's inline-flex text wrapper, and the wrapper's default stretch
pins the explicitly-sized 16px icon to the top of the 22px line box.
align-items: center on the wrapper is what actually centers it.
* fix(gallery): shorten panel tab labels in en and ru
The two-word labels overflowed the 300px panel and forced the tab bar
into scroll arrows; single-word labels fit in every locale that was
affected.
* docs: drop the image pipeline rewrite from this branch
The rewritten 'Go 侧接手后' paragraph belongs to the image attribute observation
work (#3635), which this branch carries as a prerequisite and which already
documents the caption_ocr / observation_driven split. Keeping a competing
rewrite of the same paragraph on the gallery branch makes it collide with
upstream, which extended that paragraph with the custom_instructions and inline
HTML table notes. Restoring the shared wording keeps the gallery branch free of
this file so upstream's edits merge cleanly.
* style: gofmt the gallery attribute types
Struct field tags in the resolved gallery config were one column narrow, so
gofmt rebroke the alignment block. Whitespace only.
* style(gallery): use design tokens in the gallery styles
The style guard ratchet fails on any new violation. ImageGallery.vue shipped
with 47 TDesign token fallbacks, 21 literal font sizes, 4 literal radii and a
0.15s transition; all of them now read the --app-* / --td-* tokens, which is
what the guard asks for and what dark mode depends on.
* lint(gallery): fix the golangci-lint findings in the gallery code
lll: the three over-long swagger @Param lines and one registry row were
shortened. ineffassign: ctx was assigned before the access check and then
reassigned, so the first value was dead; it is now declared once, after the
check. errcheck: the three unchecked c.Error calls in ListImages. revive: the
AttrSpec / AttrValue doc comment block was attached to the wrong type and split
into one block each. staticcheck S1016: the gallery registration converts the
registry value instead of restating its fields.
* lint(gallery): wrap the image query select and check the last c.Error
* style: reformat the wrapped image select in chunk.go
* fix(gallery): address review findings on ordering, KB switch and blob lifetime
- sortImageAssets: swap operands for desc instead of negating, keeping a
strict weak ordering so ties page stably
- ImageGallery: reload on knowledgeBaseId change, drop stale list
responses, fetch thumbnails in parallel with one cached blob per URL,
revoke blobs on unmount, default sort to builtin:created_at
- KnowledgeBase: non-wiki KBs with a stale wiki/graph tab fall back to
documents again
- handler: move ListImages godoc out of ListKnowledgeFolders' block and
document attr_filters / attr_rules
- DecideOCR: a payload with no OCR conditions uses the built-in policy
instead of silently skipping OCR
* docs(parsing): describe the caption_ocr and observation_driven image pipelines
Carried over from #3635, which this PR supersedes.
* fix(gallery): load local:// and other storage handles through the KB file proxy
The gallery's own scheme check only caught resource://, provider:// and
storage://, so local:// (the default storage), minio://, cos:// and s3://
images went straight into <img src> and never rendered. Resolve through
isProviderFileURL, the scheme list chat and wiki already use, and lock the
routing with a test.
* fix(kb-editor): keep API-customised OCR conditions on save and upload
The KB editor always wrote the registry default into image_actions.ocr.on,
so any unrelated edit replaced a knowledge base's custom OCR conditions
with the default table; the upload dialog did the same for the per-document
override. Keep a stored custom list and fall back to the registry default
only when there is none, and show the conditions actually in effect instead
of the default table in both panels.
* perf(gallery): de-duplicate, filter, sort and page images in SQL
Every gallery request used to read every image-bearing chunk of the KB,
OCR text included, decode it in Go and page in memory. A single query now
expands the image_info arrays, keeps one row per image URL, applies the
keyword / attribute / verdict / enabled filters and the sort, and returns
only the page with the filtered total.
The expansion carries narrow scalars only; filters and JSON sort keys are
evaluated on the winning entry of each image, and the page's JSON is read
back for its rows alone. Postgres runs the query with JIT off, since the
fixed 100-rows-per-call estimate of jsonb_array_elements otherwise prices
it ~100x high and compilation dominates.
On 20k images with 2 KB of OCR text each (40k chunks), a page drops from
~750 ms to ~100-200 ms on postgres and ~200 ms on sqlite; keyword search
over OCR text stays a full scan (~400-600 ms) until the text is indexed.
The filter semantics are pinned against both databases.
* perf(gallery): serve the gallery from a trigger-maintained chunk_images table
Reading images out of chunks.image_info meant parsing every image chunk of
the knowledge base on each request, so a page still cost time in
proportion to the KB. chunk_images (versioned/000113, sqlite/000032) keeps
one row per image_info entry with caption, OCR text and attributes as
columns; triggers on chunks keep it in step with every write path, and the
migration backfills existing chunks.
Copies of one image stay separate rows and the query lists the copy on the
most recently updated chunk through an indexed anti-join, so writes decide
nothing and concurrent writers need no locking. On postgres, caption and
OCR text get pg_trgm indexes when the extension is available.
On 20k images with 2 KB of OCR text each (40k chunks):
postgres first page 10 ms, page 500 41 ms, keyword 2 ms, filters 23 ms
sqlite first page 3 ms, page 500 39 ms, keyword 147 ms, filters 51 ms
down from ~100-600 ms. Writing those 40k image chunks costs 7.0 s instead
of 1.3 s on postgres (1.9 s instead of 1.0 s on sqlite), mostly the trigram
indexes; status and enable-flag updates patch rows in place.
---------
Co-authored-by: tayadwx <tayadwx@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
* fix(retrieval): keep index writes and shared-KB lookups correct
Index writes
- Image OCR/caption chunks were indexed without IsEnabled, so every engine
but Postgres stored them disabled and retrieval never returned them.
The SQLite engine re-enables the affected rows at startup.
- FAQ edits, similar-question additions and import merges re-index under
the old SourceID and relied on the engine overwriting it. Postgres and
SQLite kept the stale row (ON CONFLICT DO NOTHING); ES v8 and Qdrant
mint a new document ID and kept both. FAQ re-indexing now deletes the
old rows first, and Postgres/SQLite BatchSave upserts by
(source_id, source_type).
- Postgres: drop the GORM default tag on is_enabled, which replaced false
with true on insert and indexed disabled chunks as enabled.
- SQLite: BatchSave no longer attaches vectors and FTS rows to the wrong
rows when part of a batch already exists, and vecTables is guarded
against concurrent map writes.
- CopyIndices: Postgres filters to the copied knowledge, pages by id and
keeps is_enabled; SQLite copies every row of a chunk (generated
questions included) with stable source IDs and enabled state.
Shared knowledge bases
- Chat merge: FAQ answer population and neighbor expansion look chunks up
without the caller-tenant filter, like parent resolution (#3342).
- Neighbor expansion never cuts the retrieval hit; the length budget goes
to the text adjacent to it.
- FAQ negative questions are applied whenever an FAQ KB is in scope and
to chunks of shared FAQ KBs.
- Search results get image_info for shared-KB chunks.
- search_knowledge reads FAQ metadata from the search result instead of
one tenant-scoped query per hit, so shared FAQs render their answers.
* fix(retrieval): put every retrieval score on one scale
Scores from different retrieval paths were ranked together while living on
different scales, so ordering depended on which path produced a hit.
Fusion
- RRF ranked a chunk by its position in the concatenation of every
vector (or keyword) list, appended in goroutine completion order. The
second list's best hit ranked behind every hit of the first, and the
order changed between identical requests. Each list is now ranked on
its own by score and a chunk takes its best rank.
- RRF is divided by its maximum, so hybrid scores land in [0, 1] like
vector-only (cosine) and keyword-only (rescaled BM25) results. Raw RRF
topped out near 0.016 and lost to any single-retriever hit it was ranked
against across searches (document targets, model groups, FAQ calls);
rerank's base-score term and MMR's relevance term had nothing to use.
- Keyword-only BM25 is rescaled per list; engines' BM25 is not comparable.
- FAQ iterative retrieval fuses each round like the first pass instead of
sorting raw BM25 and cosine together, starts at twice the depth already
searched, triggers when any single vector list is full (the old summed
count never matched with more than one list) and replaces results per
round.
Rerank and chat pipeline
- The FAQ boost is no longer capped at 1; capping made strong FAQs tie at
1.0 and fall to the tie-breaker. The composite score before boosts is
kept as composite_score.
- The FAQ direct-answer threshold is compared against that pre-boost
score, and the FAQ that cleared it is the one marked match="exact"
rather than the first FAQ.
- Graph hits no longer carry a fabricated retrieval score of 1.0, which
ranked them above every scored hit when rerank did not run; the
composite score uses the model score in its place.
Engines
- OpenSearch and Weaviate return (1 + cos) / 2; their drivers now report
cosine similarity and map thresholds onto the engine scale, so a
threshold means the same on every engine.
- Milvus L2 returned squared distance, ranked farthest-first downstream,
and took the threshold as a radius; scores and radius are converted.
- The multi-store normalizer no longer maps Milvus through (s + 1) / 2,
which put it on a different scale from every other engine and from its
own single-store results.
Docs updated for fusion, normalization, FAQ boost and iterative recall.
* perf(retrieval): bound rerank, agent and pipeline cost; stop failing on slow or missing models
Rerank
- Rerankers report the longest document a request can carry
(max_document_chars / max_request_chars minus the query) and the rerank
stage fits its own passages to it, trimming appended captions, OCR and
generated questions first. One chunk with a large screenshot's OCR used
to exceed the vendor limit and fail the whole batch, so every candidate
lost its score and the threshold stopped applying.
- Rerank HTTP requests default to a 60 s timeout; chat and agent contexts
carry no deadline, so a stalled self-hosted rerank server held the turn
open.
- OCR and caption chunks no longer repeat their own text from image_info in
the passage.
- The chat pipeline and the agent tool rerank at most 200 candidates, best
retrieval score first.
- A rerank model that cannot be loaded degrades to retrieval order instead
of failing every chat turn, as the search APIs already do; without rerank
results the merge stage keeps only RerankTopK hits instead of enriching
every one.
Retrieval APIs
- knowledge-search match_count and rerank.top_k are capped at 200
(unbounded values multiplied into tens of thousands of billed rerank
passages); normalizedMatchCount clamps to the pool size so the
over-retrieval arithmetic cannot overflow.
- hybrid-search with rerank recalls at least rerank.top_k deep.
Agent
- search_knowledge fits its rendered output to four fifths of the tool
output budget, dropping the lowest-ranked rows and telling the model how
many it omitted; the model's view is rebuilt from Data, which registry
truncation never reached.
- Agent searches skip neighbor/parent/related enrichment rows, which were
never retrieved yet tripled the rerank input.
- ParallelToolCalls is enabled. The model was always told it may call tools
in parallel while the engine ran them one by one; only read-only tools
overlap, anything else stays a barrier.
Chat pipeline
- Query expansion searches keywords only (its variants are local rewrites
whose embeddings repeat the main search) and is skipped when keyword
matching is off.
- Entity extraction, a second LLM call, is skipped for intents that need no
KB retrieval and uses the rewritten query.
- Partial-overlap dedup tokenizes each text once instead of per pair, and
only compares token overlap between texts within 3x of each other's size.
Storage
- Migration 000112 adds btree indexes on embeddings(knowledge_id) and
(chunk_id); document deletes, chunk re-index and enable/tag updates
scanned the whole table.
- Shared-KB knowledge rows are fetched in one query instead of one per ID.
- Drop GORM Debug() from the ParadeDB keyword query and the knowledge batch
lookup, which logged SQL and user queries on every search.
Docs updated for the API limits, rerank timeout and passage fitting, agent
parallel tools and output budget, and pipeline changes.
* fix(retrieval): make agent tools and pipeline stages report what they did
Agent tools
- query_knowledge_graph now queries the graph store: entity names from the
query are matched in each KB's graph, relations are returned (and shown to
the model), and the entities' source chunks lead the results, with text
search as a supplement. It used to run text search only, so a graph-only
KB always answered "no relevant graph information".
- search_knowledge reports failures instead of "no matching chunks": when
every search fails the tool fails with the reason; partial failures are
listed. A failed query embedding falls back to keyword search for hybrid
KBs, as the chat pipeline does.
- Keyword mode uses rerank to order hits, not to reject them (identifiers
score low), and the empty-result hint no longer sends keyword mode back to
keyword mode.
- Dedup keeps siblings under one parent, no longer merges every image chunk
of a document through chunk_index 0, and keeps the better-scored copy.
- Search rows omit a <match> excerpt that only repeats the content; snippet
terms are segmented for Chinese queries and skip stopwords, and FAQ
matched questions are chosen by match strength.
- read_document query mode returns next_offset when truncated and resumes
from offset.
- wiki_search over-fetches under document/tag scopes before filtering and
says when every match was out of scope.
- Prompts point to read_document(id=cN, context=k) for surrounding text.
- The agent editor hides the retrieval-strategy section in agent mode,
where search_knowledge uses the global retrieval settings.
Chat pipeline
- Intent labels are normalized; an unknown label retrieves instead of
silently skipping the knowledge base, and web_search without web search
enabled searches the knowledge bases.
- Image turns get a 2048-token rewrite budget, and a reply cut off by the
cap is salvaged field by field instead of being discarded.
- History references are matched by how much of the query they cover;
Jaccard against chunk-sized text never reached its threshold.
- Entity search resolves chunks of org-shared KBs within the turn's KB
scope, skips chunks whose document is gone (it used to panic) and caps
graph chunks at 30.
- Empty history is not fetched twice; fetched pages are cut on a rune
boundary.
Retrieval
- A multi-KB search whose primary KB has no vectors embeds the query with a
vector KB's model instead of failing on an empty model ID.
- Rerank passages include the chunk's heading breadcrumb, which is part of
what was embedded.
- SQLite FTS searches the content column only and turns a lone Han
character into a prefix query; TencentVectorDB ef follows TopK.
Docs updated for the agent tools and pipeline stages.
* fix(retrieval): address review findings on the retrieval fixes
- Restore NeedsRetrieval for web_search without web search enabled: the
intent prompt "web_search" is how the user learns web search is off, and
running KB retrieval instead bypassed it.
- query_knowledge_graph keeps the case of entity terms; Neo4j matches node
names with case-sensitive CONTAINS, so lowercased terms never matched
English entities.
- An empty search_knowledge or query_knowledge_graph result states partial
failures in its text, which is all the model sees for an empty result.
- SQLite BatchSave replaces a source in one transaction, so a failed insert
keeps the old rows instead of leaving the source unindexed.
- Reuse stringSliceValue and an exported searchutil.ContainsChinese instead
of new copies; drop an unused named return and the FAQ obsolete-source
delete that the full per-chunk delete now covers.
* fix(retrieval): address second review of the retrieval fixes
- query_knowledge_graph: under a document/tag scope, keep only relations
whose entities come from an in-scope evidence chunk (the graph namespace
is the whole KB); drop chunks whose document no longer exists; report a
graph or text search failure even when the other source returned hits.
- search_knowledge: record the hybrid -> keyword fallback taken when the
query embedding fails in mode_fallbacks, so an empty result is not read
as "no semantic match".
- query understanding salvage: take a cut-off value only for the image
description; a truncated rewrite_query or intent is dropped instead of
replacing the user's query.
- Graph hits (no retrieval score) are kept past the no-rerank RerankTopK
cut and the rerank candidate cap instead of always falling below them.
- FAQ boost is capped at 1 again so scores stay on the [0, 1] scale; FAQs
tied at the cap are ordered by their pre-boost score.
- Negative-question filtering loads only candidates of FAQ knowledge bases.
- read_document query continuation starts one chunk before offset so the
first match keeps its preceding context at a page boundary.
- graphSearchTerms and searchQueryTokens share one queryTerms splitter.
* docs(readme): name skills, sandbox, browser and MCP in the overview
The overview only mentioned the built-in MCP Server; the agent's own
tools (skill sandboxes, the local browser, external MCP services) first
appeared far down in What's New and the feature tables.
Add an "Agent tools and execution" bullet to all four READMEs, and
update the Agent capability card (README artwork and the homepage it is
copied from) to mention the local browser, with concrete tags instead
of "tool calling / skills".
* docs(readme): condense the English feature and client tables
The four feature tables had 42 rows, many written as release notes
(removed backends, deprecated packages) or UI details, and repeated
what the overview, product tour and What's New already cover.
Replace them with a seven-row feature table linked to the docs and a
supported-backends table that keeps every compatibility list. Drop the
MCP Server subsection and the three client detail blocks, which repeat
the clients table, and shorten the v0.7.x entries in earlier releases.
* docs(readme): drop the v0.8.2 screenshot row
The three screenshots rendered at a third of the column width, too
small to read, and the local browser shot already appears in the
product tour.
* docs(readme): move the agent toolbox out of "beyond the three modes"
Skills, sandboxes, the local browser and MCP tools are part of the
agent mode, not something beside it. Describe them in a short paragraph
right under the capability cards and keep the list for platform
capabilities.
* docs(readme): condense the CN, JA and KO feature and client tables
Bring the translated READMEs to the same structure as the English one:
a seven-row feature table linked to the docs, a supported-backends table
with every compatibility list, no MCP Server subsection or client detail
blocks, and one-line v0.7.x entries.
* fix(desktop): keep Lite signed in with a stable JWT secret
Auto-setup fell through to the login form, and a newly generated signing key replaced the HMAC already used for presigned links. Reuse an existing HMAC, fill only a placeholder JWT secret, and clear a sticky auto-setup failure so the desktop window can enter the local account.
* fix(desktop): make wails dev start locally and store uploads at home
The dev app has no copied config, and the repo env asks for Redis and the Docker path /data/files. Stay on the repo config, use the in-memory stream manager when Redis is unset, write uploads to ~/.weknora/data/files, and allow dragging the top of the window.
* feat(lite): install and run skills on the host instead of a remote sandbox
Lite has no sandbox image, so skills are stored under ~/.weknora/skills as version directories with a live symlink, and chats execute them in the host Seatbelt. The settings UI no longer asks which sandbox to install into. The base profile opens /tmp, chat commands may use the network, and a failed ready-write rolls the symlink back to the previous version.
* fix(ci): align Lite skill checks with the JWT split and lint limits
The desktop signing test still expected the session secret to equal the AES key, and the skill settings source test still looked for the old i18n keys. Wrap the lines golangci-lint rejects so the PR checks can pass.
* fix(ci): separate multi-line test helpers for gofumpt
gofumpt requires a blank line after a multi-line function. The host skill fakes were still packed together, so lint failed on the first one.
* fix(lite): address review findings on host skill install
- Ask mode denies network again, and /tmp writes open only with a
writable root, so nothing runs unapproved.
- Lite still reports and accepts the workspace script switch; the
Skills page offers to re-enable it when it is off.
- The host process refuses symlinks the installer planted before it
clears the install cache or reads the env declaration.
- An existing signing or AES key stays the signer and is never copied
into signing.key; sessions then get their own jwt.key.
- Skill env capture writes under the host target on Lite.
- Sweep keeps the previous version and skips skills another process
holds the lock for.
- Terminal and desktop panels never resolve remote sandboxes on Lite.
- A detached install stream resubscribes instead of freezing.
- Upload migration follows the bundle name and the old relative path.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
The intro video is an HTML animation (misc/intro-video/index.html) that
draws every frame from its timestamp; render.mjs steps it through headless
Chromium into ffmpeg. The product UI is rebuilt in HTML with en-US labels,
and brand icons are loaded from the homepage asset directory rather than
copied.
* fix(desktop): pair the Lite browser against the local API listener
The Wails page origin cannot become the extension WebSocket, and a page-controlled host must not be able to redirect that pairing link.
* fix(desktop): check the proxy test response body close
golangci-lint errcheck rejects an unchecked Body.Close in the new desktop proxy tests.
---------
Co-authored-by: hailongzhao <hailongzhao@tencent.com>