Centralize ingestion base splitter defaults in chunker.NormalizeSplitterConfig
so the preview endpoint applies the same normalization as upload. Regenerate
Swagger for parent-child preview fields and add regression tests for single-
level preview, default parent/child sizes, and context headers.
Wiki generation sometimes rendered "Write in ." because async workers
read empty locale payloads without falling back to context or defaults.
Unify language resolution helpers and apply them at enqueue and consume
time so every wiki and related async prompt gets a non-empty language.
Harden public URL mode: stop logging signed URLs at INFO, copy messages
before rewriting history responses, reuse winning grants on insert races,
sync the Go client resource_urls query param, and add knowledge-search
coverage.
Follow-up on resource_urls=public. Review turned up one authorization gap plus a
set of correctness problems in the streaming path.
Authorization:
- Embed channels were documented as excluded, but nothing enforced it. The
delegating handlers hand the visitor's own gin context to the shared
handlers, so ?resource_urls=public — and a deployment-wide
RESOURCE_URL_MODE=public — reached anonymous traffic and produced
shareable, credential-free URLs that bypass the channel-scoped
/embed/:channel_id/files proxy. ensureEmbedSession now pins the request
context to handle mode, which covers every delegated endpoint including
future ones. The downgrade is silent so a client that forwards the
parameter keeps working.
- A knowledge-base-restricted API key is refused with 403. Such a key is
already denied /files because a raw storage path cannot be bound to its
allow-list, so handing it anonymous file URLs would reopen that hole from
the other side. The default handle mode stays available to it.
Streaming:
- Rewriter.ref holds its lock across resolution. Resolver implementations keep
an unsynchronised per-provider cache, so the previous unlocked window was a
real data race for any caller sharing a StreamRewriter between goroutines —
which its own doc comment invited. Serialising also collapses a concurrent
duplicate into one signature.
- Release the holdback buffer on every path that ends a stream while the
client is still connected: a user-requested stop, an error event, and
giving up on the event store. Only completion flushed before, so the tail
generated just before a stop was silently dropped.
- A released tail carries the metadata of the event it was cut from instead of
a bare event_id.
- The maxHeldBytes safety valve aligns to a UTF-8 boundary rather than cutting
a character in half, and an unclosed " to produce a link at all, and the links it produces
are time-limited but anonymously readable.
Also annotates the affected endpoints for Swagger and refreshes the generated
files for those parameters only, leaving unrelated drift in the committed
swagger output alone.
Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Includes an end-to-end ContinueStream test that asserts the full SSE wire
output. It caught a leak: the references event carries its results twice, and
the copy in Data holds the typed []*SearchResult that CopyData did not
traverse, so handles reached a caller that asked for public URLs. CopyData now
handles the typed slice, []string and map[string]string forms.
Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
API responses reference stored files as opaque resource:// handles, so an
integrating app had to make a second authenticated call to /files for every
image before it could render anything.
Add an opt-in that resolves those references server-side into time-limited
HTTP(S) URLs, using the same mechanism the IM channels already rely on:
- per request: ?resource_urls=public (default: handle, unchanged)
- per deployment: RESOURCE_URL_MODE=public
Applied to the chat SSE endpoints (knowledge-chat, agent-chat,
continue-stream), message history load, and knowledge-search. Streamed answers
buffer a trailing incomplete reference so a handle split across two deltas is
still rewritten. References that cannot become an HTTP URL (for example local
storage with no APP_EXTERNAL_URL) stay handles, so clients keep the /files
fallback. Embed channels are deliberately excluded: their visitors are
anonymous.
SSE payloads share SearchResult pointers and metadata maps with the stream
replay buffer and the message being persisted, so those are rewritten as copies.
Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Document folder tree/move/rename endpoints in docs/api, Swagger, and the Go
client; align frontend path normalization with server caps; guard rename
no-ops and duplicate folder creation; enforce KB ownership on folder rename.
The scope switch is gone. It existed because "select a folder" was ambiguous
in a list that could not show folders, and turning that ambiguity into a
control just moved the problem onto the user.
The list now shows what a folder contains: its sub-folders as entries first,
then the documents directly inside it. So the mode follows what the user is
already doing instead of a switch - browsing shows one level, and any active
filter searches the whole subtree flat, with a breadcrumb note saying so. A
document uploaded on its own needs no special row either: it sits at the top
level next to the folders, exactly where it is.
Folders can also be adjusted now, which they could not before - a mis-filed
upload previously had to be deleted and re-uploaded. Documents move via the
row menu or the batch bar into an existing or newly typed folder, and a
folder can be renamed in place, carrying its subtree. Both are plain
folder_path updates: nothing is re-parsed, re-chunked or re-embedded.
Folder uploads used to encode their relative directory inside file_name,
which made the document list render the whole path as the title and left
no way to browse or query a single folder. The directory now lives in a
dedicated folder_path column (with a backfill for existing rows), the
document list accepts folder_path / folder_recursive filters, and a new
GET /knowledge-bases/:id/knowledge/folders endpoint returns the folder
tree with per-folder document counts.
Reject invalid agent_source_tenant_id instead of silently falling back, fix a resolveAgent nil panic when a shared selector fails, and add regression tests for wiki write-tool filtering and agent resolution.
Discard stale async refreshes without clobbering newer summaries, queue manual refreshes instead of blocking HTTP, sort edited chunks by index for summaries, escape document metadata in chat headers, and distinguish lookup failures from true staleness.
Co-authored-by: Cursor <cursoragent@cursor.com>
CleanMarkdown regex patterns (script, onclick=, javascript:, etc.)
unconditionally delete matching substrings from chunk content, even when
they appear in code blocks or inline code.
JSON APIs should return raw stored data; XSS protection is the frontend
markdown renderer's responsibility.
SanitizeForDisplay and CleanMarkdown are retained in security.go.
Store superseded wiki page versions for diff and rollback, attribute edits
to user/agent/pipeline/revert sources, and expose revision APIs plus an
editor UI with optimistic locking on save.
Search tenants for invite by workspace name only (a user may belong to
multiple workspaces, so username/email matching was ambiguous). Align
organization i18n to distinguish shared spaces from workspaces, fix the
duplicate zh-CN members key, and wrap invite-code generation response in
data so the frontend and Go SDK can read it.
Treat limit as the authoritative upper bound instead of clamping it down
to the agent's Starters.Count, so ?limit=N no longer always returns 6.
Fixes#2238
Downloading exposes the original source file, which is more sensitive than viewing parsed content or previews. Align backend route guards, handler checks, and frontend UI so tenant Viewers and org-shared read-only access cannot download.
Use the deploy-time file size configuration for model debugger uploads
instead of a hardcoded 20 MB cap, and validate file size on the frontend
before sending the request.
Operators can clear all finally-failed tasks in one queue from the runtime
drawer, with audit logging and scoped API access. Also localize the
chat_attachment queue label in the runtime queue details table.
Prevent concurrent refresh-token rotation across instances, distinguish
expired-but-refreshable tokens from active authorization, and bind OAuth
popup polling to a per-attempt ID so stale tokens cannot satisfy new consent.
- Updated the listFAQEntries function to support multiple tag IDs for filtering.
- Modified the frontend components to accommodate the new tag_ids parameter.
- Enhanced the KbTagManageDrawer and FAQEntryManager components to reflect changes in tag handling.
- Improved API documentation to clarify the new tag filtering capabilities.
This commit enhances the flexibility of FAQ entry retrieval, allowing users to filter by multiple tags for more precise results.
- Updated the exportFAQEntries function to support exporting FAQ entries in both CSV and JSON formats.
- Modified the frontend to include options for selecting the export format.
- Enhanced localization files to reflect the new export options in English, Russian, and Chinese.
- Implemented new tests to validate the export functionality for both formats.
This commit improves the flexibility of FAQ exports, allowing users to choose their preferred format for better integration with other tools.
Automatically remove wiki folders that become empty when knowledge is
retracted, using the durable finalize lane so pruning waits until ingest
drains. Repository DeleteFolder now atomically checks emptiness to avoid
races with concurrent page or child-folder writes.
Store authorization timestamps as timestamptz and normalize Unix expiry
inputs to UTC so tenant/platform API keys expire at the intended instant
even when the app process timezone differs from the PostgreSQL session.
Introduce platform API keys with capability-based access to tenant management,
system settings, runtime queues, and audit logs, plus admin UI for key lifecycle
and system audit log viewing.
Restrict channel-managed sessions (API keys, IM, embed) to tenant admin
in the Web console for list/read paths, hide channel sidebar buckets from
non-admins, and add CountSessionsBySource for aggregate stats without
exposing session rows to viewers.
Add GetOwnedSession method to enforce strict ownership checks for session mutations, ensuring tenant admins can read but not modify API-key sessions. Update relevant handlers to utilize this method for operations that alter session data, maintaining security and integrity of API-key sessions.
Let tenant admins list and open API-key sessions from the Web console while
keeping per-key isolation for everyone else. Pin the session source filter so
it no longer jumps when switching buckets, and resolve session origin on read
so deep links stay in sync with the active folder after refresh.
Lark and Feishu are the same product deployed on two isolated clouds
(open.larksuite.com and open.feishu.cn). WeKnora touches both from two
places — the IM bot and the Wiki data-source connector — and this brings
Lark to parity across both. Neither API surface differs, so rather than
duplicate either implementation, each is parametrized by a Region and
registered twice.
IM (internal/im/feishu):
Every API call routes through a.api(), which prefixes the region's
host, and the WebSocket client passes larkws.WithDomain — the SDK
defaults to open.feishu.cn, so Lark long connections would otherwise
fail to authenticate.
Data source (internal/datasource/connector/feishu):
Lark was half-supported: the client honoured a base_url override and a
LarkBaseURL constant existed, but the connector registered only as
"feishu" and the UI never exposed base_url, so Lark wikis could not be
configured at all. Lark is now its own connector type. base_url is kept
as an explicit override so data sources that pointed a "feishu"
connector at open.larksuite.com keep working.
Bugs found and fixed along the way:
- Wiki resource links were hardcoded to https://feishu.cn/wiki/{token} in
four places, so a Lark wiki linked into the wrong cloud entirely.
- Bot identity is now platform-prefixed (feishu:cli_x vs lark:cli_x).
BotIdentity carries a unique index, so without this a Lark channel
would be rejected as a duplicate of an unrelated Feishu bot sharing an
app_id.
- The card image cache is now keyed by app_id. image_keys are issued per
app on one cloud; a Feishu key handed to a Lark app fails the card
update with code=200570.
- The datasource doc claimed Lark "adapts automatically" — it could not,
since the UI had no way to reach base_url.
Permissions are where the clouds genuinely diverge. Identical APIs do not
mean identical permission catalogues: aily:file:read, aily:file:write and
corehr:file:download exist on Feishu but not on Lark, so pasting Feishu's
scope JSON into Lark fails to import. That JSON is also over-broad (it
covers products WeKnora never calls) and simultaneously incomplete (it
omits the drive:*/docx:* scopes the connector needs). Lark therefore gets
its own list, derived from the 17 endpoints the code actually calls and
checked identifier-by-identifier against Lark's own API reference, plus a
full Feishu→Lark mapping table marking each scope verified, unverified,
or absent. Both features use tenant_access_token only — no user_access_
token exists anywhere in the repo — so the user dimension stays empty.
Known limitation: Region.ThinkingText makes Lark's initial card
placeholder English, but once streaming starts the think-block headers
come from the shared MarkdownThinkStyle and are Chinese on every platform
(Lark, Slack, Telegram, Mattermost alike). Fixing that means threading a
locale through FormatIMDisplayContent, which has no platform context — a
cross-platform change, not a Lark one. Documented as such.
The channel-creation 400 message is now derived from validIMPlatforms so
the list cannot drift as platforms are added.
Tests: region invariants and per-region URL construction on both sides,
the Feishu/Lark bot-identity split, platform-to-channel mapping, base_url
override precedence, an end-to-end SendReply against a fake Open Platform
(token fetch, reply, thread-unsupported fallback, hard-error surfacing),
and a ListResources drive asserting wiki links never leak the other
cloud's host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Update the SuggestedQuestionsRequest to include TagScopes for better filtering of knowledge items based on their parent knowledge bases. Modify related API documentation and frontend interfaces to accommodate the new structure. Ensure backend services handle the new tag scope logic correctly, including tests for the updated functionality.
Add functionality to normalize temporary attachment IDs, ensuring deduplication and enforcing a maximum limit of 5 attachments per message. Update related handlers and tests to support this new behavior, enhancing the robustness of attachment management in chat sessions.