Harden public URL mode: stop logging signed URLs at INFO, copy messages
before rewriting history responses, reuse winning grants on insert races,
sync the Go client resource_urls query param, and add knowledge-search
coverage.
Follow-up on resource_urls=public. Review turned up one authorization gap plus a
set of correctness problems in the streaming path.
Authorization:
- Embed channels were documented as excluded, but nothing enforced it. The
delegating handlers hand the visitor's own gin context to the shared
handlers, so ?resource_urls=public — and a deployment-wide
RESOURCE_URL_MODE=public — reached anonymous traffic and produced
shareable, credential-free URLs that bypass the channel-scoped
/embed/:channel_id/files proxy. ensureEmbedSession now pins the request
context to handle mode, which covers every delegated endpoint including
future ones. The downgrade is silent so a client that forwards the
parameter keeps working.
- A knowledge-base-restricted API key is refused with 403. Such a key is
already denied /files because a raw storage path cannot be bound to its
allow-list, so handing it anonymous file URLs would reopen that hole from
the other side. The default handle mode stays available to it.
Streaming:
- Rewriter.ref holds its lock across resolution. Resolver implementations keep
an unsynchronised per-provider cache, so the previous unlocked window was a
real data race for any caller sharing a StreamRewriter between goroutines —
which its own doc comment invited. Serialising also collapses a concurrent
duplicate into one signature.
- Release the holdback buffer on every path that ends a stream while the
client is still connected: a user-requested stop, an error event, and
giving up on the event store. Only completion flushed before, so the tail
generated just before a stop was silently dropped.
- A released tail carries the metadata of the event it was cut from instead of
a bare event_id.
- The maxHeldBytes safety valve aligns to a UTF-8 boundary rather than cutting
a character in half, and an unclosed " URLs, using the same mechanism the IM channels already rely on:
- per request: ?resource_urls=public (default: handle, unchanged)
- per deployment: RESOURCE_URL_MODE=public
Applied to the chat SSE endpoints (knowledge-chat, agent-chat,
continue-stream), message history load, and knowledge-search. Streamed answers
buffer a trailing incomplete reference so a handle split across two deltas is
still rewritten. References that cannot become an HTTP URL (for example local
storage with no APP_EXTERNAL_URL) stay handles, so clients keep the /files
fallback. Embed channels are deliberately excluded: their visitors are
anonymous.
SSE payloads share SearchResult pointers and metadata maps with the stream
replay buffer and the message being persisted, so those are rewritten as copies.
Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
The IM channels rewrite resource:// and provider:// references into loadable
HTTP URLs because IM clients cannot attach WeKnora credentials to an image
fetch. That logic was private to internal/im, so no other surface could reuse
it.
Move the reference pattern, HTTP-result guard, tenant-aware storage backend
resolver and streaming holdback helpers into internal/storageurl, and add a
Rewriter that memoises resolutions (a resource:// handle costs one access-grant
row per resolution). internal/im now delegates; behaviour is unchanged.
Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>