WeKnora/internal
wizardchen 3d645d2578 fix(datasource/ima): retry transient sync failures instead of losing documents
The sync cursor was built from the raw IMA listing before any content was
fetched, so an item whose get_media_info or download failed was still recorded
as "seen with this media_id". Every later incremental sync then skipped it as
unchanged and the document was silently missing from the knowledge base for
good. Track the listed keys and the successfully synced keys separately: the
cursor now only remembers what was actually resolved, while delete detection
keeps using the full listing so a failed download is never mistaken for a
document that vanished. Deterministic skips (unsupported media types, items
with no URL) are still recorded so they are not re-probed every cycle.

Alongside that:

- Drop media_type from logicalKey. get_knowledge_list omits the field, so
  folding it in meant every external_id would shift the day IMA starts
  populating it, which reads downstream as "every document deleted".
- Download URL-backed media that carry auth headers rather than handing the
  bare URL to the ingest layer, which cannot replay those headers.
- Derive image extensions from the response Content-Type; IMA reports every
  image as media_type=9, so a JPEG was being written as .png.
- Stop logging API response bodies, which carry pre-signed storage URLs and
  per-request auth headers.
- Validate media URLs against the SSRF policy before the first connection.
- Stop writing the redundant kb_media map into the persisted cursor.
- Say in the connector description that notes and AI sessions are skipped.
2026-08-12 14:56:25 +08:00
..
agent feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
application fix(wiki): set MaxTokens for generateWithTemplate LLM calls 2026-08-12 11:45:06 +08:00
assets feat(assets): add ASR test audio file and embed it in the application 2026-04-02 21:27:27 +08:00
common feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
config fix(config): 补齐 WEKNORA_TENANT_ENABLE_CROSS_TENANT_ACCESS 的 env 读取 2026-08-10 19:40:09 +08:00
container feat(datasource): add Tencent IMA datasource 2026-08-12 14:56:25 +08:00
database fix(knowledge): address auto-tag review findings 2026-08-07 15:01:01 +08:00
datasource fix(datasource/ima): retry transient sync failures instead of losing documents 2026-08-12 14:56:25 +08:00
errors refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
event feat(mcp): prompt and resume OAuth authorization mid-conversation 2026-06-29 23:22:28 +08:00
handler feat: support updating API key configuration 2026-08-12 11:43:03 +08:00
im feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
infrastructure feat: add Exa WebSearch (#2617) 2026-08-11 17:05:52 +08:00
logger feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
mcp fix(security): harden outbound URL SSRF protection (#2626) 2026-08-10 13:24:14 +08:00
middleware feat(auth): add /auth/oidc/start endpoint for direct 302 OIDC login 2026-08-11 15:41:48 +08:00
modelcontext feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
models fix(embedding): validate WeKnoraCloud response indexes 2026-08-11 16:59:48 +08:00
ratelimit feat(embed): add backend embed channels with auth and rate limiting (#1657) 2026-06-12 23:10:39 +08:00
router feat: support updating API key configuration 2026-08-12 11:43:03 +08:00
runtime fix(redis): harden TLS tests and surface TLS config at startup 2026-07-06 23:04:52 +08:00
sandbox feat(sandbox): enhance logging for sandbox operations and error diagnostics 2026-08-12 10:53:46 +08:00
searchutil fix(chunks): trim trusted overlaps by exact position, not text search 2026-08-06 12:26:15 +08:00
storageallowlist fix(storage): align default provider fallback with STORAGE_ALLOW_LIST 2026-07-15 11:05:00 +08:00
storageurl fix(api): address resource_urls review follow-ups 2026-08-05 14:42:51 +08:00
stream feat(redis): support TLS for Redis connections (#1930) 2026-07-06 23:02:02 +08:00
tracing/langfuse feat(llm): add prompt-cache observability and wiki cache-friendly prompts 2026-07-23 19:17:30 +08:00
types feat(datasource): add Tencent IMA datasource 2026-08-12 14:56:25 +08:00
utils fix(security): harden outbound URL SSRF protection (#2626) 2026-08-10 13:24:14 +08:00