WeKnora/internal/agent
wizardchen d0144f3586 perf(wiki): move ingest log to event table and index to on-demand API
Fixes O(n²) write amplification during wiki ingest on large KBs. Previously
every ingest/retract op re-wrote the single `slug='log'` row end-to-end and
every batch re-wrote the entire `slug='index'` directory markdown. On a 40k-
doc KB the log row grew to tens of MB and the index row to several MB, so
each batch triggered giant TOAST updates that dominated ingest wall time.

Log: new `wiki_log_entries` event table (`id DESC` indexed per KB) replaces
the single TEXT row. Batch ingest now collects entries and flushes them
once per batch via `AppendBatch`. Each entry stores `pages_affected` as
JSONB `[{slug,title}]` so the UI can render real titles; custom Scan falls
back to legacy `[]string` so older rows still deserialize.

Index: `wiki_pages[slug=index].content` keeps only the LLM-generated intro
(a few KB). The directory is now served by a structured paginated API
(`GetIndexView`) that reads `slug/title/summary` per type with cursor
pagination, so the agent and the frontend only pull the slice they need.
`RebuildIndexPage` degrades to a no-op; agent `wiki_read_page('index')`
synthesizes a small top-K overview and points callers at `wiki_search`.

Ingest resilience: LLM calls wrap with 3-attempt exponential backoff on
transient errors (5xx/408/429, transport resets/timeouts). Summary/extract
failures now bubble up so the batch's failed-op requeue path runs instead
of silently dropping the doc.

Frontend: sidebar Index/Log entries switch to dedicated views. Index view
streams intro → Summary → Entity → Concept → Synthesis → Comparison via
IntersectionObserver (with a nextTick re-check so small KBs still load
every section). Log view uses cursor pagination; pages_affected renders
titles with slug tooltip.

- new migration: migrations/versioned/000040_wiki_log_entries.{up,down}.sql
- tests: log repo pagination + legacy Scan, ListByTypeLight windowing,
  renderIndexOverviewForAgent output, isTransientLLMError classifier

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 21:48:52 +08:00
..
memory feat(agent): enhance message consolidation and context management 2026-03-25 22:08:29 +08:00
skills feat: adapt preloaded skills to smart-reasoning agent 2026-02-04 20:08:49 +08:00
token feat(agent): enhance message consolidation and context management 2026-03-25 22:08:29 +08:00
tools perf(wiki): move ingest log to event table and index to on-demand API 2026-05-08 21:48:52 +08:00
act.go feat(observability): instrument agent ReAct loop and tool calls in Langfuse 2026-04-24 19:58:08 +08:00
const.go feat: Implement wiki_config updates in knowledge base management 2026-04-22 21:17:21 +08:00
engine.go feat(ui): refactor IM channels list layout and form\n\n- Change IM channel form from dialog to drawer for better UX\n- Convert channel list from vertical layout to responsive grid cards\n- Move edit/delete actions into a dropdown menu for cleaner card design\n- Replace platform radio group with a select dropdown in the add channel form 2026-04-29 17:34:20 +08:00
engine_image_test.go feat: auto-describe MCP tool result images via VLM 2026-03-25 21:38:15 +08:00
engine_test.go fix(agent): 修复WeKnora对话完前端未显示结束标记问题 2026-05-07 16:34:20 +08:00
finalize.go fix(agent): 修复WeKnora对话完前端未显示结束标记问题 2026-05-07 16:34:20 +08:00
langfuse_test.go feat(observability): instrument agent ReAct loop and tool calls in Langfuse 2026-04-24 19:58:08 +08:00
observe.go fix(agent): terminate ReAct loop on final_answer regardless of arg parse success 2026-04-23 19:36:28 +08:00
observe_test.go fix(agent): terminate ReAct loop on final_answer regardless of arg parse success 2026-04-23 19:36:28 +08:00
prompts.go feat: Update agent type presets and prompt templates for improved knowledge base handling 2026-04-22 21:18:34 +08:00
prompts_wiki.go fix(ingest): stop hallucinating summaries from filenames on empty PDFs 2026-05-06 11:23:32 +08:00
prompts_wiki_test.go feat: Enhance wiki extraction and configuration options 2026-04-22 21:18:32 +08:00
think.go chore(agent): log natural-stop candidate when no tool calls 2026-05-07 18:28:28 +08:00