WeKnora/migrations/sqlite
lyingbug 62f7da5eea
feat(agent): persist compaction checkpoints across turns (#3407)
* feat(agent): persist compaction checkpoints across turns

Agent compaction only lived inside the turn that ran it: the summary was
discarded at turn end and LoadAgentHistory rebuilt raw history from the
messages table, so every later turn past the threshold paid for another
summarization of nearly the same history.

A compaction whose history summary ends exactly on a stored turn is now
written onto that turn's assistant message (messages.context_checkpoint).
LoadAgentHistory starts from the newest checkpoint: the summary replaces
that turn and everything before it, and later turns are replayed verbatim.
The compactor already treats a leading summary as the previous summary, so
checkpoints fold forward incrementally.

History messages carry their turn's assistant message ID (chat.Message
TurnID, engine-internal) so compaction can tell whether its cut lands on a
stored-turn boundary. Nothing is persisted when the cut ends inside a stored
turn, covers only the live turn, or the history summary degraded to a raw
archive; a split turn's prefix summary is never persisted because that turn
is replayed whole next time. Writes are column-scoped and best-effort, and a
failed lookup falls back to history without a checkpoint.

Storing the checkpoint on the covered turn keeps the boundary implicit, so
session forks copy it with the turn and deleting the turn drops it. The
HistoryTurns cap still applies, now to the turns after the checkpoint.

* test(skill): wait for the background install in the name-conflict test

InstallSkill queues the install on a goroutine, and the fixture row already
starts at installing, so reading the status right after the call raced the
mock install and failed whenever it landed on ready first. Wait for the run
to finish on the reused row instead, as the other install tests do.

* feat(agent): size agent history by tokens instead of turn count

Agent history was capped at history_turns (default 5) because every turn
used to rebuild raw history and compaction could not persist. With
compaction checkpoints persisted, the cap is no longer what keeps a session
inside the window, and it threw away context that would have fit.

LoadAgentHistory now loads the newest checkpoint summary plus the newest
turns after it that fit a token budget: the run's compaction threshold
(agent.HistoryTokenBudget, the window less the reply reserve). History
that large already triggers the first round's compaction, which persists a
new checkpoint, so compaction rather than a turn count bounds the session.
Loading more would only hand that compaction a summarizer input larger
than the model can read.

Rows are paged backwards by (created_at, id) and the read stops at the
checkpoint or once the budget is full, so a long session is never read
whole. Kept turns stay contiguous, the newest turn is kept even when it
alone exceeds the budget (compaction splits it), and a turn cut by the read
is left out rather than replayed without its original question.

Agent mode no longer reads history_turns: AgentConfig.HistoryTurns is
removed, the editor hides the input in smart-reasoning mode and explains
the automatic behavior, and the smart-reasoning builtins drop the setting.
Quick-answer (KnowledgeQA) keeps its turn cap since it has no compaction.

* Revert "feat(agent): size agent history by tokens instead of turn count"

This reverts commit d05fb6e5f1f161813a88e9127609f7a059f38f12.

Loading history up to the compaction threshold lets the loader drop the
oldest turns before compaction can trigger: whenever a single turn's
replay is larger than the system prompt plus the new question, the
trimmed history stays under the threshold, nothing is summarized, and the
session degrades into a large sliding window that never gets a
checkpoint. A turn-by-turn simulation of LoadAgentHistory with the real
compactor showed zero compactions over 60 turns in those cases.

Keep the persisted checkpoints with the HistoryTurns cap for now; token
budgeted history returns in a separate change with a budget that leaves
the overflow to compaction.
2026-09-18 20:38:56 +08:00
..
000000_init.down.sql Remove redundant Wiki operation log 2026-07-28 11:18:33 +08:00
000000_init.up.sql feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
000001_remove_wiki_log.down.sql Remove redundant Wiki operation log 2026-07-28 11:18:33 +08:00
000001_remove_wiki_log.up.sql Remove redundant Wiki operation log 2026-07-28 11:18:33 +08:00
000002_knowledge_folder_path.down.sql feat(knowledge): store folder path for folder uploads and expose folder tree API 2026-08-04 20:45:29 +08:00
000002_knowledge_folder_path.up.sql feat(knowledge): store folder path for folder uploads and expose folder tree API 2026-08-04 20:45:29 +08:00
000003_knowledge_base_auto_tag_config.down.sql feat: 增加自动添加标签的功能 2026-08-07 15:01:01 +08:00
000003_knowledge_base_auto_tag_config.up.sql feat: 增加自动添加标签的功能 2026-08-07 15:01:01 +08:00
000004_memory.down.sql feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
000004_memory.up.sql feat(memory): add forced consolidation feature and related i18n updates 2026-08-13 18:00:02 +08:00
000005_messages_attachments_and_invitation_fields.down.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000005_messages_attachments_and_invitation_fields.up.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000006_task_queue_and_dead_letters.down.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000006_task_queue_and_dead_letters.up.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000007_system_admin_and_settings.down.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000007_system_admin_and_settings.up.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000008_processing_spans_and_pending_subtasks.down.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000008_processing_spans_and_pending_subtasks.up.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000009_embed_channel_memory_flag.down.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000009_embed_channel_memory_flag.up.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000010_knowledge_multi_tags.down.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000010_knowledge_multi_tags.up.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000011_principal_model.down.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000011_principal_model.up.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000012_message_usage.down.sql feat(agent): report and attribute per-turn LLM token usage 2026-08-24 10:36:19 +08:00
000012_message_usage.up.sql feat(agent): report and attribute per-turn LLM token usage 2026-08-24 10:36:19 +08:00
000013_mcp_tool_enabled.down.sql feat(mcp): add per-tool enable toggle (#2989) 2026-09-05 17:19:20 +08:00
000013_mcp_tool_enabled.up.sql feat(mcp): add per-tool enable toggle (#2989) 2026-09-05 17:19:20 +08:00
000014_browser_authorization.down.sql feat(browser): connect local Chrome with task controls and display modes (#3180) 2026-09-11 17:59:27 +08:00
000014_browser_authorization.up.sql feat(browser): connect local Chrome with task controls and display modes (#3180) 2026-09-11 17:59:27 +08:00
000015_memory_consistency.down.sql fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
000015_memory_consistency.up.sql fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
000016_memory_vector_search.down.sql fix(memory): rank memories by the query, not by importance 2026-09-13 11:43:17 +08:00
000016_memory_vector_search.up.sql fix(memory): rank memories by the query, not by importance 2026-09-13 11:43:17 +08:00
000017_dingtalk_stream_only.down.sql fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
000017_dingtalk_stream_only.up.sql fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
000018_session_fork.down.sql feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
000018_session_fork.up.sql feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
000019_fork_snapshot_lease.down.sql feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
000019_fork_snapshot_lease.up.sql feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
000020_chunks_index_diet.down.sql perf(chunk): speed up chunk ingestion and batch updates (#3369) 2026-09-17 21:38:15 +08:00
000020_chunks_index_diet.up.sql perf(chunk): speed up chunk ingestion and batch updates (#3369) 2026-09-17 21:38:15 +08:00
000021_knowledge_profiles.down.sql feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
000021_knowledge_profiles.up.sql feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
000022_mcp_endpoints.down.sql feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
000022_mcp_endpoints.up.sql feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
000023_message_artifacts_table.down.sql feat(artifacts): artifacts library page backed by a message_artifacts table (#3384) 2026-09-18 12:10:19 +08:00
000023_message_artifacts_table.up.sql feat(artifacts): artifacts library page backed by a message_artifacts table (#3384) 2026-09-18 12:10:19 +08:00
000024_message_context_checkpoint.down.sql feat(agent): persist compaction checkpoints across turns (#3407) 2026-09-18 20:38:56 +08:00
000024_message_context_checkpoint.up.sql feat(agent): persist compaction checkpoints across turns (#3407) 2026-09-18 20:38:56 +08:00