WeKnora/migrations/versioned
wizardchen 44d6175559 feat: add knowledge parse cancellation with finalizing post-process state
Lets users stop an in-flight document parse to free up LLM / worker
resources without losing the chunks and index already written. The
core insight is that the previous parse_status=completed flipped as
soon as primary chunks landed, while the most expensive subtasks
(graph extract = N LLM calls per chunk, plus summary, question
generation) were still running in the background — so "completed"
wasn't actually terminal from a resource standpoint.

State machine

  pending -> processing -> finalizing -> completed
                              |
                              +-> cancelled (any of the three
                                            in-flight states)
                              +-> failed
                              +-> deleting

`finalizing` is the new post-process fan-out window. parse_status
only promotes to `completed` once pending_subtasks_count (a new
column tracking summary + question + per-chunk graph extract)
drains to zero via atomic FinalizeSubtask. Wiki ingest is
intentionally excluded from the counter — it's a KB-scoped
debounced batch and would otherwise pin parse_status in
`finalizing` for the wiki batch window.

Backend

- New ParseStatusFinalizing + pending_subtasks_count column with
  migration 000056.
- knowledgeRepository.SetFinalizing transitions processing -> finalizing
  conditionally so a racing cancel cannot be clobbered.
- knowledgeRepository.FinalizeSubtask atomically decrements the
  counter and self-promotes the row to completed when it hits zero.
- KnowledgePostProcess restructured to compute expected subtask
  count up front, flip to finalizing (or completed when no
  enrichment is enabled), and only then fan out subtasks. Subtask
  handlers (summary, question, graph extract) defer-decrement on
  terminal exit using the existing isFinalAsynqAttempt convention.
- New POST /api/v1/knowledge/{id}/cancel-parse handler accepting
  pending / processing / finalizing. Marks the row cancelled,
  zeroes the counter, best-effort dequeues asynq tasks via a new
  TaskInspector abstraction (asynq-mode walks pending/scheduled/
  retry queues; Lite-mode noop), and scrubs wiki ingest pending op.
- SpanTracker.AbortAttempt flat-sweeps every still-running span
  for the attempt via a new repo.CancelAllOpenSpans helper so the
  trace viewer's striped bars all flip to cancelled, even leaf
  generations whose parent stage already EndSpan'd (multimodal
  fan-out pattern). knowledge_post_process closes its postSpan
  via SkipSpan on the cancel/deleting entry guard so a worker
  that opens a span AFTER the cancel sweep doesn't leak it.
- Housekeeping and resetPendingTasks sweep finalizing rows
  identically to processing so a crash/restart can't strand them.
- DeleteKnowledge/DeleteKnowledgeList proactively dequeue
  downstream tasks via the same TaskInspector path.
- ChunkExtractService gets a cancel entry guard so the most
  expensive enrichment (graph extract) bails immediately when the
  parent knowledge is aborted.

Frontend

- New cancelKnowledgeParse API client + "Stop parsing" entry in
  both list view and card view more menus, gated on
  pending/processing/finalizing.
- Polling predicate refactored to a shared isParseInFlight helper
  that recognises `finalizing` (previously the doc list silently
  stopped polling once parse_status flipped from processing).
- Knowledge processing timeline: isPolling includes finalizing,
  new isHardTerminal short-circuits LIVE for cancelled/failed/
  completed so stranded child spans cannot pin LIVE on.
- DocumentListView.computeStatus distinguishes finalizing
  ("增强中") from completed and shows the previous "生成摘要中"
  copy when summary_status is still pending under finalizing.
  Added cancelled badge as well.
- i18n: statusFinalizing / statusCancelled / cancelParse* keys
  across zh-CN, en-US, ko-KR, ru-RU.

Docs / SDK

- docs/api/knowledge.md: documents the new finalizing state,
  cancel-parse semantics, and which statuses accept cancel.
- client (Go SDK): CancelKnowledgeParse with docstring listing
  the cancellable statuses.
2026-05-28 20:16:02 +08:00
..
000000_init.down.sql feat(database): update database initialization to conditionally handle embeddings 2025-12-10 17:13:30 +08:00
000000_init.up.sql fix: add model display name 2026-05-28 19:22:17 +08:00
000001_agent.down.sql feat(database): update database initialization to conditionally handle embeddings 2025-12-10 17:13:30 +08:00
000001_agent.up.sql feat(database): update database initialization to conditionally handle embeddings 2025-12-10 17:13:30 +08:00
000002_embeddings.down.sql feat(database): update database initialization to conditionally handle embeddings 2025-12-10 17:13:30 +08:00
000002_embeddings.up.sql feat(database): update database initialization to conditionally handle embeddings 2025-12-10 17:13:30 +08:00
000003_chunk_flags.down.sql feat: 新增FAQ推荐标签和批量操作支持,优化标签删除逻辑 2025-12-12 19:57:08 +08:00
000003_chunk_flags.up.sql feat: 新增FAQ推荐标签和批量操作支持,优化标签删除逻辑 2025-12-12 19:57:08 +08:00
000004_drop_vlm_model_id.down.sql fix: 强制移除knowledge_bases表中的历史vlm_model_id字段 2025-12-22 13:11:31 +08:00
000004_drop_vlm_model_id.up.sql fix: 强制移除knowledge_bases表中的历史vlm_model_id字段 2025-12-22 13:11:31 +08:00
000005_mentioned_items.down.sql feat: 支持在消息流中展示@的知识库和文件 2025-12-22 13:11:31 +08:00
000005_mentioned_items.up.sql feat: 支持在消息流中展示@的知识库和文件 2025-12-22 13:11:31 +08:00
000006_custom_agents.down.sql refactor: 调整智能体配置,增强智能体选择与状态管理 2025-12-26 21:31:02 +08:00
000006_custom_agents.up.sql refactor: 调整智能体配置,增强智能体选择与状态管理 2025-12-26 21:31:02 +08:00
000007_embeddings_tag_id.down.sql feat: 支持FAQ两级优先级标签过滤 2025-12-30 19:44:53 +08:00
000007_embeddings_tag_id.up.sql docs: sync changes to v0.2.8 changelog 2025-12-31 19:54:03 +08:00
000008_migrate_untagged_faq.down.sql feat: 支持删除文档类型标签 2026-01-15 19:38:16 +08:00
000008_migrate_untagged_faq.up.sql fix: guard pg_search update with skip_embedding 2026-03-19 22:03:09 +08:00
000009_add_last_faq_import_result.down.sql feat: 支持删除文档类型标签 2026-01-15 19:38:16 +08:00
000009_add_last_faq_import_result.up.sql feat: 支持删除文档类型标签 2026-01-15 19:38:16 +08:00
000010_add_seq_id.down.sql feat: 为 FAQ 和标签引入 seq_id 支持 2026-01-14 17:13:44 +08:00
000010_add_seq_id.up.sql feat: 为 FAQ 和标签引入 seq_id 支持 2026-01-14 17:13:44 +08:00
000011_pg_search_update.down.sql feat(migrations): add pg_search extension update migration files 2026-01-30 19:30:35 +08:00
000011_pg_search_update.up.sql fix: guard pg_search update with skip_embedding 2026-03-19 22:03:09 +08:00
000012_organizations.down.sql feat(agent): enhance agent and skill management features 2026-02-09 11:40:25 +08:00
000012_organizations.up.sql feat(agent): enhance agent and skill management features 2026-02-09 11:40:25 +08:00
000013_engine_configs.down.sql feat: introduce WeKnora Lite edition with lightweight configuration and deployment 2026-03-02 21:21:49 +08:00
000013_engine_configs.up.sql refactor: remove mineru_api_base_url from ParserEngineConfig and related components 2026-03-03 13:24:20 +08:00
000014_storage_provider_config.down.sql feat: introduce storage provider configuration and enhance backward compatibility 2026-03-02 21:21:49 +08:00
000014_storage_provider_config.up.sql feat: introduce storage provider configuration and enhance backward compatibility 2026-03-02 21:21:49 +08:00
000015_add_is_fallback.down.sql feat: add fallback response handling and UI indicators 2026-03-05 18:14:14 +08:00
000015_add_is_fallback.up.sql feat: add fallback response handling and UI indicators 2026-03-05 18:14:14 +08:00
000016_add_kb_pinned.down.sql feat: add pinning functionality for knowledge bases 2026-03-05 18:14:14 +08:00
000016_add_kb_pinned.up.sql feat: add pinning functionality for knowledge bases 2026-03-05 18:14:14 +08:00
000017_mcp_builtin.down.sql feat(security): add AES-256-GCM encryption for API keys at rest 2026-03-09 10:35:07 +08:00
000017_mcp_builtin.up.sql feat(security): add AES-256-GCM encryption for API keys at rest 2026-03-09 10:35:07 +08:00
000018_extend_tenant_api_key.down.sql feat(security): add AES-256-GCM encryption for API keys at rest 2026-03-09 10:35:07 +08:00
000018_extend_tenant_api_key.up.sql feat(security): add AES-256-GCM encryption for API keys at rest 2026-03-09 10:35:07 +08:00
000019_add_agent_duration_ms.down.sql chore: update AWS SDK dependencies and add database migrations 2026-03-09 11:46:56 +08:00
000019_add_agent_duration_ms.up.sql chore: update AWS SDK dependencies and add database migrations 2026-03-09 11:46:56 +08:00
000020_add_message_knowledge_id.down.sql chore: update AWS SDK dependencies and add database migrations 2026-03-09 11:46:56 +08:00
000020_add_message_knowledge_id.up.sql chore: update AWS SDK dependencies and add database migrations 2026-03-09 11:46:56 +08:00
000021_im_channel.down.sql feat: implement IM channel sessions and update message handling 2026-03-16 02:40:42 +08:00
000021_im_channel.up.sql feat: implement IM channel sessions and update message handling 2026-03-16 02:40:42 +08:00
000022_message_images.down.sql feat: implement IM channel sessions and update message handling 2026-03-16 02:40:42 +08:00
000022_message_images.up.sql feat: implement IM channel sessions and update message handling 2026-03-16 02:40:42 +08:00
000023_im_channel_kb_id.down.sql feat(im): add file support and knowledge base integration 2026-03-17 22:23:24 +08:00
000023_im_channel_kb_id.up.sql feat(im): add file support and knowledge base integration 2026-03-17 22:23:24 +08:00
000024_im_channel_bot_identity.down.sql feat(im): add file support and knowledge base integration 2026-03-17 22:23:24 +08:00
000024_im_channel_bot_identity.up.sql feat(im): add file support and knowledge base integration 2026-03-17 22:23:24 +08:00
000025_message_channel.down.sql feat(knowledge): add channel support to knowledge entries and related components 2026-03-25 22:08:29 +08:00
000025_message_channel.up.sql feat(knowledge): add channel support to knowledge entries and related components 2026-03-25 22:08:29 +08:00
000026_chunks_query_indexes.down.sql feat(agent): enhance chunk search functionality to include total chunk counts 2026-03-25 22:08:29 +08:00
000026_chunks_query_indexes.up.sql feat(agent): enhance chunk search functionality to include total chunk counts 2026-03-25 22:08:29 +08:00
000027_message_rendered_content.down.sql feat(prompt): enhance context and intent classification in templates 2026-03-26 18:57:12 +08:00
000027_message_rendered_content.up.sql feat(prompt): enhance context and intent classification in templates 2026-03-26 18:57:12 +08:00
000028_im_thread_session.down.sql [NELO-1102] feat(im): add thread-based session mode for IM channels 2026-03-30 10:43:14 +08:00
000028_im_thread_session.up.sql [NELO-1102] feat(im): add thread-based session mode for IM channels 2026-03-30 10:43:14 +08:00
000029_datasource_tables.down.sql feat(web-search): refactor web search provider 2026-03-31 20:22:34 +08:00
000029_datasource_tables.up.sql fix(migrations): correct migration numbers and remove broken trigger 2026-04-03 11:31:21 +08:00
000030_web_search_providers.down.sql feat(web-search): refactor web search provider 2026-03-31 20:22:34 +08:00
000030_web_search_providers.up.sql fix(migrations): correct migration numbers and remove broken trigger 2026-04-03 11:31:21 +08:00
000031_add_asr_config.down.sql feat(asr): add ASR model connection check and update related components 2026-04-02 21:27:27 +08:00
000031_add_asr_config.up.sql fix(migrations): correct migration numbers and remove broken trigger 2026-04-03 11:31:21 +08:00
000032_vector_stores.down.sql feat: add VectorStore entity, repository, and migrations 2026-04-09 01:39:51 +08:00
000032_vector_stores.up.sql feat: add VectorStore entity, repository, and migrations 2026-04-09 01:39:51 +08:00
000033_add_video_info_to_chunks.down.sql fix: 修正视频相关迁移文件版本号冲突,从 000032 改为 000033 2026-04-10 22:01:42 +08:00
000033_add_video_info_to_chunks.up.sql fix: 修正视频相关迁移文件版本号冲突,从 000032 改为 000033 2026-04-10 22:01:42 +08:00
000034_add_attachments.down.sql fix: rename sql file 2026-04-14 14:54:21 +08:00
000034_add_attachments.up.sql fix: rename sql file 2026-04-14 14:54:21 +08:00
000035_add_credentials.down.sql refactor(weknoracloud): update credential handling and configuration 2026-04-15 11:37:24 +08:00
000035_add_credentials.up.sql refactor(weknoracloud): update credential handling and configuration 2026-04-15 11:37:24 +08:00
000036_kb_vector_store_id.down.sql feat: add vector_store_id column to knowledge_bases for per-KB vector store binding 2026-04-20 19:53:44 +08:00
000036_kb_vector_store_id.up.sql feat: add vector_store_id column to knowledge_bases for per-KB vector store binding 2026-04-20 19:53:44 +08:00
000037_wiki_and_indexing.down.sql feat: Remove outdated wiki and indexing migration files 2026-04-22 21:24:08 +08:00
000037_wiki_and_indexing.up.sql feat: Remove outdated wiki and indexing migration files 2026-04-22 21:24:08 +08:00
000038_im_session_agent_id.down.sql fix(im): isolate IM sessions per agent and recover from deleted sessions 2026-04-28 20:53:06 +08:00
000038_im_session_agent_id.up.sql fix(im): isolate IM sessions per agent and recover from deleted sessions 2026-04-28 20:53:06 +08:00
000039_session_user_id_and_pin.down.sql feat(sessions): add keyword search, user-scoped pinning, and IM source visibility 2026-04-30 16:23:00 +08:00
000039_session_user_id_and_pin.up.sql feat(sessions): add keyword search, user-scoped pinning, and IM source visibility 2026-04-30 16:23:00 +08:00
000040_wiki_log_entries.down.sql perf(wiki): move ingest log to event table and index to on-demand API 2026-05-08 21:48:52 +08:00
000040_wiki_log_entries.up.sql perf(wiki): move ingest log to event table and index to on-demand API 2026-05-08 21:48:52 +08:00
000041_task_queue_and_wiki_indexes.down.sql perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
000041_task_queue_and_wiki_indexes.up.sql fix(migration): ensure pg_trgm is created before trigram index in 000041 2026-05-14 16:51:21 +08:00
000042_mcp_tool_approval.down.sql feat(agent): human-in-the-loop approval for MCP tool calls (#1173) 2026-05-10 22:57:12 +08:00
000042_mcp_tool_approval.up.sql feat(agent): human-in-the-loop approval for MCP tool calls (#1173) 2026-05-10 22:57:12 +08:00
000043_tenant_rbac.down.sql refactor(rbac): address PR 1 review feedback 2026-05-18 17:28:58 +08:00
000043_tenant_rbac.up.sql feat(rbac): add tenant_members schema and auth wiring (PR 1 of 3, #1303) 2026-05-18 17:28:58 +08:00
000044_audit_log.down.sql feat(migration): add index on created_at for audit_logs 2026-05-18 17:28:58 +08:00
000044_audit_log.up.sql feat(migration): add index on created_at for audit_logs 2026-05-18 17:28:58 +08:00
000045_org_tenant_members.down.sql fix(rbac): dedup pending join/upgrade requests per (org, tenant, type) 2026-05-18 17:28:58 +08:00
000045_org_tenant_members.up.sql fix(rbac): dedup pending join/upgrade requests per (org, tenant, type) 2026-05-18 17:28:58 +08:00
000046_org_owner_tenant_id.down.sql fix(rbac): pin organization owner tenant in DB instead of deriving at runtime 2026-05-18 17:28:58 +08:00
000046_org_owner_tenant_id.up.sql fix(rbac): pin organization owner tenant in DB instead of deriving at runtime 2026-05-18 17:28:58 +08:00
000047_user_resource_favorites.down.sql feat(favorites): implement user favorites and recents functionality 2026-05-18 17:28:58 +08:00
000047_user_resource_favorites.up.sql feat(favorites): implement user favorites and recents functionality 2026-05-18 17:28:58 +08:00
000048_tenant_invitations.down.sql feat(invitations): implement pending invitations feature with polling and dialog 2026-05-18 17:28:58 +08:00
000048_tenant_invitations.up.sql feat(invitations): implement pending invitations feature with polling and dialog 2026-05-18 17:28:58 +08:00
000049_user_preferences.down.sql feat(user-preferences): implement user preferences management with backend support 2026-05-18 17:28:58 +08:00
000049_user_preferences.up.sql feat(user-preferences): implement user preferences management with backend support 2026-05-18 17:28:58 +08:00
000050_user_kb_pins.down.sql feat(knowledge-base): implement per-user pinning for knowledge bases 2026-05-18 17:28:58 +08:00
000050_user_kb_pins.up.sql feat(knowledge-base): implement per-user pinning for knowledge bases 2026-05-18 17:28:58 +08:00
000051_custom_agents_creator_backfill.down.sql feat(i18n): update localization files for improved clarity and consistency 2026-05-18 17:28:58 +08:00
000051_custom_agents_creator_backfill.up.sql feat(i18n): update localization files for improved clarity and consistency 2026-05-18 17:28:58 +08:00
000052_models_managed_by.down.sql feat(builtin-models): add managed_by column to models table 2026-05-26 11:37:03 +08:00
000052_models_managed_by.up.sql feat(builtin-models): add managed_by column to models table 2026-05-26 11:37:03 +08:00
000053_system_admin_and_settings.down.sql refactor(migrations): remove deprecated user system admin migration files and introduce new system settings migration 2026-05-26 21:13:56 +08:00
000053_system_admin_and_settings.up.sql refactor(migrations): remove deprecated user system admin migration files and introduce new system settings migration 2026-05-26 21:13:56 +08:00
000054_invitation_tokens.down.sql feat(rbac): add multi-use share-link invitations for invite_only mode (#1504) 2026-05-27 19:42:01 +08:00
000054_invitation_tokens.up.sql feat(rbac): add multi-use share-link invitations for invite_only mode (#1504) 2026-05-27 19:42:01 +08:00
000055_knowledge_processing_spans.down.sql feat(migrations): add knowledge_processing_spans table and rollback script 2026-05-28 15:14:45 +08:00
000055_knowledge_processing_spans.up.sql feat(migrations): add knowledge_processing_spans table and rollback script 2026-05-28 15:14:45 +08:00
000056_knowledge_pending_subtasks.down.sql feat: add knowledge parse cancellation with finalizing post-process state 2026-05-28 20:16:02 +08:00
000056_knowledge_pending_subtasks.up.sql feat: add knowledge parse cancellation with finalizing post-process state 2026-05-28 20:16:02 +08:00
000056_models_display_name.down.sql fix: add model display name 2026-05-28 19:22:17 +08:00
000056_models_display_name.up.sql fix: add model display name 2026-05-28 19:22:17 +08:00