WeKnora/migrations/sqlite
lyingbug 1bf717a54d
feat(chat): open cited documents at the cited passage (#3700)
* feat(chat): open cited documents at the cited passage

Clicking a citation used to show only the parsed chunk. It now opens the
original file in the references panel, jumps to the cited place and
highlights it, narrowed to the sentence the citation supports.

Parsing records where each chunk sits in the original file
(chunks.source_locators): PDF page and paragraph boxes from the docreader,
MinerU and PaddleOCR-VL layouts; Word paragraphs, slides, sheet rows and
EPUB chapters aligned in Go against any parser engine's Markdown; text
offsets for Markdown/TXT; segment times for audio. Pipeline text rewrites
are followed with a line-diff remapper. Scanned PDF pages get their OCR
text as quotes at search time so a sentence can pick its page.

The frontend renders PDFs with pdf.js and every other previewable type
with the existing preview, then highlights with the CSS Custom Highlight
API in a highlighter yellow that stays visible against the brand green.

* fix(docreader): serialize source blocks outside main so tests import it

test_docx_tables stubs docreader.parser in sys.modules, so importing
docreader.main from a later test failed in the full unittest discover run.
The wire conversion now lives in docreader.source_wire, which needs only
the proto.

* fix(chat): tighten source locators from review

- Remap rewritten-line-for-line stretches by line so scanned PDF pages
  no longer drift onto the wrong page.
- Keep the source preview mounted for citations in the file already open.
- Wait for in-flight PDF page renders before looking for the text layer.
- Count image rune offsets in one pass; stream xlsx rows up to maxUnits.
- Start text locators after a UTF-8 BOM, as viewers decode without it.
- Only re-query image children for scanned-page results.
- Seek audio citations without autoplay and drop stale seek listeners.
- Fix the SQLite migration's cross-reference to versioned/000113.

* fix(migrations): renumber chunk source locators after main's chunk_images

main took versioned 000113 and sqlite 000032 for chunk_images, so the
source_locators column moves to versioned 000114 and sqlite 000033.
2026-09-25 13:26:16 +08:00
..
000000_init.down.sql Remove redundant Wiki operation log 2026-07-28 11:18:33 +08:00
000000_init.up.sql feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
000001_remove_wiki_log.down.sql Remove redundant Wiki operation log 2026-07-28 11:18:33 +08:00
000001_remove_wiki_log.up.sql Remove redundant Wiki operation log 2026-07-28 11:18:33 +08:00
000002_knowledge_folder_path.down.sql feat(knowledge): store folder path for folder uploads and expose folder tree API 2026-08-04 20:45:29 +08:00
000002_knowledge_folder_path.up.sql feat(knowledge): store folder path for folder uploads and expose folder tree API 2026-08-04 20:45:29 +08:00
000003_knowledge_base_auto_tag_config.down.sql feat: 增加自动添加标签的功能 2026-08-07 15:01:01 +08:00
000003_knowledge_base_auto_tag_config.up.sql feat: 增加自动添加标签的功能 2026-08-07 15:01:01 +08:00
000004_memory.down.sql feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
000004_memory.up.sql feat(memory): add forced consolidation feature and related i18n updates 2026-08-13 18:00:02 +08:00
000005_messages_attachments_and_invitation_fields.down.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000005_messages_attachments_and_invitation_fields.up.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000006_task_queue_and_dead_letters.down.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000006_task_queue_and_dead_letters.up.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000007_system_admin_and_settings.down.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000007_system_admin_and_settings.up.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000008_processing_spans_and_pending_subtasks.down.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000008_processing_spans_and_pending_subtasks.up.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000009_embed_channel_memory_flag.down.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000009_embed_channel_memory_flag.up.sql fix(migrations): 补齐 Lite 模式缺失的 SQLite 版本化迁移 2026-08-20 14:35:49 +08:00
000010_knowledge_multi_tags.down.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000010_knowledge_multi_tags.up.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000011_principal_model.down.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000011_principal_model.up.sql fix(migrations): 补齐 SQLite 迁移与 versioned 的索引与数据迁移缺口 2026-08-20 14:35:49 +08:00
000012_message_usage.down.sql feat(agent): report and attribute per-turn LLM token usage 2026-08-24 10:36:19 +08:00
000012_message_usage.up.sql feat(agent): report and attribute per-turn LLM token usage 2026-08-24 10:36:19 +08:00
000013_mcp_tool_enabled.down.sql feat(mcp): add per-tool enable toggle (#2989) 2026-09-05 17:19:20 +08:00
000013_mcp_tool_enabled.up.sql feat(mcp): add per-tool enable toggle (#2989) 2026-09-05 17:19:20 +08:00
000014_browser_authorization.down.sql feat(browser): connect local Chrome with task controls and display modes (#3180) 2026-09-11 17:59:27 +08:00
000014_browser_authorization.up.sql feat(browser): connect local Chrome with task controls and display modes (#3180) 2026-09-11 17:59:27 +08:00
000015_memory_consistency.down.sql fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
000015_memory_consistency.up.sql fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
000016_memory_vector_search.down.sql fix(memory): rank memories by the query, not by importance 2026-09-13 11:43:17 +08:00
000016_memory_vector_search.up.sql fix(memory): rank memories by the query, not by importance 2026-09-13 11:43:17 +08:00
000017_dingtalk_stream_only.down.sql fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
000017_dingtalk_stream_only.up.sql fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
000018_session_fork.down.sql feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
000018_session_fork.up.sql feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
000019_fork_snapshot_lease.down.sql feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
000019_fork_snapshot_lease.up.sql feat(session): fork conversations from a historical user message (#3304) 2026-09-17 14:20:18 +08:00
000020_chunks_index_diet.down.sql perf(chunk): speed up chunk ingestion and batch updates (#3369) 2026-09-17 21:38:15 +08:00
000020_chunks_index_diet.up.sql perf(chunk): speed up chunk ingestion and batch updates (#3369) 2026-09-17 21:38:15 +08:00
000021_knowledge_profiles.down.sql feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
000021_knowledge_profiles.up.sql feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
000022_mcp_endpoints.down.sql feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
000022_mcp_endpoints.up.sql feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
000023_message_artifacts_table.down.sql feat(artifacts): artifacts library page backed by a message_artifacts table (#3384) 2026-09-18 12:10:19 +08:00
000023_message_artifacts_table.up.sql feat(artifacts): artifacts library page backed by a message_artifacts table (#3384) 2026-09-18 12:10:19 +08:00
000024_message_context_checkpoint.down.sql feat(agent): persist compaction checkpoints across turns (#3407) 2026-09-18 20:38:56 +08:00
000024_message_context_checkpoint.up.sql fix(migrations): renumber the context checkpoint migration to 000105 2026-09-18 21:26:26 +08:00
000025_messages_session_created_index.down.sql feat(agent): size agent history by the window and calibrate token estimates (#3413) 2026-09-18 22:51:31 +08:00
000025_messages_session_created_index.up.sql feat(agent): size agent history by the window and calibrate token estimates (#3413) 2026-09-18 22:51:31 +08:00
000026_message_artifacts_deleted_at.down.sql feat(artifacts): let users delete generated files (#3447) 2026-09-20 13:01:46 +08:00
000026_message_artifacts_deleted_at.up.sql feat(artifacts): let users delete generated files (#3447) 2026-09-20 13:01:46 +08:00
000027_session_sandbox_config_tenant.down.sql fix(sandbox): resolve a session's pinned sandbox in the workspace that owns it (#3471) 2026-09-21 15:38:39 +08:00
000027_session_sandbox_config_tenant.up.sql fix(sandbox): resolve a session's pinned sandbox in the workspace that owns it (#3471) 2026-09-21 15:38:39 +08:00
000028_tenant_skills_and_env_vars.down.sql feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) 2026-09-22 19:35:03 +08:00
000028_tenant_skills_and_env_vars.up.sql feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) 2026-09-22 19:35:03 +08:00
000029_session_host_workspace_dir.down.sql feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) 2026-09-22 19:35:03 +08:00
000029_session_host_workspace_dir.up.sql feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) 2026-09-22 19:35:03 +08:00
000030_im_channel_locale.down.sql feat(im): add per-channel reply locale (#3455) 2026-09-23 12:01:01 +08:00
000030_im_channel_locale.up.sql feat(im): add per-channel reply locale (#3455) 2026-09-23 12:01:01 +08:00
000031_model_catalog_config.down.sql feat(models): add system admin model catalog console (#3662) 2026-09-24 16:31:45 +08:00
000031_model_catalog_config.up.sql feat(models): add system admin model catalog console (#3662) 2026-09-24 16:31:45 +08:00
000032_chunk_images.down.sql feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
000032_chunk_images.up.sql feat(gallery): add an image gallery tab with schema-driven filters (#3694) 2026-09-25 12:33:59 +08:00
000033_chunk_source_locators.down.sql feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00
000033_chunk_source_locators.up.sql feat(chat): open cited documents at the cited passage (#3700) 2026-09-25 13:26:16 +08:00