WeKnora/internal/application/service
wizardchen 3f9b09e306 fix(pipeline): make data-analysis stage opt-in per agent (#1244)
The legacy in-pipeline DuckDB SQL data-analysis stage used to run on every
quick-answer RAG request whose retrieved chunks included a CSV/Excel file,
adding one extra LLM round-trip (~3s) to generate a SQL query that most
plain Q&A users never wanted. There was no way to disable it.

Introduce a per-agent DataAnalysisEnabled flag (default off), wire it
through PipelineRequest, and gate the DATA_ANALYSIS stage on it. Surface
the toggle in the agent editor for quick-answer agents with at least one
knowledge base attached.

Refs: https://github.com/Tencent/WeKnora/issues/1244
2026-05-12 19:39:35 +08:00
..
chat_pipeline fix(graph): recover JSON from malformed/truncated fences in extract_entity 2026-05-07 15:40:19 +08:00
file feat(ks3): add KS3 object storage support 2026-05-05 00:22:27 +08:00
memory feat: implement preview of user level memory graph 2026-02-25 14:29:54 +08:00
metric fix: resolve failing test and build errors 2026-04-22 21:17:29 +08:00
retriever feat(retriever): exponential backoff + safety truncation for embeddings 2026-05-06 17:17:07 +08:00
agent_history.go fix(agent): replay attachments in multi-turn history 2026-05-12 14:46:57 +08:00
agent_history_test.go fix(agent): replay attachments in multi-turn history 2026-05-12 14:46:57 +08:00
agent_service.go feat(knowledge-base): document list filters and explicit batch-management UX 2026-05-12 18:29:47 +08:00
agent_share.go fix(agent-share): don't require rerank model for wiki-only agents 2026-04-24 10:21:05 +08:00
chunk.go feat: enhance document loading and state management in doc-content component 2026-03-02 21:21:49 +08:00
custom_agent.go fix(agent): exclude wiki-only KBs from quick-answer (RAG) mode 2026-05-12 16:27:28 +08:00
dataset.go init commit 2025-08-05 15:08:07 +08:00
datasource_service.go feat(observability): extend Langfuse tracing across asynq pipeline 2026-04-24 13:16:47 +08:00
evaluation.go feat(knowledge): add channel support to knowledge entries and related components 2026-03-25 22:08:29 +08:00
extract.go feat: correct data analysis errors using fileService. 2026-04-27 13:18:08 +08:00
graph.go feat(graph): enhance graph extraction prompt rendering and language context handling 2026-03-25 22:08:29 +08:00
image_multimodal.go fix(multimodal): unblock processing when provider:// image read fails 2026-05-12 17:40:04 +08:00
kbshare.go feat(api): integrate per-organization resource counts into organization listing 2026-02-09 11:40:25 +08:00
knowledge.go feat(knowledge-base): document list filters and explicit batch-management UX 2026-05-12 18:29:47 +08:00
knowledge_clone_move.go refactor(knowledge): split 9.8k-line service file by responsibility 2026-05-06 17:40:10 +08:00
knowledge_create.go refactor(knowledge): split 9.8k-line service file by responsibility 2026-05-06 17:40:10 +08:00
knowledge_delete.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
knowledge_faq.go refactor(knowledge): split 9.8k-line service file by responsibility 2026-05-06 17:40:10 +08:00
knowledge_faq_import.go refactor(knowledge): split 9.8k-line service file by responsibility 2026-05-06 17:40:10 +08:00
knowledge_image_cleanup_test.go fix(knowledge): delete extracted images from storage when knowledge is removed 2026-04-02 12:04:04 +08:00
knowledge_manual_test.go fix: sanitize tab chars and avoid double .md extension in manual download filename 2026-03-16 18:18:23 +08:00
knowledge_post_process.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
knowledge_process.go refactor(knowledge): split 9.8k-line service file by responsibility 2026-05-06 17:40:10 +08:00
knowledge_shared_access_test.go fix: include shared knowledge in knowledge search 2026-05-05 01:04:49 +08:00
knowledge_storage_config_test.go fix(storage): merge tos/s3/oss/ks3 tenant configs in buildStorageConfig (#1117) 2026-05-06 17:09:42 +08:00
knowledge_summary_test.go test(ingest): cover insufficient-content gate; allow threshold override 2026-05-06 11:23:32 +08:00
knowledge_util.go refactor(knowledge): split 9.8k-line service file by responsibility 2026-05-06 17:40:10 +08:00
knowledgebase.go feat(observability): extend Langfuse tracing across asynq pipeline 2026-04-24 13:16:47 +08:00
knowledgebase_search.go feat(retrieval): configurable RRF parameters per tenant 2026-05-06 17:17:07 +08:00
knowledgebase_search_faq.go feat: enhance input field and agent stream display with improved handling and sanitization 2026-03-16 02:26:47 +08:00
knowledgebase_search_fusion.go feat(retrieval): configurable RRF parameters per tenant 2026-05-06 17:17:07 +08:00
knowledgebase_search_results.go feat(logger): implement LLM debug logging functionality 2026-04-16 23:15:02 +08:00
knowledgebase_search_shared.go feat: enhance input field and agent stream display with improved handling and sanitization 2026-03-16 02:26:47 +08:00
mcp_service.go feat: add built-in MCP service support 2026-03-05 21:57:01 +08:00
mcp_tool_approval_service.go feat(agent): human-in-the-loop approval for MCP tool calls (#1173) 2026-05-10 22:57:12 +08:00
message.go feat(prompt): enhance context and intent classification in templates 2026-03-26 18:57:12 +08:00
metric_hook.go refactor: Clean up code by removing unnecessary blank lines and improving comments for better readability and maintainability 2025-11-28 16:10:37 +08:00
model.go feat(models): support custom HTTP headers across all remote model calls 2026-04-23 23:36:40 +08:00
ocr_sanitizer.go feat: use language-neutral English prompts for multilingual LLM responses 2026-03-06 21:08:12 +08:00
ocr_sanitizer_test.go feat: introduce WeKnora Lite edition with lightweight configuration and deployment 2026-03-02 21:21:49 +08:00
organization.go feat(i18n): update localization for knowledge bases and organization features 2026-02-09 11:40:25 +08:00
session.go refactor(agent): rebuild multi-turn history from DB, drop llmcontext layer 2026-05-12 12:26:24 +08:00
session_agent_qa.go fix(agent): relax rerank model requirement for custom agents 2026-05-12 16:57:24 +08:00
session_knowledge_qa.go fix(pipeline): make data-analysis stage opt-in per agent (#1244) 2026-05-12 19:39:35 +08:00
session_knowledge_qa_test.go fix(chat): include history in model fallback 2026-05-04 23:24:21 +08:00
session_qa_helpers.go fix(pipeline): make data-analysis stage opt-in per agent (#1244) 2026-05-12 19:39:35 +08:00
skill_service.go feat: adapt preloaded skills to smart-reasoning agent 2026-02-04 20:08:49 +08:00
slug_fuzzy.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
slug_fuzzy_test.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
tag.go feat(observability): extend Langfuse tracing across asynq pipeline 2026-04-24 13:16:47 +08:00
tenant.go fix(tenant): return plaintext API key after create/reset 2026-04-29 19:46:21 +08:00
user.go simplify oidc callback result payload 2026-04-01 20:47:18 +08:00
vectorstore.go feat: add Tencent VectorDB retriever backend 2026-05-09 13:14:01 +08:00
vectorstore_healthcheck.go feat: add Tencent VectorDB retriever backend 2026-05-09 13:14:01 +08:00
vectorstore_test.go feat(retriever): 接入 Apache Doris 4.1 作为向量数据库 2026-05-08 21:59:35 +08:00
web_search.go feat: search engine adds proxy addresses and adds SSRF verification. 2026-04-17 16:38:02 +08:00
web_search_provider.go fix(web-search): address review fixups for SearXNG provider 2026-05-11 16:53:47 +08:00
web_search_state.go fix: update environment configuration for debugging and logging, enhance SQLite FTS5 table management, and improve API base URL handling 2026-04-11 18:51:16 +08:00
weknoracloud.go refactor(weknoracloud): update credential handling and configuration 2026-04-15 11:37:24 +08:00
wiki_ingest.go fix(wiki-ingest,asynqdl): review fixups for PR #1241 2026-05-10 00:01:06 +08:00
wiki_ingest_batch.go fix(wiki-ingest,asynqdl): review fixups for PR #1241 2026-05-10 00:01:06 +08:00
wiki_ingest_cite.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
wiki_ingest_cite_test.go feat: Enhance wiki extraction and configuration options 2026-04-22 21:18:32 +08:00
wiki_ingest_dedup.go feat: Enhance wiki extraction and configuration options 2026-04-22 21:18:32 +08:00
wiki_ingest_dedup_test.go feat: Enhance wiki extraction and configuration options 2026-04-22 21:18:32 +08:00
wiki_ingest_retry_test.go perf(wiki): move ingest log to event table and index to on-demand API 2026-05-08 21:48:52 +08:00
wiki_ingest_test.go fix(ingest): QA follow-up — preserve OCR text and close remaining filename leaks 2026-05-06 11:23:32 +08:00
wiki_linkify.go feat: Refactor cross-link injection logic for wiki pages 2026-04-22 21:18:09 +08:00
wiki_linkify_test.go feat: Refactor cross-link injection logic for wiki pages 2026-04-22 21:18:09 +08:00
wiki_lint.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
wiki_log_entry.go perf(wiki): move ingest log to event table and index to on-demand API 2026-05-08 21:48:52 +08:00
wiki_page.go perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter 2026-05-10 00:01:06 +08:00
wiki_page_test.go perf(wiki): subgraph API and interactive exploration for large graphs 2026-05-08 13:32:54 +08:00