WeKnora/internal/application/service
voidborne-d 46cab52fb0 fix(storage): merge tos/s3/oss/ks3 tenant configs in buildStorageConfig (#1117)
knowledgeService.buildStorageConfig had hardcoded switches in two places that
only handled local/minio/cos. Knowledge bases configured for tos, s3, oss, or
ks3 reached the docreader with a DocParserStorageConfig containing only the
Provider tag — bucket, endpoint, and credentials were silently dropped, and
extraction stalled or fetched from the wrong location.

- Tenant-merge switch: add tos/s3/oss/ks3 cases mapping
  StorageEngineConfig.{TOS,S3,OSS,KS3} fields onto DocParserStorageConfig.
  Same shape as the existing minio/cos branches; AccessKey -> AccessKeyID,
  SecretKey -> SecretAccessKey, no AppID.
- Legacy compat switch: list tos/s3/oss/ks3 alongside local with hasKBFull=false
  so fall-through to the tenant-merge path is intentional rather than an
  unrecognised provider sliding past the switch. The legacy `cos_config` column
  predates these providers and they should never resolve via that path.
- Doc drift: types/custom_agent.go ImageStorageProvider comment and
  handler/system.go StorageEngineStatusItem.Name comment now list all 7
  providers (matched against types.StorageEngineConfig and ParseProviderScheme).

Tests (knowledge_storage_config_test.go, +268 LOC, 17 cases):
- TestBuildStorageConfig_TenantMergeAllProviders pins per-provider field
  population for all 7 providers; the tos/s3/oss/ks3 sub-tests are the
  regression guard for #1117.
- TestBuildStorageConfig_LegacyPathOnlyForCOSAndMinIO pins that legacy
  StorageConfig data is only consumed for cos/minio and skipped for the
  others (intentional fall-through).
- TestBuildStorageConfig_NoTenantFallsThroughToEmpty pins the no-tenant case.

Out of scope (called out in the linked issue):
- The systemic refactor (single source-of-truth provider registry) — left as
  a follow-up; this PR keeps changes local to the buggy code path.
- initialization.go validateStorageEngineConfig multimodal validation gap —
  the InitializationRequest struct does not carry tos/s3/oss/ks3 fields, so
  fixing it requires API surface additions; can ship separately.

Closes #1117 (the buildStorageConfig switches; remaining items deferred).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 17:09:42 +08:00
..
chat_pipeline fix(memory): dedupe episode write on duplicate streaming Done 2026-05-04 23:49:29 +08:00
file feat(ks3): add KS3 object storage support 2026-05-05 00:22:27 +08:00
llmcontext feat(prompt): enhance context and intent classification in templates 2026-03-26 18:57:12 +08:00
memory feat: implement preview of user level memory graph 2026-02-25 14:29:54 +08:00
metric fix: resolve failing test and build errors 2026-04-22 21:17:29 +08:00
retriever feat: add registry dual-map and engine factory for VectorStore 2026-04-17 00:15:37 +08:00
agent_service.go feat: correct data analysis errors using fileService. 2026-04-27 13:18:08 +08:00
agent_share.go fix(agent-share): don't require rerank model for wiki-only agents 2026-04-24 10:21:05 +08:00
chunk.go feat: enhance document loading and state management in doc-content component 2026-03-02 21:21:49 +08:00
custom_agent.go feat: Enhance suggested questions functionality with wiki integration 2026-04-22 21:18:20 +08:00
dataset.go init commit 2025-08-05 15:08:07 +08:00
datasource_service.go feat(observability): extend Langfuse tracing across asynq pipeline 2026-04-24 13:16:47 +08:00
evaluation.go feat(knowledge): add channel support to knowledge entries and related components 2026-03-25 22:08:29 +08:00
extract.go feat: correct data analysis errors using fileService. 2026-04-27 13:18:08 +08:00
graph.go feat(graph): enhance graph extraction prompt rendering and language context handling 2026-03-25 22:08:29 +08:00
image_multimodal.go feat(observability): extend Langfuse tracing across asynq pipeline 2026-04-24 13:16:47 +08:00
kbshare.go feat(api): integrate per-organization resource counts into organization listing 2026-02-09 11:40:25 +08:00
knowledge.go fix(storage): merge tos/s3/oss/ks3 tenant configs in buildStorageConfig (#1117) 2026-05-06 17:09:42 +08:00
knowledge_image_cleanup_test.go fix(knowledge): delete extracted images from storage when knowledge is removed 2026-04-02 12:04:04 +08:00
knowledge_manual_test.go fix: sanitize tab chars and avoid double .md extension in manual download filename 2026-03-16 18:18:23 +08:00
knowledge_post_process.go feat(observability): extend Langfuse tracing across asynq pipeline 2026-04-24 13:16:47 +08:00
knowledge_shared_access_test.go fix: include shared knowledge in knowledge search 2026-05-05 01:04:49 +08:00
knowledge_storage_config_test.go fix(storage): merge tos/s3/oss/ks3 tenant configs in buildStorageConfig (#1117) 2026-05-06 17:09:42 +08:00
knowledge_summary_test.go test(ingest): cover insufficient-content gate; allow threshold override 2026-05-06 11:23:32 +08:00
knowledgebase.go feat(observability): extend Langfuse tracing across asynq pipeline 2026-04-24 13:16:47 +08:00
knowledgebase_search.go feat: Update agent type presets and prompt templates for improved knowledge base handling 2026-04-22 21:18:34 +08:00
knowledgebase_search_faq.go feat: enhance input field and agent stream display with improved handling and sanitization 2026-03-16 02:26:47 +08:00
knowledgebase_search_fusion.go feat(repository): update keyword retrieval logic to use ParadeDB's matching operator 2026-04-15 20:16:02 +08:00
knowledgebase_search_results.go feat(logger): implement LLM debug logging functionality 2026-04-16 23:15:02 +08:00
knowledgebase_search_shared.go feat: enhance input field and agent stream display with improved handling and sanitization 2026-03-16 02:26:47 +08:00
mcp_service.go feat: add built-in MCP service support 2026-03-05 21:57:01 +08:00
message.go feat(prompt): enhance context and intent classification in templates 2026-03-26 18:57:12 +08:00
metric_hook.go refactor: Clean up code by removing unnecessary blank lines and improving comments for better readability and maintainability 2025-11-28 16:10:37 +08:00
model.go feat(models): support custom HTTP headers across all remote model calls 2026-04-23 23:36:40 +08:00
ocr_sanitizer.go feat: use language-neutral English prompts for multilingual LLM responses 2026-03-06 21:08:12 +08:00
ocr_sanitizer_test.go feat: introduce WeKnora Lite edition with lightweight configuration and deployment 2026-03-02 21:21:49 +08:00
organization.go feat(i18n): update localization for knowledge bases and organization features 2026-02-09 11:40:25 +08:00
session.go fix(sessions): review fixes for keyword search / pinning / IM titles 2026-04-30 16:23:00 +08:00
session_agent_qa.go feat: Introduce agent type presets for smart-reasoning mode 2026-04-22 21:18:20 +08:00
session_knowledge_qa.go fix(chat): include history in model fallback 2026-05-04 23:24:21 +08:00
session_knowledge_qa_test.go fix(chat): include history in model fallback 2026-05-04 23:24:21 +08:00
session_qa_helpers.go fix: update environment configuration for debugging and logging, enhance SQLite FTS5 table management, and improve API base URL handling 2026-04-11 18:51:16 +08:00
skill_service.go feat: adapt preloaded skills to smart-reasoning agent 2026-02-04 20:08:49 +08:00
tag.go feat(observability): extend Langfuse tracing across asynq pipeline 2026-04-24 13:16:47 +08:00
tenant.go fix(tenant): return plaintext API key after create/reset 2026-04-29 19:46:21 +08:00
user.go simplify oidc callback result payload 2026-04-01 20:47:18 +08:00
vectorstore.go feat: pass IndexConfig to repositories with scalability options 2026-04-17 16:52:24 +08:00
vectorstore_healthcheck.go feat: add VectorStore service layer with types and connection testing 2026-04-13 22:57:17 +08:00
vectorstore_test.go feat: pass IndexConfig to repositories with scalability options 2026-04-17 16:52:24 +08:00
web_search.go feat: search engine adds proxy addresses and adds SSRF verification. 2026-04-17 16:38:02 +08:00
web_search_provider.go feat: search engine adds proxy addresses and adds SSRF verification. 2026-04-17 16:38:02 +08:00
web_search_state.go fix: update environment configuration for debugging and logging, enhance SQLite FTS5 table management, and improve API base URL handling 2026-04-11 18:51:16 +08:00
weknoracloud.go refactor(weknoracloud): update credential handling and configuration 2026-04-15 11:37:24 +08:00
wiki_ingest.go test(ingest): cover insufficient-content gate; allow threshold override 2026-05-06 11:23:32 +08:00
wiki_ingest_batch.go chore(comment): update slug source to knowledge ID 2026-05-06 11:23:32 +08:00
wiki_ingest_cite.go fix(ingest): QA follow-up — preserve OCR text and close remaining filename leaks 2026-05-06 11:23:32 +08:00
wiki_ingest_cite_test.go feat: Enhance wiki extraction and configuration options 2026-04-22 21:18:32 +08:00
wiki_ingest_dedup.go feat: Enhance wiki extraction and configuration options 2026-04-22 21:18:32 +08:00
wiki_ingest_dedup_test.go feat: Enhance wiki extraction and configuration options 2026-04-22 21:18:32 +08:00
wiki_ingest_test.go fix(ingest): QA follow-up — preserve OCR text and close remaining filename leaks 2026-05-06 11:23:32 +08:00
wiki_linkify.go feat: Refactor cross-link injection logic for wiki pages 2026-04-22 21:18:09 +08:00
wiki_linkify_test.go feat: Refactor cross-link injection logic for wiki pages 2026-04-22 21:18:09 +08:00
wiki_lint.go feat: Implement auto-linked content updates in wiki page management 2026-04-23 11:16:47 +08:00
wiki_page.go feat: Implement auto-linked content updates in wiki page management 2026-04-23 11:16:47 +08:00
wiki_page_test.go refactor: Update wiki link syntax and rendering in WikiBrowser component 2026-04-22 21:17:26 +08:00