WeKnora/internal/utils
wizardchen dc634e863a feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling
Two encryption gaps and one fail-loud regression, fixed together because
they are inseparable in practice.

At-rest encryption gaps
-----------------------
Before this change, only Tenant.APIKey, Tenant.Credentials.WeKnoraCloud.AppSecret,
Model.Parameters.APIKey/AppSecret, and WebSearchProviderParameters.APIKey
were encrypted with SYSTEM_AES_KEY. MCP and DataSource were missed:

- MCPAuthConfig.Value/Scan went through bare json.Marshal/Unmarshal, so
  api_key and token were written to mcp_services.auth_config (json) in
  plaintext regardless of whether SYSTEM_AES_KEY was set.
- DataSource.Config (types.JSON) is a byte passthrough, so connector
  secrets (Notion token, GitHub PAT, Feishu app_secret, ...) inside the
  Credentials map were stored in plaintext too.

Now both follow the ModelParameters / WebSearchProviderParameters pattern.
Encryption operates on a local copy to avoid mutating the caller's
in-memory struct (otherwise the next read of the same *MCPAuthConfig
would surface ciphertext as if it were the API key). For DataSource the
ToJSON ↔ ParseConfig pair is the only application boundary that
materializes credentials, so encrypting there is sufficient and avoids
having to fork the generic types.JSON.

Backward compatibility
----------------------
DecryptStoredSecret returns rows without the enc:v1: prefix verbatim, so
historical plaintext records keep working. Next write re-saves them
encrypted. No migration step.

Graceful Scan-time degradation
------------------------------
The strict "fail loud on key rotation" behaviour of DecryptStoredSecret
turned into a footgun once it ran from GORM Scan: a single broken row
caused ListModels / ListMCPServices / ListDataSources to return zero
results, hiding all other rows and giving the operator no way to find
which credential needed re-entering.

New utils.DecryptStoredSecretLenient: same logic, but on failure returns
("", false). The strict variant is kept for hot paths that are about to
actually USE the credential (so a rotated key surfaces loudly there
rather than silently degrading to "missing api key").

All Scan paths switch to lenient + per-failure log line:
  ModelParameters.Scan, WebSearchProviderParameters.Scan,
  MCPAuthConfig.Scan, DataSource.ParseConfig, CredentialsConfig.Scan.

On lenient failure the field blanks out (never ciphertext), the row
still loads, the DTO reports configured=false so the UI shows "not
configured", and any upstream call attempted with the empty value fails
with a clear "missing api key" rather than sending ciphertext as the key.

Tenant.AfterFind (Tenant.APIKey) intentionally stays strict — it is the
primary HTTP auth credential and a silent empty value would change auth
semantics ambiguously.
2026-05-17 15:27:52 +08:00
..
crypto.go feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling 2026-05-17 15:27:52 +08:00
crypto_test.go fix(crypto): fail loudly when encrypted DB fields cannot be decrypted 2026-04-30 15:17:56 +08:00
debug.go feat: 使用增强的任务ID生成器替换UUID 2026-01-14 17:09:19 +08:00
extraheaders.go feat(models): support custom HTTP headers across all remote model calls 2026-04-23 23:36:40 +08:00
extraheaders_test.go feat(models): support custom HTTP headers across all remote model calls 2026-04-23 23:36:40 +08:00
filesize.go feat: 添加可配置的文件上传大小限制 2025-12-30 14:35:10 +08:00
fileutil.go fix(storage): auto-detect content-type to resolve image rendering issues 2026-04-20 21:59:54 +08:00
httputil.go feat: enhance document loading and state management in doc-content component 2026-03-02 21:21:49 +08:00
inject.go fix: improve tenant access token logging and update test cases 2026-04-22 21:17:30 +08:00
inject_test.go fix: improve tenant access token logging and update test cases 2026-04-22 21:17:30 +08:00
json.go feat: Add DataSchema tool for retrieving schema information from CSV and Excel files 2025-12-29 20:03:51 +08:00
log_sanitize.go feat: implement image upload and multimodal support 2026-03-12 10:32:26 +08:00
presign.go fix(im): revert tenant-from-context for presigned URL 2026-04-30 11:40:30 +08:00
presign_test.go fix(im): rewrite private storage URLs to HTTP in IM channel replies 2026-04-30 11:40:30 +08:00
security.go fix(web-search): address review fixups for SearXNG provider 2026-05-11 16:53:47 +08:00
security_test.go refactor(security_test): standardize struct field formatting in SSRF whitelist tests 2026-05-11 16:53:47 +08:00
taskid.go feat: 使用增强的任务ID生成器替换UUID 2026-01-14 17:09:19 +08:00