WeKnora/internal
wizardchen dc634e863a feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling
Two encryption gaps and one fail-loud regression, fixed together because
they are inseparable in practice.

At-rest encryption gaps
-----------------------
Before this change, only Tenant.APIKey, Tenant.Credentials.WeKnoraCloud.AppSecret,
Model.Parameters.APIKey/AppSecret, and WebSearchProviderParameters.APIKey
were encrypted with SYSTEM_AES_KEY. MCP and DataSource were missed:

- MCPAuthConfig.Value/Scan went through bare json.Marshal/Unmarshal, so
  api_key and token were written to mcp_services.auth_config (json) in
  plaintext regardless of whether SYSTEM_AES_KEY was set.
- DataSource.Config (types.JSON) is a byte passthrough, so connector
  secrets (Notion token, GitHub PAT, Feishu app_secret, ...) inside the
  Credentials map were stored in plaintext too.

Now both follow the ModelParameters / WebSearchProviderParameters pattern.
Encryption operates on a local copy to avoid mutating the caller's
in-memory struct (otherwise the next read of the same *MCPAuthConfig
would surface ciphertext as if it were the API key). For DataSource the
ToJSON ↔ ParseConfig pair is the only application boundary that
materializes credentials, so encrypting there is sufficient and avoids
having to fork the generic types.JSON.

Backward compatibility
----------------------
DecryptStoredSecret returns rows without the enc:v1: prefix verbatim, so
historical plaintext records keep working. Next write re-saves them
encrypted. No migration step.

Graceful Scan-time degradation
------------------------------
The strict "fail loud on key rotation" behaviour of DecryptStoredSecret
turned into a footgun once it ran from GORM Scan: a single broken row
caused ListModels / ListMCPServices / ListDataSources to return zero
results, hiding all other rows and giving the operator no way to find
which credential needed re-entering.

New utils.DecryptStoredSecretLenient: same logic, but on failure returns
("", false). The strict variant is kept for hot paths that are about to
actually USE the credential (so a rotated key surfaces loudly there
rather than silently degrading to "missing api key").

All Scan paths switch to lenient + per-failure log line:
  ModelParameters.Scan, WebSearchProviderParameters.Scan,
  MCPAuthConfig.Scan, DataSource.ParseConfig, CredentialsConfig.Scan.

On lenient failure the field blanks out (never ciphertext), the row
still loads, the DTO reports configured=false so the UI shows "not
configured", and any upstream call attempted with the empty value fails
with a clear "missing api key" rather than sending ciphertext as the key.

Tenant.AfterFind (Tenant.APIKey) intentionally stays strict — it is the
primary HTTP auth credential and a silent empty value would change auth
semantics ambiguously.
2026-05-17 15:27:52 +08:00
..
agent fix(agent): pass reasoning_content back to providers that require it 2026-05-13 12:29:10 +08:00
application refactor(credentials): introduce /credentials subresource pattern 2026-05-17 15:27:52 +08:00
assets feat(assets): add ASR test audio file and embed it in the application 2026-04-02 21:27:27 +08:00
common feat: Implement deadlock retry mechanism for chunk creation 2026-04-22 21:17:21 +08:00
config 验证完毕 2026-05-15 20:47:01 +08:00
container refactor(credentials): introduce /credentials subresource pattern 2026-05-17 15:27:52 +08:00
database feat(system-info): surface DB migration errors with troubleshooting links 2026-05-14 16:34:50 +08:00
datasource fix(feishu): tolerate partial wiki node listing failures 2026-05-12 17:40:59 +08:00
errors feat: Add agent configuration and cleanup scripts for database migrations 2025-11-05 23:18:44 +08:00
event feat(agent): human-in-the-loop approval for MCP tool calls (#1173) 2026-05-10 22:57:12 +08:00
handler fix(initialization): fall back to stored apiKey when test-connecting an existing model 2026-05-17 15:27:52 +08:00
im fix(agent): exclude wiki-only KBs from quick-answer (RAG) mode 2026-05-12 16:27:28 +08:00
infrastructure fix(docreader/auth): harden gRPC TLS/Token rollout from #1359 2026-05-16 21:45:56 +08:00
logger feat(observability): integrate Langfuse for LLM token tracking and tracing 2026-04-24 10:29:19 +08:00
mcp feat(mcp): implement reconnection logic for MCP tool calls and tool listing 2026-03-31 11:57:15 +08:00
middleware fix(middleware): mask camelCase secret fields in request logs 2026-05-12 19:27:29 +08:00
models fix(agent): pass reasoning_content back to providers that require it 2026-05-13 12:29:10 +08:00
router refactor(credentials): introduce /credentials subresource pattern 2026-05-17 15:27:52 +08:00
runtime init commit 2025-08-05 15:08:07 +08:00
sandbox feat: optimize security and deployment of agent skills 2026-02-04 20:08:49 +08:00
searchutil feat(prompt): enhance question rewriting and intent classification in prompt templates 2026-04-16 23:15:02 +08:00
stream feat(redis): add REDIS_USERNAME support for Redis ACL 2026-02-04 19:38:40 +08:00
tracing feat(observability): extend Langfuse tracing across asynq pipeline 2026-04-24 13:16:47 +08:00
types feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling 2026-05-17 15:27:52 +08:00
utils feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling 2026-05-17 15:27:52 +08:00