WeKnora/internal/models/chat
young1lin 29820e4cac docs(chat): clarify cached-token semantics for explicit-cache providers
`cached_tokens` is reported by every OpenAI-compatible provider that
supports prompt caching, but how it becomes non-zero differs by mode:

- Implicit caching (OpenAI, Azure OpenAI, DeepSeek, …) populates the
  field automatically whenever a prompt prefix matches a previous
  request within the provider's cache TTL. No client-side opt-in.

- Explicit caching (Qwen on Aliyun, Anthropic Claude, …) only
  populates the field after the caller attaches `cache_control:
  {"type": "ephemeral"}` to the relevant message / content block.
  Until that opt-in is applied upstream of the request, the field
  stays zero even when the prefix is otherwise byte-stable.

Without this distinction documented, the previous commit reads as if
`TokenUsage.CachedTokens` will show non-zero values for Qwen / Claude
once this PR lands — which is not the case. The plumbing here is a
prerequisite (stable prefix via sorted tools) and a meter (visibility
of the field), but the explicit-cache opt-in itself is out of scope
and lives elsewhere.

Document this on `TokenUsage.CachedTokens` and the `cachedTokens`
helper so callers do not mistake observability for activation.
2026-05-25 16:47:14 +08:00
..
anthropic.go fix: support Anthropic gateway streaming 2026-05-13 00:14:23 +08:00
anthropic_test.go fix: support Anthropic gateway streaming 2026-05-13 00:14:23 +08:00
chat.go fix(agent): pass reasoning_content back to providers that require it 2026-05-13 12:29:10 +08:00
chat_provider_spec.go refactor: Simplify WeKnoraCloud request customization and message conversion 2026-04-23 17:52:43 +08:00
config_from_model_test.go feat(models): support custom HTTP headers across all remote model calls 2026-04-23 23:36:40 +08:00
image_resolve.go feat: enhance multimodal support and image handling 2026-03-16 02:26:47 +08:00
json_field_extractor.go feat: implement final_answer tool and enhance agent duration tracking 2026-03-09 11:46:56 +08:00
json_field_extractor_test.go feat: implement final_answer tool and enhance agent duration tracking 2026-03-09 11:46:56 +08:00
langfuse_wrapper.go feat(observability): integrate Langfuse for LLM token tracking and tracing 2026-04-24 10:29:19 +08:00
llm_debug.go feat(logger): implement LLM debug logging functionality 2026-04-16 23:15:02 +08:00
llm_debug_wrapper.go feat(logger): implement LLM debug logging functionality 2026-04-16 23:15:02 +08:00
ollama.go fix(ollama): adapt to ToolCallFunctionArguments struct API 2026-05-09 13:58:55 +08:00
remote_api.go docs(chat): clarify cached-token semantics for explicit-cache providers 2026-05-25 16:47:14 +08:00
remote_api_test.go feat(chat): surface cached prompt tokens from upstream usage 2026-05-25 16:47:14 +08:00
sse_reader.go fix: ReadEvent incompatible sse data format 2026-04-20 22:29:38 +08:00
timeout_test.go fix(chat): LLM 兜底超时仅在上层无 deadline 时生效,并修复 ChatStream raw HTTP 路径的 cancel 泄漏 2026-05-10 00:16:12 +08:00