Provider prefix caches only hit when system/tools stay byte-stable and cache routing is actually on the request. Stop rewriting those prefixes mid-turn, attach prompt_cache_key / cache_control, and log hit rate instead of showing it in the conversation tree. |
||
|---|---|---|
| .. | ||
| asr | ||
| chat | ||
| embedding | ||
| limiter | ||
| provider | ||
| rerank | ||
| utils | ||
| vlm | ||