WeKnora/internal/models
Martín Alcalá Rubí 83da0903b6 fix(rerank): stop sending vLLM truncate_prompt_tokens by default
The generic OpenAI-compatible reranker hardcoded `truncate_prompt_tokens:
511` on every /rerank request. Providers that honor this vLLM-specific
field (e.g. SiliconFlow serving Qwen3-Reranker) keep only the last 511
tokens of the templated rerank prompt, cutting the instruction and query
off any document longer than ~511 tokens. Every relevance score then
collapses to near zero (~0.0067 instead of ~0.998), falls below the score
threshold, and knowledge-base search returns empty results. A direct curl
without the field returns the correct scores.

WeKnora already removed this exact hardcoded parameter from the embedding
path in #960; the rerank path was never fixed.

- Add `omitempty` to `truncate_prompt_tokens` and `additional_data` so
  empty optional fields are not sent.
- Make truncation opt-in via `extra_config.truncate_prompt_tokens`
  (validated > 0 at construction), preserving an escape hatch for
  self-hosted vLLM deployments with small-context rerankers.
- Add regression tests reproducing the collapsed-score symptom and the
  opt-in path.

Fixes #2143
2026-07-21 12:12:56 +08:00
..
asr fix(security): harden model client redirects 2026-07-02 19:15:46 +08:00
chat feat(resource): add stable resource registry with LLM-context alias compaction 2026-07-15 17:02:43 +08:00
embedding feat(system): expose model concurrency stats on runtime queues dashboard 2026-07-11 07:17:28 +08:00
limiter feat(system): expose model concurrency stats on runtime queues dashboard 2026-07-11 07:17:28 +08:00
provider fix(rerank): batch Volcengine rerank requests over API document limit 2026-07-20 20:07:21 +08:00
rerank fix(rerank): stop sending vLLM truncate_prompt_tokens by default 2026-07-21 12:12:56 +08:00
utils feat: integrate ChatBot provider and docreader 2026-04-13 17:47:24 +08:00
vlm feat(system): expose model concurrency stats on runtime queues dashboard 2026-07-11 07:17:28 +08:00