WeKnora/internal/models/rerank
Martín Alcalá Rubí 83da0903b6 fix(rerank): stop sending vLLM truncate_prompt_tokens by default
The generic OpenAI-compatible reranker hardcoded `truncate_prompt_tokens:
511` on every /rerank request. Providers that honor this vLLM-specific
field (e.g. SiliconFlow serving Qwen3-Reranker) keep only the last 511
tokens of the templated rerank prompt, cutting the instruction and query
off any document longer than ~511 tokens. Every relevance score then
collapses to near zero (~0.0067 instead of ~0.998), falls below the score
threshold, and knowledge-base search returns empty results. A direct curl
without the field returns the correct scores.

WeKnora already removed this exact hardcoded parameter from the embedding
path in #960; the rerank path was never fixed.

- Add `omitempty` to `truncate_prompt_tokens` and `additional_data` so
  empty optional fields are not sent.
- Make truncation opt-in via `extra_config.truncate_prompt_tokens`
  (validated > 0 at construction), preserving an escape hatch for
  self-hosted vLLM deployments with small-context rerankers.
- Add regression tests reproducing the collapsed-score symptom and the
  opt-in path.

Fixes #2143
2026-07-21 12:12:56 +08:00
..
aliyun_reranker.go fix(security): harden model client redirects 2026-07-02 19:15:46 +08:00
config_from_model_test.go feat(models): support custom HTTP headers across all remote model calls 2026-04-23 23:36:40 +08:00
jina_reranker.go fix(security): harden model client redirects 2026-07-02 19:15:46 +08:00
langfuse_wrapper.go feat(observability): enhance Langfuse retrieve and rerank tracing 2026-06-16 01:09:23 +08:00
lkeap_reranker.go feat(lkeap): integrate Tencent Cloud LKEAP Rerank support 2026-06-04 17:22:18 +08:00
lkeap_reranker_test.go feat(lkeap): integrate Tencent Cloud LKEAP Rerank support 2026-06-04 17:22:18 +08:00
llm_debug.go feat(logger): implement LLM debug logging functionality 2026-04-16 23:15:02 +08:00
logging.go feat: enhance multimodal support and image handling 2026-03-16 02:26:47 +08:00
nvidia_reranker.go fix(security): harden model client redirects 2026-07-02 19:15:46 +08:00
remote_api.go fix(rerank): stop sending vLLM truncate_prompt_tokens by default 2026-07-21 12:12:56 +08:00
remote_api_test.go fix(rerank): stop sending vLLM truncate_prompt_tokens by default 2026-07-21 12:12:56 +08:00
reranker.go feat: support Volcengine rerank 2026-07-20 19:58:48 +08:00
reranker_test.go feat: Compatibility with rerank API output fields 2025-09-08 11:20:40 +08:00
transport.go fix(security): harden model client redirects 2026-07-02 19:15:46 +08:00
transport_security_test.go fix(security): harden model client redirects 2026-07-02 19:15:46 +08:00
volcengine_reranker.go fix(rerank): batch Volcengine rerank requests over API document limit 2026-07-20 20:07:21 +08:00
volcengine_reranker_test.go fix(rerank): batch Volcengine rerank requests over API document limit 2026-07-20 20:07:21 +08:00
weknoracloud.go fix(security): harden model client redirects 2026-07-02 19:15:46 +08:00
zhipu_reranker.go fix(security): harden model client redirects 2026-07-02 19:15:46 +08:00