Commit Graph

12 Commits (main)

Author SHA1 Message Date
lyingbug 94fae6de43
feat(models): add WeMM embedding presets and Transformers serving (#4238) 2026-10-13 17:22:24 +08:00
kwsYeager e5f9f9deab feat(rerank): add Cohere provider for the v2 rerank API 2026-10-12 11:57:53 +08:00
alexcastgonz 9efe94cc6f
feat(models): add Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 to the catalog (#4158)
These three models were missing from the catalog, so they resolved to the
anthropic defaults: budget thinking (they reject budget_tokens) and
temperature/top_p (they reject non-default sampling values).

seed.json pins their metadata from the models overview and pricing pages:
1M context, 128K output, and base, 5m cache write, cache hit and output
prices. Haiku 5.5 is priced by prompt length, and the catalog has one price
per model, so it carries the prompts-up-to-100k tier. overrides.json gives
all three adaptive thinking with effort (all five levels), with temperature
and top_p off.

Thinking off differs per model. Opus 5.5 can't turn thinking off, so off is
unsupported, like Fable 5.1. Haiku 5.5 accepts "disabled". Sonnet 5.5
rejects "disabled" and turns off up-front thinking with "between_tools". The
new thinking_off_type compat field carries that value. Empty keeps
"disabled", and it is omitempty so the parity hashes of other entries don't
change.


Claude-Session: https://claude.ai/code/session_01BeNHkjF4spsH9d18UiYPiU

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:59:51 +08:00
alexcastgonz 5343ac47b7
fix(models): stop sending temperature to Claude models that reject it (#4155)
Disable temperature and top_p for Claude models that reject non-default sampling values. Regenerate the model catalog and parity baseline, add request-body regression coverage, and clarify the documented sampling restrictions.
2026-10-11 22:44:56 +08:00
lyingbug e372b8e42e
perf(embedding): batch Qwen3-VL message inputs (#4126)
Batch independent Qwen3-VL embedding conversations through an explicit compatibility setting. Preserve singleton requests and serial overrides, validate response ordering, and add wire tests and opt-in live numerical parity checks.

Live verification confirms valid batched responses but does not meet strict numerical parity bounds; document the deployment-side reproducibility limitation.
2026-10-10 16:07:29 +08:00
lyingbug c3f077ed18
fix(models): align configuration and multimodal embedding workflows (#4080)
* fix(models): align configuration defaults and multimodal workflows

* feat(image): enhance image retrieval diagnostics and filtering

- Implemented `retrieveObservedImagePool` to observe engine calls and record image recall decisions without altering results.
- Added detailed diagnostics in `summarizeImagePool` and `summarizeImageCandidateCut` to track image filtering decisions and scores.
- Introduced tests for image pool diagnostics to ensure accurate filtering and decision recording.
- Updated existing functions to utilize new retrieval and summarization methods for improved observability and accuracy in image handling.

* feat(embeddings): enhance image handling with new formats and message support

- Introduced `EmbeddingImageFormat` to manage image input formats, allowing for both object and messages formats.
- Updated `EmbeddingsSettings` to include new fields for `ImageFormat`, `ImagePrompt`, and `TextAsMessages` to support multimodal embedding workflows.
- Enhanced `Client` methods to build request bodies for images in both formats, ensuring compatibility with vLLM's requirements.
- Added tests to validate the new image message handling and ensure correct request formatting for different scenarios.
- Updated documentation to reflect changes in image input handling and configuration options.

* feat(similarity): add embedding similarity mode for enhanced document scoring

- Introduced a new `similarity` option in `ModelDebugOptions` to score documents and an optional image against a query.
- Updated `ModelDebugDrawer.vue` to support a new input mode for similarity, allowing users to select similarity as an input type.
- Enhanced the UI to display similarity results, including ranked candidates based on cosine similarity.
- Added tests to validate the functionality of the new similarity mode in embedding models.
- Updated localization files to include new strings related to similarity mode in multiple languages.

* feat(embeddings): add Qwen3-VL-Embedding support for multimodal input

- Introduced Qwen3-VL-Embedding model with support for both text and image inputs.
- Updated model configuration files to include compatibility settings for the new embedding type.
- Enhanced tests to validate the functionality of Qwen3-VL-Embedding, ensuring correct handling of inputs and outputs.
- Updated documentation to reflect the new model's capabilities and usage instructions.

* test(models): wrap embedding assertions to satisfy lint

* test(opensearch): isolate HTTP transports between parallel tests
2026-10-10 11:06:00 +08:00
lyingbug 65f88f4121
feat: judge image hits by their image in rerank and show them to vision models (#4013)
* feat(models): expose embedding input modalities

Whether an embedding model maps images into the same space as text is
what will decide if a knowledge base can store image vectors. The
catalog already had the field, but most multimodal embedding models
were missing it and embedding rows never reported it.

- Catalog: declare input modalities for the Doubao and Tongyi vision
  embeddings, the Aliyun multimodal / qwen-vl embedding families and
  jina-embeddings-v4, as listed in each vendor's documentation. The
  migration baseline changes for exactly these nine entries.
- Model responses resolve capabilities for embedding rows too, so
  capabilities.input reflects the catalog merged with spec.input.
- Model editor: an "Image input" switch for embedding models. It follows
  the catalog and writes spec.input only when the row differs from it;
  picking another catalog model or switching vendor drops the
  declaration. Model cards show the image badge for such models.

* feat(embedding): embed images with multimodal embedding models

Adds the image side of an embedder, for models that map images into the
same space as their text vectors so a text query can find an image.

- api: EmbedImage (bytes + MIME type, sent inline as base64 because the
  object store is usually unreachable from the vendor) and an
  ImageEmbedder implemented by all four protocols, each in the image
  shape its reference documents: Ark image_url, DashScope contents[].image,
  Gemini inlineData parts, and on the OpenAI shape an object keyed by the
  vendor's image_field (Jina: "image"). A plain OpenAI-compatible
  endpoint names no image field and takes no images.
- Settings: image_field, max_image_batch_size (default one per request),
  max_image_bytes and image_mime_types, filled from vendor docs for the
  DashScope multimodal models, gemini-embedding-2 and Jina.
- embedding: an optional ImageEmbedder interface reached through
  AsImageEmbedder, so existing Embedder implementations and fakes are
  untouched. An embedder takes images only when the catalog (or the
  row's spec.input) lists image AND the endpoint can carry one. Images
  are checked against the documented limits before any request, then
  split at the documented images per request. The concurrency, debug
  and Langfuse decorators forward it; logs and traces carry the MIME
  type and size, never the bytes.

The migration baseline changes only for the entries whose resolved
embedding settings now carry image limits.

* feat(knowledge): index images by their own vector and recall them

When the knowledge base's embedding model embeds images, each image now
gets a vector of its own next to its caption and OCR chunks, so a query
can find an image by what it shows even where the caption left it out.

Ingestion
- After the caption/OCR chunks are indexed, the multimodal task embeds
  the image (embedding.PrepareImage converts and shrinks it to the
  vendor's documented limits) and stores it as a new image_vector chunk:
  Content is the caption (OCR text when there is none) for reranking and
  the answer context; its index row carries the image's vector with
  SourceType ImageSourceType. Scanned PDF pages are skipped, their OCR
  text serves better. A failure here is logged and never fails the task.
- The image vector gets its own chunk rather than a second index row on
  the caption chunk: OpenSearch and Weaviate key documents by ChunkID,
  so a second row would overwrite the caption's text vector, and copy,
  move and enable/disable all follow chunk IDs.
- CompositeRetrieveEngine.BatchIndexVectors writes precomputed vectors
  through each engine's normal BatchIndex, keeping row order (the
  composite's de-duplication does not).
- Paths that rebuild vectors from text leave image_vector alone:
  syncChunkIndex only syncs its enabled flag, updateChunkVector skips
  it, and a caption edit updates its Content only.
- The multimodal task's caption/OCR index rows now set IsEnabled; only
  backends with a column default were storing them as enabled.

Retrieval
- A text query scores lower against an image than against text, so when
  the model embeds images the document vector search is widened by half
  (capped at 500) and run at min(VectorThreshold, 0.1); each store
  group's hits are then held to their own threshold before score
  normalization: text at VectorThreshold, images at min(VectorThreshold,
  0.1). Keyword hits on image rows are dropped, their Content being the
  already-indexed caption.
- image_vector hits are searchable and merge like caption hits: parent
  context is attached, and the caption is not repeated from ImageInfo.

Adds golang.org/x/image for scaling and WebP/BMP decoding.

* fix(models): report no thinking levels for embedding, rerank and ASR models

Embedding rows now carry capabilities, and the end-to-end run showed them
listing off/auto/minimal/low/medium/high: these references resolve no chat
protocol, and an empty level map reads as "every level supported". A model
without a chat protocol does not think, so its capabilities say so.

* fix(compaction): never read a cancelled turn's stream as a summary

streamSummary selects on the turn's context and the provider stream at
once. When the turn is already cancelled both are ready (providers close
the stream once their context ends), and select picks at random, so a
stopped turn could come back as a completed, partial summary with a nil
error. TestSummarizationIsNotRetriedOnceTheTurnIsCancelled caught it
intermittently in CI and reproduces under -race with several -cpu values.

Check the context before each receive and again when the stream closes.

* feat(rerank): score images with multimodal rerank models

Rerank models that read images can now be sent an image document and
score it against a text query on the same scale as text, which is what a
recall made by an image's own vector needs: a text reranker only sees the
caption, and rejects an image for what the caption left out.

- api: ImageReranker, implemented by the Cohere and NIM shapes. Image
  documents go in requests of their own, so text requests keep their
  exact shape. NIM sends image-only passages without text, as base64 data
  URLs (its VLM reranking reference requires both). The Cohere shape sends
  {"<image_field>": data URI} documents and asks for no echo of them.
- The image settings embeddings already had (image_field,
  max_image_batch_size, max_image_bytes, image_mime_types) move into a
  shared ImageInput embedded in both settings; resolved embedding settings
  serialize exactly as before.
- Catalog: jina-reranker-m0 and nvidia/llama-nemotron-rerank-vl-1b-v2 take
  text and image. Jina names "image" documents; its base64 form is not
  shown in any reranker example, so the data URI its embeddings reference
  documents is used and marked unverified. NVIDIA lists its six formats
  and 512 passages per request.
- rerank: an optional ImageReranker reached through AsImageReranker, gated
  on the model's catalog input and the endpoint; images are checked against
  the documented limits before any request, scores are normalized like
  text, and the debug and Langfuse decorators forward it without logging
  image bytes.
- Image preparation (format conversion, shrink to fit) moves from the
  embedding package to internal/models/imageprep, since rerank needs it
  too.

* feat(retrieval): keep a vector hit's own similarity through fusion

Hybrid fusion replaces a hit's Score with a rank-based RRF score. That is
right for ordering, but it erases the evidence reranking needs to judge an
image found by its own vector: how close the vector actually was. Record
the engine's similarity on each vector hit (IndexWithScore.VectorScore)
before normalization and fusion rewrite Score, and carry it onto the
search result (SearchResult.VectorScore, not serialized).

* feat(reranking): judge image hits by their image, not only their caption

An image found by its own vector was reranked on its caption and OCR text,
so a text reranker rejected it for exactly what the caption left out.

- With a reranker that reads images and a LoadImage callback, image_vector
  hits are scored by their image (prepared to the model's limits) on the
  same scale as text, and pass or fail the same threshold. A hit whose
  image cannot be read, prepared or scored falls back to its text.
- With a text reranker, its verdict stands for images that are mostly
  text (OCR of 200 characters or more). A pictorial hit it rejects is kept
  when its vector similarity (VectorScore) reaches Options.ImageKeepScore:
  at most two, strongest vector first, appended after the ranked results
  and marked kept_by=image_vector. This also returns an image when no text
  passes at all.
- Diagnostics report images_scored and images_kept.

* feat(rerank): read image hits' images in every rerank path

The chat pipeline, the agent search_knowledge tool and the hybrid-search
API now pass the rerank stage a way to read an image hit's image and an
image keep score, so a multimodal reranker scores images by what they show
and a text reranker can no longer drop a pictorial image its caption
undersells.

- KnowledgeBaseService.ReadChunkImage reads the first image of a result's
  image_info from the storage of its knowledge base (the knowledge base's
  tenant, so shared knowledge bases read from their owner's storage),
  capped at 32 MB. Plain http(s) images go through the SSRF-safe
  downloader.
- The backend resolution the image multimodal task used moves into a
  shared knowledgeImageReader, so ingestion and reranking find an image in
  the same place.
- reranking.ImageKeepScoreFor: an image is kept over a text reranker only
  when its vector was at least as close as a text hit had to be, and never
  below 0.25 (empirical).

* feat(chat): show a vision model the images its answer rests on

An image recalled by its own vector reached the answer as its caption only,
so a vision model could not answer from what the caption left out.

- Knowledge QA: INTO_CHAT_MESSAGE reads, for a model that supports vision,
  the images of the contexts that rest on an image vector match (at most
  three distinct ones, prepared to what every vision API takes: PNG, JPEG,
  WebP or GIF within 5 MB), attaches them after the user's own images, and
  names the contexts they belong to.
- Agent: search_knowledge attaches the same images when the agent's model
  supports vision (the agent describes tool images to models that cannot
  see them, and these already carry captions). Stored history drops them;
  they are read again from storage.
- An image hit and its caption hit read the same once merged, and
  de-duplication keeps either. The kept copy now inherits the image
  (image_vector_match), in the pipeline's exact and partial-overlap
  de-duplication and in the agent tool's.
- Results the rerank stage kept outside its top-k (kept_by) no longer fall
  to the later top-k cuts: the pipeline's FilterTopK and the agent tool's
  MMR and limit let them ride along after the ranked results.

* docs(website): describe how image hits are reranked and shown to vision models

* Fix image evidence preservation and model source mapping
2026-10-09 13:01:48 +08:00
lyingbug 0c5d28abb7
feat: index and recall images with multimodal embedding models (#3683)
* feat(models): expose embedding input modalities

Whether an embedding model maps images into the same space as text is
what will decide if a knowledge base can store image vectors. The
catalog already had the field, but most multimodal embedding models
were missing it and embedding rows never reported it.

- Catalog: declare input modalities for the Doubao and Tongyi vision
  embeddings, the Aliyun multimodal / qwen-vl embedding families and
  jina-embeddings-v4, as listed in each vendor's documentation. The
  migration baseline changes for exactly these nine entries.
- Model responses resolve capabilities for embedding rows too, so
  capabilities.input reflects the catalog merged with spec.input.
- Model editor: an "Image input" switch for embedding models. It follows
  the catalog and writes spec.input only when the row differs from it;
  picking another catalog model or switching vendor drops the
  declaration. Model cards show the image badge for such models.

* feat(embedding): embed images with multimodal embedding models

Adds the image side of an embedder, for models that map images into the
same space as their text vectors so a text query can find an image.

- api: EmbedImage (bytes + MIME type, sent inline as base64 because the
  object store is usually unreachable from the vendor) and an
  ImageEmbedder implemented by all four protocols, each in the image
  shape its reference documents: Ark image_url, DashScope contents[].image,
  Gemini inlineData parts, and on the OpenAI shape an object keyed by the
  vendor's image_field (Jina: "image"). A plain OpenAI-compatible
  endpoint names no image field and takes no images.
- Settings: image_field, max_image_batch_size (default one per request),
  max_image_bytes and image_mime_types, filled from vendor docs for the
  DashScope multimodal models, gemini-embedding-2 and Jina.
- embedding: an optional ImageEmbedder interface reached through
  AsImageEmbedder, so existing Embedder implementations and fakes are
  untouched. An embedder takes images only when the catalog (or the
  row's spec.input) lists image AND the endpoint can carry one. Images
  are checked against the documented limits before any request, then
  split at the documented images per request. The concurrency, debug
  and Langfuse decorators forward it; logs and traces carry the MIME
  type and size, never the bytes.

The migration baseline changes only for the entries whose resolved
embedding settings now carry image limits.

* feat(knowledge): index images by their own vector and recall them

When the knowledge base's embedding model embeds images, each image now
gets a vector of its own next to its caption and OCR chunks, so a query
can find an image by what it shows even where the caption left it out.

Ingestion
- After the caption/OCR chunks are indexed, the multimodal task embeds
  the image (embedding.PrepareImage converts and shrinks it to the
  vendor's documented limits) and stores it as a new image_vector chunk:
  Content is the caption (OCR text when there is none) for reranking and
  the answer context; its index row carries the image's vector with
  SourceType ImageSourceType. Scanned PDF pages are skipped, their OCR
  text serves better. A failure here is logged and never fails the task.
- The image vector gets its own chunk rather than a second index row on
  the caption chunk: OpenSearch and Weaviate key documents by ChunkID,
  so a second row would overwrite the caption's text vector, and copy,
  move and enable/disable all follow chunk IDs.
- CompositeRetrieveEngine.BatchIndexVectors writes precomputed vectors
  through each engine's normal BatchIndex, keeping row order (the
  composite's de-duplication does not).
- Paths that rebuild vectors from text leave image_vector alone:
  syncChunkIndex only syncs its enabled flag, updateChunkVector skips
  it, and a caption edit updates its Content only.
- The multimodal task's caption/OCR index rows now set IsEnabled; only
  backends with a column default were storing them as enabled.

Retrieval
- A text query scores lower against an image than against text, so when
  the model embeds images the document vector search is widened by half
  (capped at 500) and run at min(VectorThreshold, 0.1); each store
  group's hits are then held to their own threshold before score
  normalization: text at VectorThreshold, images at min(VectorThreshold,
  0.1). Keyword hits on image rows are dropped, their Content being the
  already-indexed caption.
- image_vector hits are searchable and merge like caption hits: parent
  context is attached, and the caption is not repeated from ImageInfo.

Adds golang.org/x/image for scaling and WebP/BMP decoding.

* fix(models): report no thinking levels for embedding, rerank and ASR models

Embedding rows now carry capabilities, and the end-to-end run showed them
listing off/auto/minimal/low/medium/high: these references resolve no chat
protocol, and an empty level map reads as "every level supported". A model
without a chat protocol does not think, so its capabilities say so.

* fix(compaction): never read a cancelled turn's stream as a summary

streamSummary selects on the turn's context and the provider stream at
once. When the turn is already cancelled both are ready (providers close
the stream once their context ends), and select picks at random, so a
stopped turn could come back as a completed, partial summary with a nil
error. TestSummarizationIsNotRetriedOnceTheTurnIsCancelled caught it
intermittently in CI and reproduces under -race with several -cpu values.

Check the context before each receive and again when the stream closes.

* feat(knowledge): put image vectors behind a knowledge-base switch, off by default

Image vector indexing and recall used to turn on whenever the knowledge
base's embedding model declared image input. A model that takes images is
often chosen for text alone, so an upgrade would have added an embedding
call per image, widened vector recall 1.5x and lowered the image threshold
to 0.1 for those knowledge bases without anyone asking.

ImageProcessingConfig gains image_vector_enabled (default false; stored
JSON without the key reads as off, so existing knowledge bases stay off
with no migration). Both sides now need the switch and the model:

- indexImageVector returns before touching the model unless the KB opted
  in (KnowledgeBase.IsImageVectorEnabled, which also needs vector indexing).
- applyImageRecall resolves the embedding model only when some KB in the
  search opted in, and records the opted-in KBs per store group
  (storeGroup.ImageKBIDs). Only those groups widen the pool and lower the
  threshold; image hits of any other KB, including ones indexed before the
  switch was turned off, are dropped.

The KB editor shows the switch in the image processing section, with a
warning when the selected embedding model does not declare image input;
docs describe the switch, its cost and what turning it off does.

* fix(knowledge): leave image vectors behind when cloning into a KB that is off

CloneChunk copied every chunk of the source document and then CopyIndices
for all of them, so copying a document from a knowledge base with image
vectors on into one with the switch off (same model and store) still
created image_vector chunks and their index rows, bypassing the switch
that indexImageVector honours.

Image vector chunks are now dropped from the copy when the target has not
opted in. Every engine's CopyIndices copies only the rows of chunks in the
mapping, so their index rows stay behind too.

* fix(search): refill the vector pool past stale image vectors

Image hits of a knowledge base whose switch is off were dropped only after
the engine had ranked and cut its results at TopK. Image vectors indexed
before the switch was turned off could fill that pool and push out valid
text hits: with MatchCount 5 the pool is 50, and 50 stale image rows
ranked above the text left the search empty.

No engine can filter rows by source type (Postgres and SQLite do not even
take ExcludeChunkIDs), so the exclusion cannot move into the query. When
the document vector pool comes back full and holds stale image rows, that
search alone runs again on a pool twice as large, until it holds TopK rows
that are not stale, the index runs out, or the pool reaches
maxRetrievalPoolSize (at most four extra calls from a pool of 50). The pool
is then cut back to TopK such rows, matching an engine-side exclusion
unless more than maxRetrievalPoolSize-TopK stale rows rank above the text.

* feat(i18n): translate the image vector switch and image input keys into zh-TW

The zh-TW locale arrived from main after this branch added five keys: the
knowledge base's image vector switch (label, description, unsupported-model
warning) and the embedding model's image input flag (label, description).

* fix(knowledge): leave image vectors behind when moving into a KB that is off

A reuse_vectors move relocates a document's chunks and index rows in place:
MoveKnowledgeIndices rebinds the rows by document ID and
MoveChunksByKnowledgeID rebinds the chunks. Moving a document from a
knowledge base with image vectors on into one with the switch off therefore
carried its image_vector chunks and their vectors along, the same bypass the
clone path had.

When the target has not opted in, the image_vector chunks are now deleted
during the move, their index rows first and then the chunks, so a retry
finds the chunks again rather than leaving orphaned rows. A target that
opted in still receives them unchanged.

* fix(retriever): keep image vectors out of keyword-only engines

BatchIndexVectors handed image rows to every engine of the composite,
including one that serves keyword retrieval alone (Elasticsearch v7 next to
a vector store). There the row holds nothing but the image's caption, which
the caption's own chunk already indexes, and retrieval discards every
keyword hit on it.

Image rows now go only to engines that serve vector retrieval. An engine
that serves both keeps one row for both kinds, so its keyword search still
meets them; the search side handles that.

* fix(search): refill the keyword pool past image rows

Every engine that serves both kinds of retrieval stores one row per index
entry and runs its keyword search over that row's content, so an image
vector row, whose content is its caption, is a keyword hit wherever the
caption's own chunk is. filterImageHits drops all of them, but only after
the engine has cut its ranking at TopK, so they crowded out text and caption
hits; a keyword ranking that put the image rows first came back empty.

The keyword pool is now refilled the way the vector pool is refilled past
stale image vectors: when it came back full and held image rows, the
keyword search runs again on a pool twice as large, up to
maxRetrievalPoolSize, and is cut back to TopK hits that are kept.

Leaving the rows out of the keyword index instead would mean writing them
without searchable content (Milvus and Tencent VectorDB derive a BM25
sparse vector from it, and how they take an empty one is unverified) or a
source-type filter in every engine's keyword query (ten engines, and
filterable fields that existing collections lack).

* revert: let image vectors move with the document into a KB that is off

This reverts commit e259977fe915f7d0bc30c4b02bc7af04da1c8dd4.

Deleting the image vector index rows by chunk ID before the move is not
reliable: the Weaviate and Elasticsearch v8 delete wrappers ignore partial
failures, and OpenSearch rejects more than 1000 IDs with ErrBatchTooLarge.
A partial delete followed by the chunk delete leaves index rows whose chunks
are gone, which nothing ever cleans up.

Without it, the image_vector chunks and their index rows move together
with the document. While the target's switch is off they are filtered at
query time (the pool refill keeps them from crowding out text), they are
removed with the document on delete, reparse or knowledge base delete, both
chunks and index rows by knowledge ID, and they become active again if the
target turns the switch on. The clone path keeps dropping them, since there
the rows are copied rather than moved.

* fix(search): keep the image row refill from losing hits it already had

refillPool replaced a pool with the answer to each larger search. A refill
that answered only in part (RetrieveResult.Error, swallowed into a warning
by retainPartialResults, as when some collections of the store did not
answer) could hold fewer text rows than the complete smaller pool, or none:
40 stale image rows above 10 text came back from a partial refill as the
40 image rows alone, and the search lost all 10 text hits. A refill that
failed outright failed the whole store group, discarding a first round that
had succeeded.

A refill is now best effort. A failed or incomplete refill search stops
the refill and leaves the last complete pool in place, cut back to TopK
kept rows as before, so a refill can no longer leave a search worse off
than it was without it. Merging the partial answer into the pool was not
done: its rows would need deduplicating and its ranking is cut at a
different TopK over a different set of collections, while keeping the
previous pool is exact.

* docs(knowledge): say what happens to image vectors in a KB that is off and what the refill costs

The image vector section now says that with the switch off, image vectors
that were indexed before it was turned off, or that came along when the
document was moved in from a KB that has it on, are kept but filtered at
query time, are removed when the document is deleted or reparsed or the
KB is deleted, and are recalled again once the switch is turned on; a copy
into such a KB leaves them behind.

It also states the cost of keeping dropped image rows from crowding text
out of the pools: the vector and keyword pools are searched again with
TopK doubled up to the 500 candidate cap, which adds queries for store
groups that hold such rows, and text can still be missed when more than
about 500 candidates are image rows. A failed or partial refill keeps the
previous complete pool.

* fix(search): log unexpected refill result sets, and describe the refill stop condition precisely
2026-10-08 18:19:53 +08:00
ZW. 5a25e26142
feat(rerank): support Pinecone Inference protocol (#3957)
* feat(rerank): support Pinecone Inference protocol

* test(rerank): align Pinecone catalog baseline and lint

* test(models): expect 29 built-in vendors with Pinecone and TEI

---------

Co-authored-by: bigdu332 <bigdu332@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-08 17:54:15 +08:00
ZW. 99b4c3a9b0
feat(rerank): support Hugging Face TEI protocol (#3956)
* feat(rerank): support Hugging Face TEI protocol

* style(rerank): satisfy TEI lint checks

* fix(rerank): cap Hugging Face TEI batches at 32 documents

TEI rejects requests above --max-client-batch-size, which defaults to 32.
Without a declared ceiling the shared batching layer sends the whole
candidate set in one request, so a default TEI deployment fails rerank
whenever hybrid retrieval returns more than 32 candidates.

* fix(rerank): send truncate and merge extra_body in the TEI client

TEI before 1.9 starts with --auto-truncate=false and rejects the whole
/rerank call when one passage is longer than the model's input limit, so
retrieval silently falls back to the original order. Ask for truncation
explicitly, as the NIM client does with truncate=END.

The TEI client also ignored compat extra_body, unlike the Cohere, NIM and
DashScope clients. Merge it the same way: query, texts and raw_scores stay
owned by the client, truncate is a default that extra_body may override.

---------

Co-authored-by: bigdu332 <bigdu332@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-08 14:02:49 +08:00
wizardchen a1b0d0cb6c fix(models): serve qwen3-rerank on the native DashScope rerank protocol
The catalog rebuild marked qwen3-rerank unsupported because Alibaba also
documents a flat /compatible-api/v1/reranks shape for it. The native
text-rerank endpoint serves the same model with identical scores (#3558),
and v0.8.0 routed it there, so existing Aliyun rerank rows naming it
failed with "failed to get rerank model" after upgrading.

Drop the unsupported override and the deprecated flag so the model
resolves to dashscope-rerank again and shows up in the picker.
2026-09-24 14:18:55 +08:00
lyingbug e179f44445
refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589)
* refactor(models): separate providers, catalog and runtime configuration

* refactor(models): isolate provider definitions and runtime catalogs

* docs: exclude refactoring analysis notes from the pull request
2026-09-23 14:20:08 +08:00