WeKnora/internal/models/api/openairesponses
ordinary being f0adfac334
fix(models/api): cap non-streaming provider response bodies (#4397)
Endpoint.Do bounds the error body at 1 MiB. Thirty lines below it, roundTrip
read the success body whole, and the four chat protocols each did the same in
their own read: a reply was buffered without limit before anything inspected it.
The older models/chat layer already caps the same replies at 20 MiB.

Cap them through one helper, and return an over-cap body without the
TransportError wrapper. withRetry retries only TransportError, and repeating an
oversized reply would bill the request again for the same bytes - the same
reason its own doc gives for not retrying a reply that arrives whole but does
not decode.

64 MiB clears every payload the catalog can legitimately return: an embeddings
batch is bounded by BATCH_EMBED_SIZE (default 5) times the model dimensions, and
a chat reply by the model's output-token ceiling. Streaming replies are
assembled incrementally and are unaffected.
2026-10-16 20:31:42 +08:00
..
client.go fix(models/api): cap non-streaming provider response bodies (#4397) 2026-10-16 20:31:42 +08:00
openairesponses_test.go fix(llm): treat cut-off streams as broken and tighten LLM retries 2026-09-25 18:22:14 +08:00
request.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00