WeKnora/internal/application
lyingbug eaca7364cd
fix(knowledge): stop two parse paths from stranding a row in processing (#3445)
* fix(knowledge): stop two parse paths from stranding a row in processing

Both of these left parse_status on "processing" with nothing left to move
it, so the document sat there until the housekeeping sweep failed it an
hour later with "task stuck in processing, recovered by housekeeping" —
reaching the user as a slow, unexplained parse failure.

processChunks returned silently when the embedding model could not be
resolved (deleted model, expired credentials, unreachable provider). It
reports nothing to its callers, so the attempt ended with the row still
marked processing. Persist the failure with the underlying cause and
record the embedding stage as failed, matching how the sibling
GetASRModel branch already handles a model it cannot resolve.

enqueueImageMultimodalTasks seeded the multimodal fan-in counter with
len(images) before the fan-out, then only warned when a payload failed
to marshal or asynq refused the write. Those slots have no task behind
them to drain them, so the counter never reached zero and post-process
was never triggered. Release the shortfall after the fan-out, and drive
post-process directly when the release itself brings the counter to zero
or when nothing reached the queue at all — the same reconciliation
KnowledgePostProcess already performs for its enrichment subtasks.

Seeding the counter before enqueueing is kept deliberately: an image
task that finishes early would otherwise decrement a key the fan-out is
about to overwrite. The release is what settles the resulting gap.

Also collapse the Redis key into multimodalPendingKey so the fan-out and
fan-in cannot drift, and share one enqueueKnowledgePostProcessTask
between the no-multimodal path and the slot release.

* fix(knowledge): keep the embedding-model guard from clobbering abort state

Review follow-up on the two stranded-row fixes.

The embedding-model guard wrote parse_status=failed unconditionally, which
made two cases worse than the silent return it replaced. Resolving a model
reads the database, so the entry guard's verdict is already stale by the
time the error comes back:

  - A user cancel signals asynq CancelProcessing, which cancels the
    handler context. GetEmbeddingModel then fails with context.Canceled
    and the guard overwrote "cancelled" with "failed: ... context
    canceled" — mislabelling an interrupted run as a broken model. Worker
    preemption and graceful shutdown look the same.
  - ReplaceKnowledgeFile keeps the knowledge ID and swaps file_path.
    UpdateKnowledge is a full-row Save and file_path is not among
    omitFieldsOnUpdate, so the stale row rolled the path back over the
    replacement and marked the replacement's live attempt failed.

So the guard now returns without writing when the context is gone or the
error is a context error, re-checks isKnowledgeAborted before writing, and
persists through updateKnowledgeUnlessSourceReplaced like the completion
path already does.

The multimodal slot release gave up on the first Redis error. Since the
release is the only thing that can drain a slot with no task behind it,
that put the row straight back into the stranded state this fix exists to
remove. It now retries a bounded number of times and logs at error level
when it still cannot settle the counter. DECRBY is not idempotent, so a
retry after a lost reply can over-decrement; that finalizes early rather
than stranding, which is the direction to fail in, and post-process
already tolerates repeat deliveries. The nothing-enqueued branch's Del
also moves onto the detached context, so a cancelled parent no longer
leaves the seeded key to expire on its own 24h TTL.

Two reporting corrections while here: the span now carries the same
message as the document list instead of a fixed generic string, because
the timeline renders error_message and withholds error_detail; and the
comment no longer claims expired credentials or an unreachable provider
surface here — resolving a model only reads the row and constructs a
client, so those fail later on the first BatchIndex round-trip.

Adds 6 tests for the paths above. Two of them initially passed against
the unfixed code because the entry guard caught them before the model
call, so the repository fake now serves the cancelled / replaced row only
after the guard's own reads, which is the window that actually exists.
2026-09-20 13:02:05 +08:00
..
access feat(artifacts): let users delete generated files (#3447) 2026-09-20 13:01:46 +08:00
repository feat(artifacts): let users delete generated files (#3447) 2026-09-20 13:01:46 +08:00
service fix(knowledge): stop two parse paths from stranding a row in processing (#3445) 2026-09-20 13:02:05 +08:00