WeKnora/internal/handler
LuckVd b5c8d90cc2
fix(chat_pipeline): treat non-JSON model refusals as skipped graph extraction (#3606)
* fix(chat_pipeline): treat non-JSON model refusals as skipped graph extraction

When a chunk carries nothing extractable (e.g. a table-of-contents page),
the LLM correctly answers with a prose refusal such as "抱歉…". parseOutput
then fed that prose to json.Unmarshal, failing with a confusing
"invalid character 'æ'" (the first byte of a CJK character), marking the
chunk task GRAPH_EXTRACT_FAILED and letting asynq replay the identical
doomed LLM call up to the retry limit.

Classify responses that contain no JSON structure at all (no '{' or '[')
as errModelDeclined and complete the extraction with an empty graph so the
task succeeds as an explicit skip. Malformed or truncated JSON (structure
present) still fails and retries as before.

Fixes #3600

* fix(chat_pipeline): keep empty model responses retriable in graph extraction

An empty response is usually transient (gateway returning 200 with no
content, a reasoning model spending its whole budget on reasoning,
content filtering), and a retry often succeeds. Only non-empty prose
with no JSON structure is treated as a declined extraction.

* fix(graph): skip prose refusals only in the async chunk task, keep truncation retriable

ParseGraph no longer turns a prose answer into an empty graph. It returns
an error wrapping ErrModelDeclined, and ChunkExtractService.Handle is the
only caller that treats it as a skip (span output skipped=model_declined).
The /initialization "try extract" endpoint and query-side entity extraction
report the refusal as a failure again, so the graph settings form no longer
gets its few-shot examples replaced with an empty result.

Extractor.Extract drops the ErrModelDeclined marker when finish_reason is
"length": prose with no JSON because the output budget ran out is
truncation, and asynq should retry it.

* fix(graph): retry every truncation spelling, not only finish_reason=length

Extract kept a prose answer retriable only when FinishReason was exactly
"length". max_tokens / max_output_tokens (and case/space variants that
IsLengthFinishReason already accepts), a stream that broke before the stop
event ("incomplete") and an adapter that reports no reason were still read
as a refusal, so the async chunk task skipped the chunk for good.

Only a reason that proves the model stopped on its own now counts as a
refusal. Ollama's non-streaming Chat also forwards done_reason, so a
num_predict cut-off is reported as "length" instead of an empty string.

* fix(graph): only treat a response with no JSON anywhere as a refusal

The refusal check looked at the first extracted fence, so a prose fence
followed by a JSON fence was skipped as a refusal instead of failing as a
retriable parse error.

---------

Co-authored-by: LuckVd <LuckVd@users.noreply.github.com>
Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-10-08 14:57:11 +08:00
..
dto revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
session fix(embed): 内嵌网页只传图片不输入文字时不再返回 400 2026-09-30 15:26:51 +08:00
audit_log.go Remove redundant Wiki operation log 2026-07-28 11:18:33 +08:00
audit_log_test.go feat(knowledge): add scoped activity audit trail 2026-07-23 10:50:32 +08:00
auth.go feat(auth): 新增仅限邀请注册档位 (#3900) 2026-09-30 13:32:55 +08:00
auth_browser_preferences_test.go feat(browser): polish connection settings, open tasks in a new window, and streamline prompts (#3193) 2026-09-11 21:28:58 +08:00
auth_change_password_test.go feat: support complex password (#2929) 2026-09-01 11:50:10 +08:00
auth_desktop_security_test.go fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
auth_oidc_start_test.go feat(auth): add /auth/oidc/start endpoint for direct 302 OIDC login 2026-08-11 15:41:48 +08:00
auth_register_by_invite.go feat(auth): 新增仅限邀请注册档位 (#3900) 2026-09-30 13:32:55 +08:00
auth_register_by_invite_tenant_test.go feat(auth): 新增仅限邀请注册档位 (#3900) 2026-09-30 13:32:55 +08:00
auth_register_invite_only_test.go feat(auth): 新增仅限邀请注册档位 (#3900) 2026-09-30 13:32:55 +08:00
chunk.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
chunker_debug.go fix(chunking): 修复内联 HTML 表格与保护模式导致的超长 chunk (#3404) 2026-09-18 21:10:17 +08:00
chunker_debug_test.go fix(chunking): normalize line endings before splitting / 在分块前统一换行符 2026-08-05 19:24:45 +08:00
custom_agent.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
custom_agent_api_key_scope_test.go feat(agent): enhance suggested questions API with tag scopes 2026-07-17 10:19:43 +08:00
custom_agent_avatar_test.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
custom_agent_reasoning_effort_test.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
custom_agent_sandbox_test.go feat(sandbox) (#2301) 2026-08-10 21:28:20 +08:00
custom_agent_test.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
datasource.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
datasource_credentials.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
datasource_schedule_test.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
datasource_test.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
deployment_capabilities.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
deployment_capabilities_test.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
document_write_scope_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
embed_allowed_origins_test.go fix(embed): enforce host origin allowlists on framed pages 2026-09-14 18:03:57 +08:00
embed_channel.go fix(embed): warn at startup when no signing key is configured 2026-09-24 20:29:17 +08:00
embed_channel_exchange_test.go refactor(context): replace middleware.EmbedChannelContextKey with types.EmbedChannelContextKey 2026-08-13 18:00:02 +08:00
embed_flow_test.go fix(security): close the low-severity shared-space findings (#3416) 2026-09-18 22:58:46 +08:00
embed_resource_urls_test.go fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
embed_session_test.go fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
evaluation.go feat: 新增API Key认证支持,优化Swagger文档安全配置 2025-12-22 15:01:09 +08:00
faq.go feat(faq): filter entries by enabled status (#3157) 2026-09-10 14:42:59 +08:00
faq_enabled_filter_test.go feat(faq): filter entries by enabled status (#3157) 2026-09-10 14:42:59 +08:00
host_project_picker.go feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) 2026-09-22 19:35:03 +08:00
host_project_picker_test.go feat(sandbox): run unpinned Lite sessions on the host OS sandbox (#3549) 2026-09-22 19:35:03 +08:00
im.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
im_locale_test.go feat(im): add per-channel reply locale (#3455) 2026-09-23 12:01:01 +08:00
im_platform_test.go feat(im,datasource): support Lark, Feishu's international edition 2026-07-17 11:05:54 +08:00
initialization.go fix(model): treat 400 as reachable in chat connection test again (#3929) 2026-10-08 14:02:35 +08:00
initialization_chat_check_test.go fix(model): treat 400 as reachable in chat connection test again (#3929) 2026-10-08 14:02:35 +08:00
initialization_config_redaction_test.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
initialization_extract_relations_test.go fix(chat_pipeline): treat non-JSON model refusals as skipped graph extraction (#3606) 2026-10-08 14:57:11 +08:00
initialization_fill_secrets_test.go fix(initialization): pass stored ExtraConfig to model connection test 2026-09-07 11:43:44 +08:00
initialization_model_merge_test.go refactor(models): rebuild vendor integration on protocol + catalog + vendor layers (#3470) 2026-09-20 21:28:23 +08:00
initialization_model_tenant_test.go fix(initialization): stamp the KB's tenant onto created models 2026-09-17 13:09:34 +08:00
initialization_owner_test.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
kb_access.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
kb_access_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
knowledge.go fix(knowledge): reject invalid JSON uploads before async parse (#3360) 2026-09-25 22:23:27 +08:00
knowledge_api_key_scope_test.go fix(api-keys): close KB scope gaps in handlers and CloneContext 2026-07-07 12:00:13 +08:00
knowledge_download.go feat(knowledge): add batch file downloads (#3300) 2026-09-16 15:40:10 +08:00
knowledge_download_test.go feat(knowledge): add batch file downloads (#3300) 2026-09-16 15:40:10 +08:00
knowledge_folder_test.go feat(knowledge): store folder path for folder uploads and expose folder tree API 2026-08-04 20:45:29 +08:00
knowledge_last_activity_test.go feat(knowledge): flag documents that stop making progress (#3557) 2026-09-22 21:34:08 +08:00
knowledge_move_gate_test.go fix: reject cross-store reuse_vectors knowledge move 2026-06-05 15:00:41 +08:00
knowledge_mutation_admission_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
knowledge_ownership_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
knowledge_preview_security_test.go fix(security): harden outbound requests and file previews 2026-07-02 19:15:46 +08:00
knowledge_sort_test.go feat: add configurable sorting to document and resource lists (#3318) 2026-09-23 12:00:21 +08:00
knowledge_spans_test.go feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535) 2026-09-25 22:22:58 +08:00
knowledge_tag_ids_test.go fix(knowledge): harden multi-tag data integrity and UX follow-ups 2026-06-26 10:52:52 +08:00
knowledge_transfer_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
knowledgebase.go fix(kb): let the update API set vlm_config (#3704) 2026-10-08 14:37:23 +08:00
knowledgebase_copy_preflight_test.go Simplify KB duplicate API and improve naming/docs coverage. 2026-07-08 20:55:54 +08:00
knowledgebase_hybrid_search_resource_urls_test.go feat(api): honor resource_urls=public on hybrid-search 2026-08-26 17:40:54 +08:00
knowledgebase_hybrid_search_test.go feat(retrieval): unify rerank across search APIs and expose diagnostics (#3675) 2026-09-24 20:41:49 +08:00
knowledgebase_not_found_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
knowledgebase_pr3_response_test.go feat(knowledge-base): validate vector store bindings on create, copy, and delete 2026-05-18 15:58:46 +08:00
knowledgebase_pr5_list_test.go feat: expose KB ↔ vector store binding in list, editor, and detail UI 2026-05-22 17:40:10 +08:00
knowledgebase_request_test.go feat(knowledge-base): validate vector store bindings on create, copy, and delete 2026-05-18 15:58:46 +08:00
knowledgebase_vlm_update_test.go fix(kb): let the update API set vlm_config (#3704) 2026-10-08 14:37:23 +08:00
list_pagination.go fix(handler): validate knowledge search pagination 2026-08-24 14:16:35 +08:00
list_pagination_test.go fix(handler): validate knowledge search pagination 2026-08-24 14:16:35 +08:00
mcp_credentials.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
mcp_endpoint.go feat(mcp): add built-in MCP server with per-workspace endpoints (#3371) 2026-09-17 22:40:05 +08:00
mcp_metadata.go feat(mcp)!: persist tool catalogs and add guided configuration (#3118) 2026-09-09 13:34:56 +08:00
mcp_metadata_test.go feat(mcp)!: persist tool catalogs and add guided configuration (#3118) 2026-09-09 13:34:56 +08:00
mcp_oauth.go fix(mcp): coordinate OAuth token refresh and authorization polling 2026-07-23 15:35:24 +08:00
mcp_service.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
mcp_tool_approval_test.go feat(mcp): add per-tool enable toggle (#2989) 2026-09-05 17:19:20 +08:00
mcp_usage_instructions.go feat(mcp)!: streamline setup and use short routing handles 2026-09-09 16:11:52 +08:00
mcp_usage_instructions_test.go feat(mcp)!: streamline setup and use short routing handles 2026-09-09 16:11:52 +08:00
me_env_var.go feat(sandbox): inject per-user and workspace skill env vars at execution 2026-08-27 14:32:00 +08:00
me_env_var_test.go feat(sandbox): inject per-user and workspace skill env vars at execution 2026-08-27 14:32:00 +08:00
memory.go fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
memory_consistency_test.go fix(memory): preserve extraction progress and atomic memory updates (#3224) 2026-09-13 00:04:16 +08:00
message.go fix(api): address resource_urls review follow-ups 2026-08-05 14:42:51 +08:00
message_resource_urls_test.go fix(api): close the resource_urls=public gaps found in review 2026-08-05 14:42:51 +08:00
message_session_not_found_test.go fix(chat): stabilize history pagination and keep message order 2026-05-20 21:03:39 +08:00
message_suggestion.go fix(sessions): 放行 embed 运行时读取自身会话,修复追问建议 500 与刷新丢对话 2026-07-30 17:51:34 +08:00
message_suggestion_test.go fix(sessions): 放行 embed 运行时读取自身会话,修复追问建议 500 与刷新丢对话 2026-07-30 17:51:34 +08:00
model.go fix(frontend): 修复复制模型时模型名称被改写的问题 (#3830) 2026-09-29 19:32:08 +08:00
model_catalog.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
model_catalog_visibility_test.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
model_copy_test.go fix(frontend): 修复复制模型时模型名称被改写的问题 (#3830) 2026-09-29 19:32:08 +08:00
model_credentials.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
model_secret_extra_test.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
model_spec_flow_test.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
model_test.go refactor(models): separate provider definitions, model catalogs, and runtime configuration (#3589) 2026-09-23 14:20:08 +08:00
organization.go fix(security): close the low-severity shared-space findings (#3416) 2026-09-18 22:58:46 +08:00
organization_discovery_security_test.go fix: align client sessions and integration request handling (#3305) 2026-09-16 16:20:57 +08:00
organization_invite_test.go fix(security): close the low-severity shared-space findings (#3416) 2026-09-18 22:58:46 +08:00
organization_share_path_test.go fix(security): close cross-workspace access paths in shared spaces (#3408) 2026-09-18 22:00:02 +08:00
rbac_lookups.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
rbac_lookups_test.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
sandbox_check.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
sandbox_check_test.go feat(sandbox): add tenant-configurable network policy (#2995) 2026-09-03 10:49:46 +08:00
sandbox_config.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
sandbox_config_test.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
sandbox_skill.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
sandbox_skill_env_test.go feat(sandbox): inject per-user and workspace skill env vars at execution 2026-08-27 14:32:00 +08:00
sandbox_skill_test.go feat(lite): install and run skills on the host (#3661) 2026-09-24 21:30:34 +08:00
shared_agent_access.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
shared_agent_access_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
shared_agent_picker_test.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
skill_catalog.go refactor(skills): enhance skill upload size management and error handling 2026-09-02 15:08:05 +08:00
skill_catalog_test.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
skill_handler.go fix(docs): make `make docs` reproduce the committed Swagger again (#3751) 2026-10-08 01:26:08 +08:00
skill_handler_test.go fix(agent): scope the @Skill and @MCP pickers to the agent that will run (#3475) 2026-09-21 13:02:34 +08:00
storage_allowlist.go fix(storage): align default provider fallback with STORAGE_ALLOW_LIST 2026-07-15 11:05:00 +08:00
storagebackend.go feat(storage): add storage backend management API and UI enhancements 2026-07-15 14:54:39 +08:00
system.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
system_admin_audit_test.go test(system): cover system.* audit constants + emitAdminAudit contract 2026-05-26 21:13:56 +08:00
system_model_catalog.go feat(models): add system admin model catalog console (#3662) 2026-09-24 16:31:45 +08:00
system_model_catalog_test.go feat(models): add system admin model catalog console (#3662) 2026-09-24 16:31:45 +08:00
system_parser_check_test.go fix(security): close init config and parser-check redaction gaps 2026-07-02 14:41:42 +08:00
system_password_reset_test.go feat(system): add admin password reset with session revocation 2026-07-13 20:11:00 +08:00
system_runtime_test.go feat(knowledge): flag documents that stop making progress (#3557) 2026-09-22 21:34:08 +08:00
system_storage_engine_test.go feat(storage): improve storage connectivity error handling and multi-instance support 2026-07-15 14:54:39 +08:00
system_storage_ssrf_test.go fix(security): close remaining GHSA gaps from security triage 2026-07-02 15:03:53 +08:00
system_user_create_test.go refactor(system): drop idempotent flag from create-user response body 2026-09-07 17:30:47 +08:00
tag.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
tag_delete_test.go refactor(resource-access): unify scoped authorization and transfer recovery (#3097) 2026-09-08 15:36:33 +08:00
task_progress_auth.go fix(security): enforce KB access on wiki reads and related IDOR paths 2026-07-02 12:06:22 +08:00
task_progress_auth_test.go fix(security): enforce KB access on wiki reads and related IDOR paths 2026-07-02 12:06:22 +08:00
tenant.go feat(knowledge): derive knowledge base descriptions from document profiles (#3370) 2026-09-17 22:21:55 +08:00
tenant_api_key_validation_test.go feat: support updating API key configuration 2026-08-12 11:43:03 +08:00
tenant_invitation.go fix(invitation): harden auto-accept follow-ups from code review 2026-08-10 15:47:29 +08:00
tenant_invitation_accept_by_token_test.go feat: 邀请链接在 invite_only 模式下改为登录后加入空间 2026-08-10 14:57:38 +08:00
tenant_invitation_auto_accept_test.go fix(invitation): harden auto-accept follow-ups from code review 2026-08-10 15:47:29 +08:00
tenant_invite_link.go fix: prevent API keys from assigning owner role 2026-07-23 16:04:42 +08:00
tenant_member.go fix(invitation): harden auto-accept follow-ups from code review 2026-08-10 15:47:29 +08:00
tenant_member_test.go feat(memory): add cross-session long-term memory 2026-08-13 18:00:02 +08:00
tenant_parser_ssrf_test.go fix(security): redact IM list credentials and harden MinerU SSRF 2026-07-02 17:58:20 +08:00
tenant_policy.go feat(auth): support tenantless provisioning and gated self-service workspace creation 2026-07-13 18:13:51 +08:00
tenant_secret_disclosure_test.go refactor(api-keys): remove API key reset endpoint and update documentation 2026-07-07 12:00:13 +08:00
tenant_self_service_policy_test.go feat: support complex password (#2929) 2026-09-01 11:50:10 +08:00
upload_limit.go refactor(skills): improve skill bundle size handling and error management 2026-09-02 15:08:05 +08:00
upload_limit_test.go refactor(skills): improve skill bundle size handling and error management 2026-09-02 15:08:05 +08:00
user_resource_favorite.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
vectorstore.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
vectorstore_env_test.go fix: use consistent qdrant env settings (#3960) 2026-10-07 23:59:56 +08:00
web_search.go docs(api): add and fix swag annotations on handlers 2026-05-12 13:16:58 +08:00
web_search_provider.go revert: take the plugin system back out of main 2026-09-26 14:58:18 +08:00
web_search_provider_credentials.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
wechat_qrcode.go docs(api): add and fix swag annotations on handlers 2026-05-12 13:16:58 +08:00
weknoracloud.go refactor(terminology): rename user-facing tenant labels to workspace 2026-07-14 16:14:43 +08:00
wiki_page.go feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535) 2026-09-25 22:22:58 +08:00
wiki_rebuild_links_failure_test.go fix(wiki): 重建链接逐页写失败不再静默,半成品不再回 200 (#3884) 2026-10-08 01:25:53 +08:00
wiki_search_test.go feat(wiki): 增加跨知识库 Wiki 搜索 POST /wiki-search (#3535) 2026-09-25 22:22:58 +08:00