WeKnora/internal/infrastructure
Jundong Zhang d074ec7b0b
fix(websearch): return results for non-webpage Metaso scopes (#4010)
* fix(websearch): return results for non-webpage Metaso scopes

Metaso 把结果放在与 scope 同名的键下(webpage→webpages、scholar→scholars…),
而 Search() 只读 response.Webpages,导致 extra_config.scope 配成 document /
scholar / podcast / video / image 时永远返回 0 条。validMetasoScopes 与
website-docs/03-features/11-web-search.md 都把 6 个 scope 列为可用,属于
"文档承诺了、代码没实现"。

- 新增 itemsForScope:按 scope 选取对应结果数组,未知 scope 回退到首个非空数组并告警
- 新增 location():依次回退 link → url → imageUrl。image 条目只有 imageUrl,
  只认 link 会让这些条目在 agent 侧的 URL 校验里被全部丢弃
- parseMetasoDate 补 "2006年01月02日"、"2006/01/02" 两种实测格式
- metasoWebpage 只建模真正会被读取的字段:多声明一个类型不符的键,整次搜索就会因
  反序列化失败而返回 0 条。补 TestMetasoProviderToleratesUnmodelledFields 锁死这一点
- 日志补 scope= / array= 维度,便于区分"provider 没结果"与"读错了数组"
- 文档补充结果数组键名与 scope 的对应关系

测试:go test ./internal/infrastructure/web_search/... -count=1

* fix(websearch): scope Metaso result arrays only

Follow-up to the review. This change now does exactly one thing: read the
result array whose key matches the configured scope, and log which array
was read.

Removed — all of it added by the original commit, none of it supported by
a captured response:

- the `url` and `imageUrl` fields on metasoWebpage, and the location()
  helper. Neither key appears in the document, scholar or image
  responses, and a real image entry carries no `link` at all.
- the fallback to the first non-empty array when the target array is
  empty. Real responses key each array strictly by scope, so for any
  scope that passed validation this branch cannot be reached.
- the unknown-scope warning. NewMetasoProvider rejects unknown scopes
  before a provider is ever constructed.
- the "2006年01月02日" and "2006/01/02" date layouts. Neither format
  appears in any captured response, so parseMetasoDate is back to main.
  The accompanying TestMetasoParseDate was introduced with them and is
  removed too.

Tests: the TestMetasoProviderSearch fixture now returns `scholars` for a
scholar-scoped request — the previous fixture answered `webpages`, which
is what let the bug through. The document and scholar cases in
TestMetasoProviderScopeArrays are built from sanitised real responses,
and two new cases pin that `url`/`imageUrl` stay undeclared and that
image scope returns entries with an empty URL. Comments are English
again, matching the rest of the file.

The image scope limitation is unchanged from main and is now documented:
image entries carry no `link`, so the provider produces an empty URL and
the agent drops the result.

* style(websearch): drop added comments and revert doc paragraph

The PR is scoped to reading the result array that matches the scope, so the
extra doc comment blocks and the website-docs paragraph are noise. Remove the
12 comment lines added to metaso.go and the 29 added to metaso_test.go, and
restore website-docs/03-features/11-web-search.md to main (zero diff).

No behaviour change.
2026-10-10 10:25:40 +08:00
..
chunker fix(chunker): respect Markdown fences during heading detection 2026-10-08 21:06:59 +08:00
docparser fix(docparser): 云解析超时改为跟随既有配置,不再硬编码 (#3891) 2026-10-08 14:02:42 +08:00
web_fetch feat(security): add a whitelist-only outbound mode that refuses before DNS (#3393) 2026-09-23 12:02:06 +08:00
web_search fix(websearch): return results for non-webpage Metaso scopes (#4010) 2026-10-10 10:25:40 +08:00