WeKnora/internal/infrastructure/web_fetch
jack 1fc00cdc08
fix(agent): share one configurable budget for web_fetch (#4432)
web_fetch stopped at 60s inside the page fetch. The tool wrapper, storePage,
and the HTTP and chromedp fetcher each had their own cap, so a page that
needed about 150s never finished.

WEKNORA_WEB_FETCH_TIMEOUT is the single page budget (default 3m). HTTP and
chromedp share it, and the agent wrapper waits 5 seconds longer so the tool
can return its own result. Other tools and the 15s pipeline fetcher keep
their current timeouts.

Fixes #2298

Co-authored-by: Frank-zhu2000 <332350688+Frank-zhu2000@users.noreply.github.com>
2026-10-17 22:28:32 +08:00
..
testdata/extraction fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
fetcher.go fix(agent): share one configurable budget for web_fetch (#4432) 2026-10-17 22:28:32 +08:00
fetcher_ssrf_test.go fix(security): harden web_fetch and datasource connector SSRF paths 2026-07-02 12:27:48 +08:00
fetcher_test.go fix(agent): share one configurable budget for web_fetch (#4432) 2026-10-17 22:28:32 +08:00
fetcher_whitelist_only_test.go feat(security): add a whitelist-only outbound mode that refuses before DNS (#3393) 2026-09-23 12:02:06 +08:00
markdown.go fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
markdown_test.go fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00