WeKnora/internal/infrastructure/web_fetch/testdata/extraction
lyingbug 647848f395
fix(agent): improve web search and persistent page reading (#3086)
* fix(agent): improve web search and persistent page reading

* fix(agent): make web fetch snapshots waitable and retryable

Keep in-flight page fetches independent of a caller's short deadline so
content=true and continuations cannot cancel a shared download or miss a
snapshot that is still loading. Mark snapshot_expired as retryable, fetch
only the first three search hits in parallel, and stop forcing Brave's
country to US.
2026-09-07 23:09:27 +08:00
..
README.md fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
article.html fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
fallback.html fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00
structure.html fix(agent): improve web search and persistent page reading (#3086) 2026-09-07 23:09:27 +08:00

README.md

Markdown extraction fixtures

These HTML fixtures are authored for the extraction regression tests:

  • article.html: retains sibling sections around a nested .content element after successful Readability extraction.
  • structure.html: preserves links, lists, tables and code indentation while removing empty anchors and navigation noise.
  • fallback.html: retains directory entries and resolves relative links when article extraction falls back to document content.

Run the fixture tests from the repository root:

go test ./internal/infrastructure/web_fetch -run TestMarkdownExtractionFixtures

The tests assert content and structural preservation independently of incidental Markdown spacing and list marker choices.