Fix SearchGraph silent failure: raise clear error + retry transient search blocks#1113
Open
ghshhf wants to merge 1 commit into
Open
Fix SearchGraph silent failure: raise clear error + retry transient search blocks#1113ghshhf wants to merge 1 commit into
ghshhf wants to merge 1 commit into
Conversation
- SearchGraph.run() now raises SearchGraphEmptyAnswerError (carrying the considered URLs) when the graph finishes with no usable answer, instead of silently returning 'No answer found.'. This distinguishes a real blocking/empty failure from a genuine empty result (related to ScrapeGraphAI#1110). - search_on_web gains max_retries (default 1) with exponential backoff and UA rotation on Bing/SearXNG, so transient 403/rate-limit blocks are retried before surfacing a clear SearchRequestError. - SearchInternetNode / SearchGraph thread max_retries through to search_on_web.
dosubot
Bot
added
bug
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the long-standing silent-failure behavior behind #1110 ("empty output / NA").
SearchGraph no longer fails silently
SearchGraph.run()used to return the opaque string"No answer found."whenever the graph produced no answer (e.g. search returned nothing, or every considered URL was blocked / returned empty). It now raises a dedicatedSearchGraphEmptyAnswerErrorthat includes the list of considered URLs, so callers can tell a real blocking/configuration problem apart from a genuine empty result.Search layer is more resilient to anti-bot blocks
search_on_webgains amax_retriesparameter (default 1) with exponential backoff. Bing / SearXNG rotate the User-Agent on every retry attempt.403/ rate-limit failures are retried before surfacing a clearSearchRequestError.Tests
tests/test_search_graph.pyto assert the new exception behavior (including the with-URLs case).tests/utils/research_web_retry_test.pycovering the retry mechanism, exhausted-retry raising, and UA rotation on retry (all mock-based, no network).Related to #1110.