OwlBurst/Ponysearch
1
0
Fork 0
forked from Ponysearch/Ponysearch
Code Pull requests Activity

[fix] wikipedia engine: don't raise an error when the query is not found

Browse source 
Add a new parameter "raise_for_status", set by default to True.
When True, any HTTP status code >= 300 raise an exception ( #2332 )
When False, the engine can manage the HTTP status code by itself.
This commit is contained in:
Alexandre Flament 2020-12-04 20:04:39 +01:00
parent f56e78ee80
commit f0054d67f1
3 changed files with 11 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
docs/dev/engine_overview.rst
View file

@ -134,9 +134,9 @@ The function ``def request(query, params):`` always returns the ``params``
variable. Inside searx, the following paramters can be used to specify a search
request:
================== =========== ========================================================================
================== =========== ==========================================================================
argument type information
================== =========== ========================================================================
================== =========== ==========================================================================
url string requested url
method string HTTP request method
headers set HTTP header information
@ -145,7 +145,8 @@ cookies set HTTP cookies
verify boolean Performing SSL-Validity check
max_redirects int maximum redirects, hard limit
soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
================== =========== ========================================================================
raise_for_status bool True by default: raise an exception if the HTTP code of response is >= 300
================== =========== ==========================================================================
example code

4
searx/engines/wikipedia.py
View file

@ -37,13 +37,15 @@ def request(query, params):
language=url_lang(params['language']))
params['headers']['User-Agent'] = searx_useragent()
params['raise_for_status'] = False
params['soft_max_redirects'] = 2
return params
# get response from search-request
def response(resp):
if not resp.ok:
if resp.status_code == 404:
return []
results = []

6
searx/search.py
View file

@ -143,7 +143,8 @@ def send_http_request(engine, request_params):
response = req(request_params['url'], **request_args)
# check HTTP status
response.raise_for_status()
if request_params.get('raise_for_status'):
response.raise_for_status()
# check soft limit of the redirect count
if len(response.history) > soft_max_redirects:
@ -340,7 +341,8 @@ def default_request_params():
'url': '',
'cookies': {},
'verify': True,
'auth': None
'auth': None,
'raise_for_status': True
}