Commit graph

1348 commits

Author SHA1 Message Date
Markus Heiser
36aee70c24
Merge pull request #910 from tiekoetter/fix-909
[fix] google images engine: Fix 'scrap_img_by_id' function
2022-02-20 18:29:50 +01:00
Markus Heiser
2921d3cd17 [mod] add artwork to mixcloud & soundcloud engines
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-19 21:59:12 +01:00
Markus Heiser
4a28b593c2 [fix] google images engine: Fix 'scrap_img_by_id' function
The 'scrap_img_by_id' function didn't return any longer anything useful.  This
fix allows the google images engine to present the full source image instead of
only the thumbnail.

The function scrap_img_by_id() is rpelaced by a fully rewrite to parse image
URLs by a regular expression. The new function parse_urls_img_from_js(dom)
returns a mapping of data-id to image URL.

Closes: https://github.com/searxng/searxng/issues/909
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-19 14:33:56 +01:00
Alexandre Flament
ace5401632
Merge pull request #900 from return42/fix-883
[fix] bandcamp: fix itemtype (album|track) and exceptions
2022-02-19 13:42:53 +01:00
Markus Heiser
943a7fdcb5 [mod] mediathekviewweb engine: add iframe_src and use videos template
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-19 00:50:54 +01:00
Markus Heiser
05c105b837 [fix] bandcamp: fix itemtype (album|track) and exceptions
BTW: polish implementation and show tracklist for albums

Closes: https://github.com/searxng/searxng/issues/883
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-18 22:44:43 +01:00
Markus Heiser
7352c6bc79 [mod] templates: rename field for <iframe> URL to iframe_src
Rename result field data_src to iframe_src

Suggested-by: @dalf https://github.com/searxng/searxng/pull/882#issuecomment-1037997402
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-18 19:00:49 +01:00
Markus Heiser
98cab4cf75 [mod] result_templates/default.html replace embedded HTML by data_src audio_src
Embedded HTML breaks SearXNG architecture.  To modularize, HTML is generated in
the templates (oscar & simple) and result parameter 'embedded' is replaced by
'data_src' (and 'audio_src'), an URL for embedded content (<iframe>).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-13 14:20:47 +01:00
Markus Heiser
46e131fdad [mod] result_templates/videos.html: replace embedded HTML by data_src
Embedded HTML breaks SearXNG architecture.  To modularize, HTML is generated in
the templates (oscar & simple) and result parameter 'embedded' is replaced by
'data_src', an URL for embedded content (<iframe>).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-13 14:20:47 +01:00
Émilien Devos
7d3e8118b0
Update the XPath for fetching the Google results 2022-02-09 14:34:14 +01:00
Markus Heiser
906a0a99cd [fix] openstreatmap: load thumbnail from uploads.wikimedia.org
Openstreatmap images are now loaded from uploads.wikimedia.org instead of
commons.wikimedia.org to prevent redirects.

With `image_proxy` enabled images from commons.wikimedia.org cant be loaded
since they are redirected.  We already discussed this issue [875] and
@tiekoetter fixed this issue in PR [878].

Related-to:
- [875] https://github.com/searxng/searxng/issues/875
- [878] https://github.com/searxng/searxng/pull/878
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-07 13:05:52 +01:00
Markus Heiser
a967e59590 [pylint] searx/engines/wikidata.py (no functional change)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-07 10:15:32 +01:00
Léon Tiekötter
1c151ae92b
[fix] wikidata: URL decoding and file extension handling
Add '.png' to the second img_src_name if it has the extension '.svg'.
Use urllib.parse.unquote for URL decoding.
2022-02-07 00:21:02 +01:00
Markus Heiser
a13c5d70c7 [fix] wikidata engine: select image with higher (not lower) priority
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-06 23:35:55 +01:00
Léon Tiekötter
a50f32bcfc
wikidata: load thumbnail instead of full image 2022-02-06 23:25:50 +01:00
Léon Tiekötter
560a14e77b
[fix] wikidata info box images
Wikidata info box images are now loaded from uploads.wikimedia.org instead of commons.wikimedia.org to prevent redirects

Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-06 22:16:06 +01:00
Markus Heiser
b35ef9789b [pylint] engines/invidious.py
Fix remarks from pylint and remove usless comments

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04 15:42:06 +01:00
Markus Heiser
e2ec6b4211 [fix] invidious engine: store random base_url in param
Two different threads ( = two different user queries) can call the request
function in a row and then the response function.  The namespace will be same
since this is the same engine.

To keep exactly the same value ``base_url`` must be stored in params and then
retrieve using ``resp.search_params["base_url"]``.

Suggested-by: @dalf https://github.com/searxng/searxng/pull/862#discussion_r799324861
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04 15:42:06 +01:00
Markus Heiser
ddc2102a07 [fix] solidtorrents engine: store random bas_url in param
Two different threads ( = two different user queries) can call the request
function in a row and then the response function.  The namespace will be same
since this is the same engine.

To keep exactly the same value ``base_url`` must be stored in params and then
retrieve using ``resp.search_params["base_url"]``.

Suggested-by: @dalf https://github.com/searxng/searxng/pull/862#discussion_r799324861
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04 14:55:21 +01:00
Markus Heiser
d6061b7c8a [mod] solidtorrents engine: add metadata & torrentfile
BTW: define min_len in eval_xpath_list of 'stats' list

Suggested-by: @dalf https://github.com/searxng/searxng/pull/862#pullrequestreview-872910744
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04 14:53:42 +01:00
Markus Heiser
f9c4868142 [fix] solidtorrents engine: use get_torrent_size from searx.utils
Suggested-by: @dalf https://github.com/searxng/searxng/pull/862#pullrequestreview-872858489
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04 14:53:42 +01:00
Markus Heiser
d92b3d96fd [fix] solidtorrents engine: JSON API no longer exists
The API endpoint, we where using does not exist anymore.  This patch is a
rewrite that parses the HTML page.

Related: https://github.com/paulgoio/searxng/issues/17
Closes: https://github.com/searxng/searxng/issues/858

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04 14:53:37 +01:00
Markus Heiser
50a56532c4 [pylint] engines/currency_convert.py
Fix remarks from pylint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-01 08:02:42 +01:00
Markus Heiser
15320b5eec [fix] engines description - currency_convert.py
Currency engine has DuckDuckGo metadata

In the engine selector of the preferences window, the currency search engine has
the same metadata and wikidata url as duckduckgo, I'd assume there should be a
difference of some sort there clarifying what source the currency uses or, if
it's a duckduckgo service, at least clarifying that it's a currency service by
duck duck go.

Closes: https://github.com/searxng/searxng/issues/787
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-31 23:17:28 +01:00
Markus Heiser
60e7fee47a
Merge pull request #475 from return42/tineye
[enh] engine - add Tineye reverse image search
2022-01-31 08:51:35 +01:00
Alexandre Flament
ebd3013a1a [mod] tineye engine: minor changes
* remove "disable: false" in settings.yml
* use the json() method from httpx.Response (faster character encoding detection)
2022-01-30 20:49:22 +01:00
Léon Tiekötter
a6673a1a94 [fix] 1x engine
1x changed the XML result layout.
2022-01-30 19:48:40 +01:00
Markus Heiser
a6b879f19c [mod] tineye engine: set engine_type to 'online_url_search'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-30 16:30:52 +01:00
Alexandre Flament
116802852d [fix] ina engine
based on a45408e8e2
2022-01-28 22:33:41 +01:00
Markus Heiser
b7f74fbe42 [mod] tineye - add some documentation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-28 09:06:44 +01:00
Allen
880555e263 [enh] engine - add Tineye reverse image search
Other optional parameter ..

`&sort=crawl_date`
    can be appended to search_string to sort results by date.

`&domain=example.org`
    can be implemented to search_string to get results from just one domain.

Public instances could get relatively fast timed-out for 3600s.

--

Merged from @allendema's commit [1] and slightly modfied / see [2].

Related-to: [1] 455b2b4460
Related-to: [2] https://github.com/searx/searx/pull/3040
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-28 09:06:44 +01:00
Léon Tiekötter
0cbf73a1f4
Allow 'using_tor_proxy' to be set for each engine individually
Check 'using_tor_proxy' for each engine individually instead of checking globally

[fix] searx.network: update _rdns test to the last httpx version

Co-authored-by: Alexandre Flament <alex@al-f.net>
2022-01-27 22:37:02 +01:00
Markus Heiser
1a0760c10a [fix] googel engine - "some results are invalids: invalid content"
Fix google issues listet in the `/stats?engine=google` and message::

    some results are invalids: invalid content

The log is::

    DEBUG   searx                         : result: invalid content: {'url': 'https://de.wikipedia.org/wiki/Foo', 'title': 'Foo - Wikipedia', 'content': None, 'engine': 'google'}
    WARNING searx.engines.google          : ErrorContext('searx/search/processors/abstract.py', 111, 'result_container.extend(self.engine_name, search_results)', None, 'some results are invalids: invalid content', ()) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-18 13:23:35 +01:00
Markus Heiser
f0102a95c9 [fix] google engine: remove adds and fix mobile_ui selector
1. Fix issue reported in comment [1]
2. Fix XPath selector for the response of google's mobile UI, reported in
   comment [2]

[1] https://github.com/searxng/searxng/pull/777#issuecomment-1015121322
[2] https://github.com/searxng/searxng/pull/777#issuecomment-1015236238

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-18 11:05:45 +01:00
Émilien Devos
6670063e0d
Update XPath for Google engine 2022-01-17 21:49:57 +00:00
Alexandre Flament
e07417848f
Merge pull request #695 from return42/fix-sp
[fix] startpage engine / modified API
2022-01-16 20:27:36 +01:00
Alexandre Flament
f9271d595f [fix] startpage: workaround to use the startpage network
workaround for the issue #762
2022-01-15 22:56:34 +01:00
Markus Heiser
bf593af423 [mod] engine mysql_server: make port configurable
Cherry piked from https://github.com/searx/searx/commit/82ac634070

Suggested-by: https://github.com/searx/searx/issues/3117
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-11 23:47:40 +01:00
Markus Heiser
df238e944c [mod] starpage engine: add comment about Startpage's FFox add-on
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:38 +01:00
Markus Heiser
21e884f369 [fix] startpage engine: fetch CAPTCHA & issues related to PR-695
In case of CAPTCHA raise a SearxEngineCaptchaException and suspend for 7 days.
When get_sc_code() fails raise a SearxEngineResponseException and suspend for 7
days.

[1] https://github.com/searxng/searxng/pull/695

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:38 +01:00
Markus Heiser
2f4e567e90 [fix] Get an actual sc argument from startpage's home page.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:38 +01:00
Markus Heiser
1cbcddb3f7 [pylint] Startpage engine
Fix remarks from pylint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:38 +01:00
Markus Heiser
f1f5e69c42 [fix] startpage engine - avoid captcha
Startpage has introduced new anti-scraping measures that make SearXNG instances
run into captchas:

1. some arguments has been removed and a new `sc` has been added.
2. search path changed from `do/search` to `sp/search`
3. POST request is no longer needed

Closes: https://github.com/searxng/searxng/issues/692
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:12 +01:00
Martin Fischer
576e19dad1 [fix] add default for "about" engine property
Fixes #732.
2022-01-10 08:40:06 +01:00
Markus Heiser
4fc5e5299c [fix] ccengine engine - avoid unwanted redirects
api.openverse.engineering is a little picky and wants to have a trailing slash
in the path:

    /v1/images? -->/ v1/images/?

otherwise it redirects, here is the debug log:

    DEBUG   searx.network.openverse       : HTTP Request: GET https://api.openverse.engineering/v1/images?&page=1&page_size=20&format=json&q=foo "HTTP/2 301 Moved Permanently" (text/html; charset=utf-8)
    DEBUG   searx.network.openverse       : HTTP Request: GET https://api.openverse.engineering/v1/images/?&page=1&page_size=20&format=json&q=foo "HTTP/2 200 OK" (application/json)
    WARNING searx.engines.openverse       : ErrorContext('searx/search/processors/online.py', 105, 'count_error(', None, '1 redirects, maximum: 0', ('200', 'OK', 'api.openverse.engineering')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 14:14:31 +01:00
Léon Tiekötter
37baf46ece [fix] Rename ccengine engine to openverse
The CC engine was merged with WordPress and renamed to Openverse

Source: https://wordpress.org/news/2021/05/welcome-to-openverse/
2022-01-07 13:06:05 +01:00
Léon Tiekötter
4be6deb0a1 [fix] ccengine engine
Change domain to api.openverse.engineering
2022-01-07 13:01:37 +01:00
Markus Heiser
ced656606f
Merge pull request #709 from return42/drop-etools
[fix] drop etools engine module
2022-01-07 11:18:47 +01:00
Markus Heiser
5dd3442f83 [fix] drop etools engine module
The implementation of the etools engine is poor.  No date-range support, no
language support and it is broken by a CAPTCHA.

etools is a metasearch engine, the major search engines it supports (google,
bing, wikipedia, Yahoo) are already available in SeaarXNG.

While etools does support several engines we currently don't support directly,
support for them should be added directly to SearXNG if there is demand.

In practice: in SearXNG the worse etools results will be mixed with good results
from other engines we have (as long as there is no captcha).

At best case, what we win with etools is in e.g. results from de.ask.com in a
query from a german request .. in all other cases worse results are bubble up in
SearXNG's result list.

[1] https://github.com/searxng/searxng/issues/696#issuecomment-1005855499

Closes: https://github.com/searxng/searxng/issues/696
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 10:41:09 +01:00
Martin Fischer
e12525a1fa
Merge pull request #708 from not-my-profile/pref-refactor
Refactor `preferences`
2022-01-07 09:45:23 +01:00
Léon Tiekötter
3ab826de22 Drop microsoft academic engine
Microsoft academic was discontinued on 2021-12-31.

Source: https://www.microsoft.com/en-us/research/project/academic/articles/microsoft-academic-to-expand-horizons-with-community-driven-approach/
2022-01-07 01:35:13 +01:00
Martin Fischer
bb06758a7b [refactor] add type hints & remove Setting._post_init
Previously the Setting classes used a horrible _post_init
hack that prevented proper type checking.
2022-01-06 14:21:14 +01:00
Alexandre Flament
aedd6279b3
Merge pull request #634 from not-my-profile/powered-by
Introduce `categories_as_tabs` & group engines in tabs
2022-01-06 09:22:02 +01:00
Alexandre Flament
d3ecadd3f8
Merge pull request #679 from dalf/brand-searxng
searxng.org: update setup.py & settings.yml
2022-01-05 19:07:53 +01:00
Martin Fischer
d01e8aa8cc [mod] introduce searx.engines.Engine for type hinting 2022-01-05 11:03:44 +01:00
Martin Fischer
1e195f5b95 [mod] move group_engines_in_tab to searx.webutils 2022-01-05 11:03:44 +01:00
Martin Fischer
5d74bf3820 [enh] move dictionaries, Erowid & IMDb out of general category
The general category is the category that is searched by default.
From a privacy standpoint it doesn't make sense to send all general
queries to specialized search engines that cannot deal with those
queries anyway.
2022-01-05 11:03:44 +01:00
Martin Fischer
ab90e2ac49 [enh] show categories not in any tab category in "Other" preferences tab
Previously we didn't have a good place to put search engines that don't
fit into any of the tab categories. This commit automatically puts
search engines that don't belong to any tab category in an "other"
category, that is only displayed in the user preferences (and not above
search results).
2022-01-05 11:03:44 +01:00
Martin Fischer
b02f762687 [enh] add more categories 2022-01-05 11:00:11 +01:00
Martin Fischer
8e9ad1ccc2 [enh] introduce categories_as_tabs
Previously all categories were displayed as search engine tabs.
This commit changes that so that only the categories listed under
categories_as_tabs in settings.yml are displayed.

This lets us introduce more categories without cluttering up the UI.
Categories not displayed as tabs  can still be searched with !bangs.
2022-01-03 07:01:49 +01:00
Martin Fischer
df34b1ddcf [enh] settings.yml: allow granular overwrites for about 2022-01-03 07:01:49 +01:00
Alexandre Flament
d83aa2b0d2
Merge pull request #613 from return42/pylint-bing-images
[pylint] Bing (Images) engine
2022-01-02 22:00:55 +01:00
Alexandre Flament
76cbfbbdda reference docs.searxng.org 2022-01-02 21:18:29 +01:00
Markus Heiser
61ce0c2244 [fix] bing engines: fetch_supported_languages
The Request to and the Response from https://www.bing.com/account/general has
been changed.

[1] https://github.com/searxng/searxng/pull/672#discussion_r777104919

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-01 17:31:38 +01:00
Markus Heiser
dc4f1f705d [pylint] Bing (Images) engine
Fix remarks from pylint and remove obsolete try/except block

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-28 14:43:39 +01:00
Markus Heiser
6d7a38a912 [pylint] Bing (Videos) engine
Fix remarks from pylint and remove obsolete try/except block

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-28 14:33:05 +01:00
Markus Heiser
d84226bf63 [fix] issues reported by pylint
Fix pylint issues from commit (3d96a983)

    [format.python] initial formatting of the python code

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-27 10:16:20 +01:00
Markus Heiser
3d96a9839a [format.python] initial formatting of the python code
This patch was generated by black [1]::

    make format.python

[1] https://github.com/psf/black

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-27 09:26:22 +01:00
Markus Heiser
fcdc2c2cd2 [format.python] disable py code formatting for some hunks of code
Disable the python code formatting from python-black, where the readability of
code suffers by formatting.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-27 09:16:03 +01:00
Martin Fischer
e28c6bda35 [doc] introduce about.language and sort engines by it 2021-12-21 09:58:51 +01:00
Markus Heiser
7a215e07e7
Merge pull request #611 from return42/fix-bing
[fix] bing engine: fix paging support, show inital page.
2021-12-20 10:08:52 +01:00
Markus Heiser
2af50c2588 [pylint] Reddit engine
Add Reddit engine to pylint process

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-18 17:59:47 +01:00
Markus Heiser
6b85607274 [fix] bing engine: fix paging support, show inital page.
Follow up queries for the pages needed to be fixed.

- Split search-term in one for initial query and one for following queries.
- Set some headers in HTTP requests, bing needs for paging support.
- IMO //div[@class="sa_cc"] does no longer match in a bing response.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-18 13:50:38 +01:00
Markus Heiser
b2177e5916 [pylint] Bing (Web) engine
Fix remarks from pylint and improved code-style.  In preparation for a bug-fix
of the Bing (Web) engine I add this engine to the pylint-list.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-18 13:40:36 +01:00
Markus Heiser
f41734a543 [fix] engine bing-news: replace the http:// by https://
BTW: add bing_news to the pylint process

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-17 13:25:50 +01:00
Markus Heiser
8cc7c880ae
Merge pull request #587 from dalf/fix-gigablast
[fix] gigablast engine
2021-12-12 15:58:13 +01:00
Markus Heiser
b5c9cc4ff3
Merge pull request #586 from dalf/remove-yggtorrent
[del] remove yggtorrent
2021-12-07 07:00:47 +01:00
Alexandre Flament
1a6207574e [fix] gigablast engine
fetch extra params after 3000 seconds
2021-12-06 22:55:15 +01:00
Alexandre Flament
fbc2a6ab4b [del] remove yggtorrent
yggtorrent is behind cloudflare now
close #580
2021-12-06 21:59:51 +01:00
Alexandre Flament
037cb7dd3d [fix] imdb: don't crash when there is no result 2021-12-06 21:49:18 +01:00
Markus Heiser
6e06618e0c [fix] google-videos engine: ignore news articles
In the video search, google also sometimes includes news.  E.g. in the DE
language when you search for `!gov paris`, google adds an article from a german
newspaper (FAZ), I assume these are sponsored link (not tagged advertisement?)

Those links do not have an image / this patch ignores *video links* wqithout an
image ID.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-26 17:11:20 +01:00
Markus Heiser
1ce09df9aa [fix] google video engine - rework of the HTML parser
The google video response has been changed slightly, a rework of the parser was
needed.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-26 01:14:17 +01:00
Markus Heiser
488ace1da9 [fix] google engine - suggestion
BTW: google no longer offers *spelling suggestions*

Closes: https://github.com/searxng/searxng/issues/442
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-25 19:42:03 +01:00
Markus Heiser
5b28c9109f [fix] google images: @href index 0 not found
Sometimes there is no href in the `<a ..>` tag of a *link_node* [1].

[1] https://github.com/searxng/searxng/issues/532

Reported-by: @TheEssem
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-21 09:55:59 +01:00
Markus Heiser
4c82ac7670 [drop] engine digg - https://digg.com/api is no longer available
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-19 15:00:22 +01:00
Tom
e1d60051ca
[fix] Qwant search query string
Search string: "!qwant time"
Resulting request URL: https://api.qwant.com/v3/search/web?q=q=time&count=10&offset=0&device=desktop&safesearch=1&locale=en_US
Notice the double "q="

Resulting request URL after fix: https://api.qwant.com/v3/search/web?q=time&count=10&offset=0&device=desktop&safesearch=1&locale=en_US
2021-11-17 18:13:54 +01:00
MrPaulBlack
41494d9f47 [fix] make reddit only in social media category avail.
fix https://github.com/searxng/searxng/issues/470
2021-11-01 20:37:17 +01:00
Alexandre Flament
64b29ad838 [mod] microsoft academic: increase timeout to 6 seconds
also avoid a crash when there is no result

close #433
2021-10-26 12:26:43 +02:00
Markus Heiser
713814547a [fix] yahoo engine - don't lump all search suggestions together
Closes: https://github.com/searxng/searxng/issues/421
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-21 07:51:05 +00:00
Markus Heiser
f63ffbb22b [fix] engine - yahoo: rewrite and fix issues
Languages are supported by mapping the language to a domain.  If domain is not
found in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.

BTW: fix issue reported at https://github.com/searx/searx/issues/3020

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-16 20:05:26 +00:00
Markus Heiser
38a157b56f [pylint] engines: yahoo fix several issues reported from pylint
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-16 20:05:26 +00:00
MrPaulBlack
00b0394e19 [fix] language param for qwant 2021-10-14 16:11:44 +00:00
Noémi Ványi
4cc1ee8565 [fix] qwant engine - only get results from categories
Reported-by: https://github.com/searx/searx/issues/3014
Cherry-picked: https://github.com/searx/searx/commit/3bcca43
2021-10-12 18:42:50 +00:00
Paolo Basso
64df011e2f [mod] engines - add zlibrary engine 2021-10-11 14:58:44 +00:00
Markus Heiser
3abbe6d25b [fix] engine torznab - categories, before join convert int to str
BTW add init() function and replace SearxEngineAPIException by ValueError.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-07 15:27:55 +00:00
Markus Heiser
9fb77065bd [fix] engine torznab - marginal issues reported from linters
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-07 15:27:55 +00:00
Paolo Basso
d803df8d89 [mod] engines - add torznab WebAPI 2021-10-07 15:27:55 +00:00
Markus Heiser
19e41c137e [mod] set 'engine.supported_languages' from the origin python module
The key of the dictionary 'searx.data.ENGINES_LANGUAGES' is the *engine name*
configured in settings.xml.  When multiple engines are configured to use the
same origin engine (e.g. `engine: google`)::

    - name: google
      engine: google
      use_mobile_ui: false
      ...

    - name: google italian
      engine: google
      use_mobile_ui: false
      language: it
      ...

    - name: google mobile ui
      engine: google
      shortcut: gomui
      use_mobile_ui: true

There exists no entry for ENGINES_LANGUAGES[engine.name] (e.g. `name: google
mobile ui` or `name: google italian`).  This issue can be solved by recreate the
ENGINES_LANGUAGES::

    make data.languages

But this is nothing an SearXNG admin would like to do when just configuring
additional engines, since this just doubles entries in ENGINES_LANGUAGES and
BTW: `make data.languages` has various external requirements which might be not
installed or not available, on a production host.

With this patch, if engine.name fails, ENGINES_LANGUAGES[engine.engine] is used
to get the engine.supported_languages (e.g. `google` for the engine named
`google mobile`).

For an engine, when there is `language: ...` in the YAML settings, the engine
supports only one language, in this case engine.supported_languages should
contains this value defined in settings.yml (e.g. `it` for the engine named
`google italian`).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Closes: https://github.com/searxng/searxng/issues/384
2021-10-07 08:45:02 +02:00
Alexandre Flament
8a897b86f1 [mod] engines - IMDB: add thumbnails 2021-10-05 09:10:02 +02:00
Paul Alcock
823d44ed0a [mod] engines - add IMDB / Internet Movie Database
Merged from @Guilvareux's commit [1] and slightly modfied / see [2].

[1] https://github.com/searx/searx/pull/2980/commits/f2f90071
[2] https://github.com/searx/searx/pull/2980
2021-10-03 11:44:25 +02:00