Commit graph

532 commits

Author SHA1 Message Date
Markus Heiser
f2e16904a5 [mod] disable deezer engine by default
To play content from deezer a account is needed, the majority of the SearXNG
won't have.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-07 22:58:05 +01:00
Markus Heiser
e2ec6b4211 [fix] invidious engine: store random base_url in param
Two different threads ( = two different user queries) can call the request
function in a row and then the response function.  The namespace will be same
since this is the same engine.

To keep exactly the same value ``base_url`` must be stored in params and then
retrieve using ``resp.search_params["base_url"]``.

Suggested-by: @dalf https://github.com/searxng/searxng/pull/862#discussion_r799324861
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04 15:42:06 +01:00
Markus Heiser
d92b3d96fd [fix] solidtorrents engine: JSON API no longer exists
The API endpoint, we where using does not exist anymore.  This patch is a
rewrite that parses the HTML page.

Related: https://github.com/paulgoio/searxng/issues/17
Closes: https://github.com/searxng/searxng/issues/858

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04 14:53:37 +01:00
Alexandre Flament
f79b0fce06 [enh] limiter plugin
can replace filtron:
* rate limite the number of request per IP and per (IP, User-Agent)
* block some bots

use Redis
data stored in Redis never contains the IP addresses, only HMAC using the secret_key

Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-02 09:15:59 +01:00
Alexandre Flament
ebd3013a1a [mod] tineye engine: minor changes
* remove "disable: false" in settings.yml
* use the json() method from httpx.Response (faster character encoding detection)
2022-01-30 20:49:22 +01:00
Markus Heiser
a6b879f19c [mod] tineye engine: set engine_type to 'online_url_search'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-30 16:30:52 +01:00
Allen
880555e263 [enh] engine - add Tineye reverse image search
Other optional parameter ..

`&sort=crawl_date`
    can be appended to search_string to sort results by date.

`&domain=example.org`
    can be implemented to search_string to get results from just one domain.

Public instances could get relatively fast timed-out for 3600s.

--

Merged from @allendema's commit [1] and slightly modfied / see [2].

Related-to: [1] 455b2b4460
Related-to: [2] https://github.com/searx/searx/pull/3040
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-28 09:06:44 +01:00
Markus Heiser
ccffab3a43
Merge pull request #813 from MarcAbonce/change-wolfram-category
Move WolframAlpha away from Science category
2022-01-26 14:10:19 +01:00
Markus Heiser
10c9b9775d [mod] improve setup of invidious engine
- My experience is, that a timeout of 5 sec is not need, I got fast response
  less than a second.

- https://invidious.tube/ redirects to http://ww25.invidious.tube/
  - in SearXNG defaults the http protocol is unsafe and raise an error
  - https://ww25.invidious.tube has SSL_ERROR_UNSAFE_NEGOTIATION

Related-to: https://github.com/searxng/searxng/issues/821
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-25 19:02:11 +01:00
Marc Abonce Seguin
1311745f24 move WolframAlpha away from Science category 2022-01-23 23:24:02 -06:00
Markus Heiser
977e9a4330
Merge pull request #686 from return42/lib_redis
Add redis DB and connector
2022-01-11 19:55:14 +01:00
Léon Tiekötter
37baf46ece [fix] Rename ccengine engine to openverse
The CC engine was merged with WordPress and renamed to Openverse

Source: https://wordpress.org/news/2021/05/welcome-to-openverse/
2022-01-07 13:06:05 +01:00
Markus Heiser
ced656606f
Merge pull request #709 from return42/drop-etools
[fix] drop etools engine module
2022-01-07 11:18:47 +01:00
Markus Heiser
5dd3442f83 [fix] drop etools engine module
The implementation of the etools engine is poor.  No date-range support, no
language support and it is broken by a CAPTCHA.

etools is a metasearch engine, the major search engines it supports (google,
bing, wikipedia, Yahoo) are already available in SeaarXNG.

While etools does support several engines we currently don't support directly,
support for them should be added directly to SearXNG if there is demand.

In practice: in SearXNG the worse etools results will be mixed with good results
from other engines we have (as long as there is no captcha).

At best case, what we win with etools is in e.g. results from de.ask.com in a
query from a german request .. in all other cases worse results are bubble up in
SearXNG's result list.

[1] https://github.com/searxng/searxng/issues/696#issuecomment-1005855499

Closes: https://github.com/searxng/searxng/issues/696
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 10:41:09 +01:00
Léon Tiekötter
3ab826de22 Drop microsoft academic engine
Microsoft academic was discontinued on 2021-12-31.

Source: https://www.microsoft.com/en-us/research/project/academic/articles/microsoft-academic-to-expand-horizons-with-community-driven-approach/
2022-01-07 01:35:13 +01:00
Alexandre Flament
aedd6279b3
Merge pull request #634 from not-my-profile/powered-by
Introduce `categories_as_tabs` & group engines in tabs
2022-01-06 09:22:02 +01:00
Markus Heiser
0ebad8220f
Merge pull request #688 from dalf/settings_enable_stats
[enh] settings.yml: implement general.enable_metrics
2022-01-05 18:53:49 +00:00
Alexandre Flament
2134703b4b [enh] settings.yml: implement general.enable_metrics
* allow not to record metrics (response time, etc...)
* this commit doesn't change the UI. If the metrics are disabled
  /stats and /stats/errors will return empty response.
  in /preferences, the columns response time and reliability will be empty.
2022-01-05 19:03:04 +01:00
Martin Fischer
a4c2cfb837 [enh] change categories_as_tabs from a list to a dict
The tab icon names are currently hard coded in the templates.
This commit lets us introduce an icon property in the future, e.g:

categories_as_tabs:
  general:
    icon: search-outline
2022-01-05 11:03:44 +01:00
Martin Fischer
3f76c9c96c [enh] enable English dictionaries by default
These dictionaries are no longer part of the general category,
so they're no longer queried by default -> we can enable them
by default without degrading general query performance.
2022-01-05 11:03:44 +01:00
Martin Fischer
5d74bf3820 [enh] move dictionaries, Erowid & IMDb out of general category
The general category is the category that is searched by default.
From a privacy standpoint it doesn't make sense to send all general
queries to specialized search engines that cannot deal with those
queries anyway.
2022-01-05 11:03:44 +01:00
Martin Fischer
b02f762687 [enh] add more categories 2022-01-05 11:00:11 +01:00
Markus Heiser
d7b4306444 [fix] links in comments - https://docs.searxng.org/
fixed links from https://searxng.github.io/searxng/ to https://docs.searxng.org/
2022-01-04 16:04:49 +01:00
Markus Heiser
a6cfab93fa [enh] add redis connector searx/shared/redisdb.py
Add a redis connector, the default DB connector is a socket at::

    unix:///usr/local/searxng-redis/run/redis.sock?db=0

To set up a redis instance simply use::

    $ ./manage redis.build
    $ sudo -H ./manage redis.install

A hint for developers:

To get access rights to this instance, your developer account needs to be added
to the *searxng-redis* group::

    $ sudo -H ./manage redis.addgrp "${USER}"
    # don't forget to logout & login to get member of group

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-03 20:56:39 +01:00
Martin Fischer
8e9ad1ccc2 [enh] introduce categories_as_tabs
Previously all categories were displayed as search engine tabs.
This commit changes that so that only the categories listed under
categories_as_tabs in settings.yml are displayed.

This lets us introduce more categories without cluttering up the UI.
Categories not displayed as tabs  can still be searched with !bangs.
2022-01-03 07:01:49 +01:00
Alexandre Flament
76cbfbbdda reference docs.searxng.org 2022-01-02 21:18:29 +01:00
Martin Fischer
19339067c6 [del] remove geektimes (redirects to habrahabr) 2021-12-22 13:20:56 +01:00
Martin Fischer
fb037e7a83 [del] remove whaleslide (website is down)
Closes #639.
2021-12-22 13:19:27 +01:00
Martin Fischer
e28c6bda35 [doc] introduce about.language and sort engines by it 2021-12-21 09:58:51 +01:00
Martin Fischer
1fb1cca519 [doc] fix about.website for hoogle 2021-12-20 12:12:09 +01:00
Martin Fischer
06beffdf31 [doc] add about.website for wiby 2021-12-20 07:59:51 +01:00
Alexandre Flament
6bb6f7ffc1
Merge pull request #602 from return42/fix-600
[fix] move engines gpodder and AZlyrics into category music
2021-12-16 18:18:16 +01:00
Markus Heiser
07f2a9ee3c [fix] engine: MediathekViewWeb (mvw) disable by default
The MediathekViewWeb delivers only content for the german speaking area.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-15 19:03:35 +01:00
Markus Heiser
709482088e [fix] move engines gpodder and AZlyrics into category music
Closes: https://github.com/searxng/searxng/issues/600
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-15 09:43:56 +01:00
Markus Heiser
8cc7c880ae
Merge pull request #587 from dalf/fix-gigablast
[fix] gigablast engine
2021-12-12 15:58:13 +01:00
Alexandre Flament
1a6207574e [fix] gigablast engine
fetch extra params after 3000 seconds
2021-12-06 22:55:15 +01:00
Alexandre Flament
fbc2a6ab4b [del] remove yggtorrent
yggtorrent is behind cloudflare now
close #580
2021-12-06 21:59:51 +01:00
Alexandre Flament
42c1a938f0
Merge pull request #485 from dalf/pref_query_in_title
[mod] new preference: query_in_title
2021-11-24 13:10:27 +01:00
Alexandre Flament
d7eb373371 [fix] npm engine: don't ignore the first result
"from" parameter start at 0
2021-11-20 08:22:48 +01:00
Alexandre Flament
bc0cf9e83e
Merge pull request #521 from dalf/simple-theme-style-pref
Simple theme: allow the user to choose auto, light, dark style
2021-11-19 17:14:35 +01:00
Alexandre Flament
ec5a82fccd [enh] simple theme: add "simple-style" preferences 2021-11-19 16:45:50 +01:00
Markus Heiser
86090e13d3 [mod] enable reddit - to have at least one in category social-media
Since digg no longer works, we do nat have a active engine in the social-media
category.  Enable reddit by default to have at least one engine back in this
category.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-19 15:06:42 +01:00
Markus Heiser
4c82ac7670 [drop] engine digg - https://digg.com/api is no longer available
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-19 15:00:22 +01:00
Markus Heiser
4c2df9358a [fix] enigne library genesis - remove 'enable_http: true'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-19 13:25:08 +01:00
mrpaulblack
af79cce7cf [engines] move library of genesis from general to files category 2021-11-17 00:57:13 +01:00
Markus Heiser
6debda2404 [mod] settings.yml - polish up comments about plugins
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-13 15:40:00 +01:00
Alexandre Flament
80fb77476f [mod] new preference: query_in_title
* disable by default
* settings.yml: ui.query_in_title
* in /preferences: privacy tab

when enabled, the result page's title contains the user query.

previously:
* oscar theme: the query was always included
* simple theme: the query was included with the GET method
2021-11-08 21:40:53 +01:00
Alexandre Flament
64b29ad838 [mod] microsoft academic: increase timeout to 6 seconds
also avoid a crash when there is no result

close #433
2021-10-26 12:26:43 +02:00
Alexandre Flament
60db344cc0 [enh] reduce the available language list
close #406

settings['search']['languages'] contains the languages codes and 'all'
2021-10-26 10:27:05 +02:00
Paolo Basso
64df011e2f [mod] engines - add zlibrary engine 2021-10-11 14:58:44 +00:00