Commit graph

3534 commits

Author SHA1 Message Date
Markus Heiser
7cdd31440e [fix] external bangs: don't overwrite Bangs in data trie
Bangs with a `*` suffix (e.g. `!!d*`) overwrite Bangs with the same
prefix (e.g. `!!d`) [1].  This can be avoid when a non printable character is
used to tag a LEAF_KEY.

[1] https://github.com/searxng/searxng/pull/740#issuecomment-1010411888

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-12 19:37:13 +01:00
Markus Heiser
6d7e86eece [fix] get_bang_url: handle ambiguous !!bangs without error
An ambiguous bang like `!!d` raises an exception in function get_bang_url().  A
bang is only unique when the bang_definition from get_bang_definition_and_ac() is
a string / for a ambiguous bang the returned bang_definition is a dictionary.

Reported-by: user prg at #searxng:matrix.org on 2022/01/11
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-11 09:58:10 +01:00
Markus Heiser
3457408285 [build] /static 2022-01-10 17:42:13 +01:00
Markus Heiser
85159ad307 [fix] autocomple.js: register li.onmousedown instead li.onclick
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 17:30:57 +01:00
Markus Heiser
6420322cd1
Merge pull request #729 from k2s/fix-tokens
fix(security): filtered_engines were not correctly evaluating validate_token()
2022-01-10 11:51:45 +01:00
Martin Fischer
576e19dad1 [fix] add default for "about" engine property
Fixes #732.
2022-01-10 08:40:06 +01:00
Martin Minka
b76a815827 [fix] filtered_engines were not correctly evaluating validate_token() 2022-01-09 15:53:58 +01:00
Markus Heiser
4fc5e5299c [fix] ccengine engine - avoid unwanted redirects
api.openverse.engineering is a little picky and wants to have a trailing slash
in the path:

    /v1/images? -->/ v1/images/?

otherwise it redirects, here is the debug log:

    DEBUG   searx.network.openverse       : HTTP Request: GET https://api.openverse.engineering/v1/images?&page=1&page_size=20&format=json&q=foo "HTTP/2 301 Moved Permanently" (text/html; charset=utf-8)
    DEBUG   searx.network.openverse       : HTTP Request: GET https://api.openverse.engineering/v1/images/?&page=1&page_size=20&format=json&q=foo "HTTP/2 200 OK" (application/json)
    WARNING searx.engines.openverse       : ErrorContext('searx/search/processors/online.py', 105, 'count_error(', None, '1 redirects, maximum: 0', ('200', 'OK', 'api.openverse.engineering')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 14:14:31 +01:00
Markus Heiser
1461d6a07e [upd] engine_descriptions.json
Updated by:

    ./manage pyenv.cmd python3 \
        ./searxng_extra/update/update_engine_descriptions.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 13:59:03 +01:00
Léon Tiekötter
37baf46ece [fix] Rename ccengine engine to openverse
The CC engine was merged with WordPress and renamed to Openverse

Source: https://wordpress.org/news/2021/05/welcome-to-openverse/
2022-01-07 13:06:05 +01:00
Léon Tiekötter
4be6deb0a1 [fix] ccengine engine
Change domain to api.openverse.engineering
2022-01-07 13:01:37 +01:00
Markus Heiser
ced656606f
Merge pull request #709 from return42/drop-etools
[fix] drop etools engine module
2022-01-07 11:18:47 +01:00
Markus Heiser
5dd3442f83 [fix] drop etools engine module
The implementation of the etools engine is poor.  No date-range support, no
language support and it is broken by a CAPTCHA.

etools is a metasearch engine, the major search engines it supports (google,
bing, wikipedia, Yahoo) are already available in SeaarXNG.

While etools does support several engines we currently don't support directly,
support for them should be added directly to SearXNG if there is demand.

In practice: in SearXNG the worse etools results will be mixed with good results
from other engines we have (as long as there is no captcha).

At best case, what we win with etools is in e.g. results from de.ask.com in a
query from a german request .. in all other cases worse results are bubble up in
SearXNG's result list.

[1] https://github.com/searxng/searxng/issues/696#issuecomment-1005855499

Closes: https://github.com/searxng/searxng/issues/696
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 10:41:09 +01:00
Martin Fischer
e12525a1fa
Merge pull request #708 from not-my-profile/pref-refactor
Refactor `preferences`
2022-01-07 09:45:23 +01:00
Markus Heiser
2eb95f1532
Merge pull request #718 from searxng/dependabot/pip/master/pygments-2.11.2
Bump pygments from 2.11.0 to 2.11.2
2022-01-07 08:40:55 +01:00
Markus Heiser
c956ed6c43 [build] /static 2022-01-07 08:36:32 +01:00
Markus Heiser
d8db85638e make pygments.less
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 08:35:14 +01:00
searxng-bot
651342d71a [translations] update from Weblate
7781625e - 2022-01-06 - Linerly <linerly@protonmail.com>
dcab2ef1 - 2022-01-06 - Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 07:17:58 +00:00
Léon Tiekötter
3ab826de22 Drop microsoft academic engine
Microsoft academic was discontinued on 2021-12-31.

Source: https://www.microsoft.com/en-us/research/project/academic/articles/microsoft-academic-to-expand-horizons-with-community-driven-approach/
2022-01-07 01:35:13 +01:00
Martin Fischer
180d4d068b [refactor] refactor SwitchableSetting
The previous implementation used two hash sets and a list.
... that's not necessary ... a single hash map suffices.

And it's also less error prone ... because the previous data structure
allowed a setting to be enabled and disabled at the same time.
2022-01-06 18:56:02 +01:00
Martin Fischer
56fbf22108 [refactor] stop SwitchableSetting from subclassing Setting
Previously the default_value was abused for the cookie name.
Having SwitchableSetting subclass Setting doesn't even make sense
in the first place since none of the Setting methods apply.
2022-01-06 18:13:13 +01:00
Martin Fischer
1196abcfce
Merge pull request #706 from not-my-profile/remove-broken-quest-op
[fix] remove broken ? search operator
2022-01-06 17:44:34 +01:00
Martin Fischer
a0486cf827 [fix] prevent leaflet.{css,js} from polluting git grep results
Fixes #704.
2022-01-06 14:33:19 +01:00
Martin Fischer
83f8a8fc6d [refactor] remove pointless tuple 2022-01-06 14:21:15 +01:00
Martin Fischer
bb06758a7b [refactor] add type hints & remove Setting._post_init
Previously the Setting classes used a horrible _post_init
hack that prevented proper type checking.
2022-01-06 14:21:14 +01:00
Martin Fischer
61935c72ef [fix] remove broken ? search operator
The ? search operator has been broken for some time and
currently only raises the question why it's still there.

## Context ##

The query "Paris !images" searches for "Paris" in the "images" category.

Once upon a time Searx supported "Paris ?images" to search for "Paris"
in the currently enabled categories and the "images" category.

The feature makes sense ... the ? syntax does not.
We will hopefully introduce a +!images syntax in the future.

Fixes #702.
2022-01-06 14:10:58 +01:00
Alexandre Flament
03189d60f4
Merge pull request #697 from searxng/remove-nl-be
Delete searx/translations/nl_BE
2022-01-06 09:26:12 +01:00
Alexandre Flament
aedd6279b3
Merge pull request #634 from not-my-profile/powered-by
Introduce `categories_as_tabs` & group engines in tabs
2022-01-06 09:22:02 +01:00
Alexandre Flament
40684f6b5e
Delete searx/translations/nl_BE
See https://github.com/searxng/searxng/issues/540
2022-01-05 20:36:24 +01:00
Markus Heiser
0ebad8220f
Merge pull request #688 from dalf/settings_enable_stats
[enh] settings.yml: implement general.enable_metrics
2022-01-05 18:53:49 +00:00
Alexandre Flament
9004e84b6e
Merge pull request #596 from dalf/upgrade-httpx
Upgrade httpx
2022-01-05 19:13:48 +01:00
Alexandre Flament
d3ecadd3f8
Merge pull request #679 from dalf/brand-searxng
searxng.org: update setup.py & settings.yml
2022-01-05 19:07:53 +01:00
Alexandre Flament
2134703b4b [enh] settings.yml: implement general.enable_metrics
* allow not to record metrics (response time, etc...)
* this commit doesn't change the UI. If the metrics are disabled
  /stats and /stats/errors will return empty response.
  in /preferences, the columns response time and reliability will be empty.
2022-01-05 19:03:04 +01:00
Alexandre Flament
e64c3deab7 [mod] upgrade httpx 0.21.2
httpx 0.21.2 and httpcore 0.14.4 fix multiple issues:
* https://github.com/encode/httpx/releases/tag/0.21.2
* https://github.com/encode/httpcore/releases/tag/0.14.4

so most of the workarounds in searx.network have been removed.
2022-01-05 18:46:00 +01:00
Martin Fischer
a4c2cfb837 [enh] change categories_as_tabs from a list to a dict
The tab icon names are currently hard coded in the templates.
This commit lets us introduce an icon property in the future, e.g:

categories_as_tabs:
  general:
    icon: search-outline
2022-01-05 11:03:44 +01:00
Martin Fischer
b38036d519 [fix] stop setting superfluous cookies for "non tab categories" 2022-01-05 11:03:44 +01:00
Martin Fischer
3dd534e5c0 [refactor] make group_engines_in_tab more readable 2022-01-05 11:03:44 +01:00
Martin Fischer
d01e8aa8cc [mod] introduce searx.engines.Engine for type hinting 2022-01-05 11:03:44 +01:00
Martin Fischer
3f76c9c96c [enh] enable English dictionaries by default
These dictionaries are no longer part of the general category,
so they're no longer queried by default -> we can enable them
by default without degrading general query performance.
2022-01-05 11:03:44 +01:00
Martin Fischer
abe53946d4 [enh] display help text for "Other" engines tab in preferences 2022-01-05 11:03:44 +01:00
Martin Fischer
6f07d51b62 [mod] improve variable name 2022-01-05 11:03:44 +01:00
Martin Fischer
1e195f5b95 [mod] move group_engines_in_tab to searx.webutils 2022-01-05 11:03:44 +01:00
Martin Fischer
5d74bf3820 [enh] move dictionaries, Erowid & IMDb out of general category
The general category is the category that is searched by default.
From a privacy standpoint it doesn't make sense to send all general
queries to specialized search engines that cannot deal with those
queries anyway.
2022-01-05 11:03:44 +01:00
Martin Fischer
ab90e2ac49 [enh] show categories not in any tab category in "Other" preferences tab
Previously we didn't have a good place to put search engines that don't
fit into any of the tab categories. This commit automatically puts
search engines that don't belong to any tab category in an "other"
category, that is only displayed in the user preferences (and not above
search results).
2022-01-05 11:03:44 +01:00
Martin Fischer
4ac6b5d32d [build] /static 2022-01-05 11:03:44 +01:00
Martin Fischer
a12cb6a807 [simple] improve styling of subcategory table headers in preferences 2022-01-05 11:03:44 +01:00
Martin Fischer
20a5e43e35 [enh] display language of engines in preferences 2022-01-05 11:03:44 +01:00
Martin Fischer
31e206361f [enh] group engines in preference tabs 2022-01-05 11:03:44 +01:00
Martin Fischer
b02f762687 [enh] add more categories 2022-01-05 11:00:11 +01:00
Markus Heiser
d7b4306444 [fix] links in comments - https://docs.searxng.org/
fixed links from https://searxng.github.io/searxng/ to https://docs.searxng.org/
2022-01-04 16:04:49 +01:00