Commit graph

7771 commits

Author SHA1 Message Date
zutto
ca518c6803 add option to change yacy search mode 2023-08-03 19:58:51 +02:00
Markus Heiser
203f1f0928 [fix] engine piped: 'invalid content'
SearXNG does not allow a None value in the content field of a result item.

If the key (shortDescription, uploaderName) in the JSON response from piped
exists but is set to None, SearXNG ignores this result item::

  DEBUG   searx    : result: invalid content: { ..,  'content': None,  ..}

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-03 16:23:36 +02:00
Markus Heiser
207fcc0c8c [mod] engine piped: add paging support
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-03 16:23:36 +02:00
Markus Heiser
ef5831cd84 [mod] engine piped: split into two dedicated engiens for video & music
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-03 16:23:36 +02:00
Markus Heiser
7aa95d2d52 [doc] engine piped: add documentation to docs/dev/engines/online/
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-03 16:23:36 +02:00
Bnyro
636bfdac68 [feat] engine: implementation of Piped 2023-08-03 16:23:36 +02:00
dalf
8b4ba204b7 Update searx.data - update_engine_descriptions.py 2023-07-29 11:02:54 +02:00
dalf
d421385dc6 Update searx.data - update_wikidata_units.py 2023-07-29 11:02:23 +02:00
dalf
290f16207b Update searx.data - update_currencies.py 2023-07-29 11:02:14 +02:00
dalf
fcd2013511 Update searx.data - update_firefox_version.py 2023-07-29 11:01:28 +02:00
dalf
cfa8169509 Update searx.data - update_ahmia_blacklist.py 2023-07-29 09:25:24 +02:00
Markus Heiser
0623d5ae76 [doc] reduce copyright remark in the footer to the SearXNG team
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-07-29 09:18:14 +02:00
dependabot[bot]
112e4c08fd Bump sphinx from 7.0.1 to 7.1.1
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 7.0.1 to 7.1.1.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/master/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v7.0.1...v7.1.1)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-29 09:18:14 +02:00
searxng-bot
f45d1920d9 [translations] update from Weblate
ba4888c96 - 2023-07-26 - return42 <markus.heiser@darmarit.de>
6ec8a8a28 - 2023-07-22 - return42 <markus.heiser@darmarit.de>
0a7b701b3 - 2023-07-24 - artnay <jiri.gronroos@iki.fi>
c0b34cbdb - 2023-07-23 - MonsoonFire <re1qnb5mq@mozmail.com>
37cbd41c2 - 2023-07-22 - return42 <markus.heiser@darmarit.de>
2023-07-29 08:15:21 +02:00
Markus Heiser
b8fe71a1c2
Merge pull request #2569 from searxng/dependabot/pip/certifi-2023.7.22
Bump certifi from 2023.5.7 to 2023.7.22
2023-07-29 08:13:37 +02:00
dependabot[bot]
d3c3259bc7 Bump pylint from 2.17.4 to 2.17.5
Bumps [pylint](https://github.com/pylint-dev/pylint) from 2.17.4 to 2.17.5.
- [Release notes](https://github.com/pylint-dev/pylint/releases)
- [Commits](https://github.com/pylint-dev/pylint/compare/v2.17.4...v2.17.5)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-29 08:05:11 +02:00
dependabot[bot]
b0f7d18274
Bump certifi from 2023.5.7 to 2023.7.22
Bumps [certifi](https://github.com/certifi/python-certifi) from 2023.5.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2023.05.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-25 23:08:25 +00:00
mrpaulblack
65d8b1a310 [fix] remove disabled: false from engine definitions in settings.yml
* setting disabled: false is not needed, since it is by default enabled
2023-07-22 18:19:01 +02:00
searxng-bot
51c531d450 [translations] update from Weblate
b7f1e9ae - 2023-07-17 - Hudobni Volk <hudobni.volk@tuta.io>
3c7c821e - 2023-07-16 - alextecplayz <alextec70@outlook.com>
0e305f84 - 2023-07-17 - return42 <markus.heiser@darmarit.de>
80745a22 - 2023-07-15 - tentsbet <remendne@pentrens.jp>
afef0e2e - 2023-07-16 - Salif Mehmed <mail@salif.eu>
4a7687ac - 2023-07-14 - Ivan Gabaldon <admin@inetol.net>
2023-07-22 17:47:05 +02:00
dependabot[bot]
a259bd4d4e Bump pyyaml from 6.0 to 6.0.1
Bumps [pyyaml](https://github.com/yaml/pyyaml) from 6.0 to 6.0.1.
- [Changelog](https://github.com/yaml/pyyaml/blob/6.0.1/CHANGES)
- [Commits](https://github.com/yaml/pyyaml/compare/6.0...6.0.1)

---
updated-dependencies:
- dependency-name: pyyaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-22 17:46:12 +02:00
Paul Braeuning
a446dea1bb
Merge pull request #2535 from KiruPoruno/master
theme: fix "#backToTop" button always being clickable
2023-07-19 15:12:57 +02:00
mrpaulblack
b477349824 [build] /static 2023-07-19 15:07:45 +02:00
Kiru
de5c1cedca fix "#backToTop" button always being clickable
`pointer-events` never gets set to "none" when the button is hidden,
allowing you to click the button. And your mouse further changes it's
cursor to the pointer style.
2023-07-19 15:06:09 +02:00
dependabot[bot]
4380fc2bac Bump mock from 5.0.2 to 5.1.0
Bumps [mock](https://github.com/testing-cabal/mock) from 5.0.2 to 5.1.0.
- [Changelog](https://github.com/testing-cabal/mock/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/testing-cabal/mock/compare/5.0.2...5.1.0)

---
updated-dependencies:
- dependency-name: mock
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-14 10:21:45 +02:00
searxng-bot
b7b184244d [translations] update from Weblate
01350cf1 - 2023-07-13 - return42 <markus.heiser@darmarit.de>
5f037a4d - 2023-07-12 - return42 <markus.heiser@darmarit.de>
820a78ad - 2023-07-12 - return42 <markus.heiser@darmarit.de>
73037743 - 2023-07-12 - return42 <markus.heiser@darmarit.de>
e656795c - 2023-07-09 - Linerly <linerly@protonmail.com>
0ee18285 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
c087c7fb - 2023-07-08 - return42 <markus.heiser@darmarit.de>
6eb318c5 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
3b4a3d1f - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b3187499 - 2023-07-09 - return42 <markus.heiser@darmarit.de>
c1226646 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
2356a402 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
c9a74b52 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
3d9f2938 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
68af8585 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
261a2a72 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
fcea15cf - 2023-07-08 - return42 <markus.heiser@darmarit.de>
7685385e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
ec0a3727 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
0130ddf7 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b93f9609 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
4a5cdcb3 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
9cba3939 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
d973d937 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
ce076245 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
5c36ccab - 2023-07-08 - return42 <markus.heiser@darmarit.de>
226ff7d4 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
8148a9ed - 2023-07-08 - return42 <markus.heiser@darmarit.de>
840bc189 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
51ffc22e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
394ec63e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
428c16a8 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
218cf51e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
70260934 - 2023-07-09 - ghose <correo@xmgz.eu>
c6244c2b - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b92dc5c1 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
43917957 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
df1bf630 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
d1c00dff - 2023-07-08 - return42 <markus.heiser@darmarit.de>
0a6da54f - 2023-07-08 - return42 <markus.heiser@darmarit.de>
12377e28 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b5b8ea78 - 2023-07-07 - gallegonovato <fran-carro@hotmail.es>
ec31e65f - 2023-07-08 - return42 <markus.heiser@darmarit.de>
6c33b1fe - 2023-07-08 - return42 <markus.heiser@darmarit.de>
393d390c - 2023-07-08 - return42 <markus.heiser@darmarit.de>
a4f6b353 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
0f8d6b6b - 2023-07-08 - return42 <markus.heiser@darmarit.de>
67f2fc96 - 2023-07-08 - Fjuro <ifjuro@proton.me>
5f2d3f02 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
5ae2b8dc - 2023-07-08 - return42 <markus.heiser@darmarit.de>
0bd4fb1e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
ce768726 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
a22ae2f2 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b5b8774f - 2023-07-08 - return42 <markus.heiser@darmarit.de>
2023-07-14 10:21:27 +02:00
Paolo Basso
cada89ee36 [feat] engine: re-enables z-library (zlibrary-global.se)
- re-enables z-library as the new domain zlibrary-global.se is now available
  from the open web.   The announcement of the domain:

    https://www.reddit.com/r/zlibrary/comments/13whe08/mod_note_zlibraryglobalse_domain_is_officially/

  It is an official domain, it requires to log in to the "personal" subdomain
  only to download files, but the search works.

- changes the result template of zlibrary to paper.html, filling the appropriate fields
- implements language filtering for zlibrary
- implement zlibrary custom filters (engine traits)
- refactor and document the zlibrary engine
2023-07-07 21:36:51 +02:00
Hackurei
cb92767f19 [feat] enigine: add CrowdView forum search engine 2023-07-07 21:36:11 +02:00
dependabot[bot]
535c6ea39c Bump lxml from 4.9.2 to 4.9.3
Bumps [lxml](https://github.com/lxml/lxml) from 4.9.2 to 4.9.3.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.9.2...lxml-4.9.3)

---
updated-dependencies:
- dependency-name: lxml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-07 21:16:21 +02:00
dependabot[bot]
6ece9412eb Bump typing-extensions from 4.7.0 to 4.7.1
Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.7.0 to 4.7.1.
- [Release notes](https://github.com/python/typing_extensions/releases)
- [Changelog](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/python/typing_extensions/compare/4.7.0...4.7.1)

---
updated-dependencies:
- dependency-name: typing-extensions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-07 21:14:06 +02:00
searxng-bot
4a2f310da3 [translations] update from Weblate
152f2008 - 2023-07-05 - return42 <markus.heiser@darmarit.de>
9dbf6b22 - 2023-07-01 - return42 <markus.heiser@darmarit.de>
4ad4c00f - 2023-07-01 - Bananhylsa <thayer@hjemmeserver.net>
2023-07-07 21:13:47 +02:00
dependabot[bot]
55d238de2e Bump redis from 4.5.5 to 4.6.0
Bumps [redis](https://github.com/redis/redis-py) from 4.5.5 to 4.6.0.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](https://github.com/redis/redis-py/compare/v4.5.5...v4.6.0)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-05 07:40:27 +02:00
Markus Heiser
5720844fcd [doc] rearranges Settings & Engines docs for better readability
We have built up detailed documentation of the *settings* and the *engines* over
the past few years.  However, this documentation was still spread over various
chapters and was difficult to navigate in its entirety.

This patch rearranges the Settings & Engines documentation for better
readability.

To review new ordered docs::

   make docs.clean docs.live

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-07-01 22:45:19 +02:00
dependabot[bot]
8e8d8dabe9 Bump linuxdoc from 20230506 to 20230629
Bumps [linuxdoc](https://github.com/return42/linuxdoc) from 20230506 to 20230629.
- [Changelog](https://github.com/return42/linuxdoc/blob/master/CHANGELOG)
- [Commits](https://github.com/return42/linuxdoc/commits)

---
updated-dependencies:
- dependency-name: linuxdoc
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-30 11:49:43 +02:00
searxng-bot
81c9a18456 [translations] update from Weblate
2238e87b - 2023-06-28 - jenishngl <jenishngl+codeberg@gmail.com>
c70d228a - 2023-06-24 - nogb <u8cn71wq@yogibo.anonaddy.me>
389c0c62 - 2023-06-24 - return42 <markus.heiser@darmarit.de>
656d9fcb - 2023-06-23 - return42 <markus.heiser@darmarit.de>
a9c9b116 - 2023-06-25 - alma <alma@users.noreply.translate.codeberg.org>
528b845f - 2023-06-24 - nogb <u8cn71wq@yogibo.anonaddy.me>
b8c50f23 - 2023-06-23 - return42 <markus.heiser@darmarit.de>
39f47c0f - 2023-06-23 - return42 <markus.heiser@darmarit.de>
ae0aa811 - 2023-06-24 - Fjuro <ifjuro@proton.me>
c8216259 - 2023-06-26 - lemonadeforlife <nahianlabiblimon44@gmail.com>
2023-06-30 11:49:07 +02:00
dependabot[bot]
69dd54ebf9 Bump typing-extensions from 4.6.3 to 4.7.0
Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.6.3 to 4.7.0.
- [Release notes](https://github.com/python/typing_extensions/releases)
- [Changelog](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/python/typing_extensions/compare/4.6.3...4.7.0)

---
updated-dependencies:
- dependency-name: typing-extensions
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-30 11:44:36 +02:00
Markus Heiser
e2917e64ff [mod] Upgrade Sphinx from 6.2.1 to 7.0.1
To upgrade Sphinx, MyST-Parser and markdown-it-py must also be updated at the
same time:

Closes: https://github.com/searxng/searxng/pull/2433
Closes: https://github.com/searxng/searxng/pull/2492
Closes: https://github.com/searxng/searxng/pull/2504
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-29 14:33:19 +02:00
dalf
fbb72fc1f4 Update searx.data - update_engine_descriptions.py 2023-06-29 13:59:25 +02:00
Markus Heiser
749b04ac1a
Merge [feat] engine: implementation of Anna's Archive
Anna's Archive [1] is a free non-profit online shadow library metasearch engine
providing access to a variety of book resources (also via IPFS), created by a
team of anonymous archivists [2].

[1] https://annas-archive.org/
[2] https://annas-software.org/AnnaArchivist/annas-archive
2023-06-29 13:56:19 +02:00
Markus Heiser
87e7926ae9 [fix] engine: Anna's Archive - grep results from '.js-scroll-hidden' elements
The renderuing of the WEB page is very strange; except the firts position all
other positions of Anna's result page are enclosed in SGML comments.  These
cooments are *uncommented* by some JS code, see query of the class
'.js-scroll-hidden' in Anna's HTML template [1].

[1] https://annas-software.org/AnnaArchivist/annas-archive/-/blob/main/allthethings/templates/macros/md5_list.html

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-29 09:32:57 +02:00
Markus Heiser
e2df6b77a3 [mod] engine: Anna's Archive - additionl settings (content, sort, ext)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-29 09:32:57 +02:00
Markus Heiser
eafc2906f1 [mod] engine: Anna's Archive - fetch search arguments from search form
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-29 09:32:57 +02:00
Paolo Basso
7adb9090e5 [mod] engine: Anna's Archive - add language support 2023-06-29 09:32:57 +02:00
Paolo Basso
e5637fe7b9 [feat] engine: implementation of Anna's Archive
Anna's Archive [1] is a free non-profit online shadow library metasearch engine
providing access to a variety of book resources (also via IPFS), created by a
team of anonymous archivists [2].

[1] https://annas-archive.org/
[2] https://annas-software.org/AnnaArchivist/annas-archive
2023-06-29 09:32:57 +02:00
Markus Heiser
fd26f37073 [upd] make data.all
- ahmia_blacklist.txt
- currencies.json
- engine_descriptions.json
- engine_traits.json
- osm_keys_tags.json
- useragents.json

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-28 21:21:53 +02:00
Markus Heiser
0ebff871a5 [fix] update_currencies.py - AttributeError: 'str' object has no attribute 'insert'
Replace lists with one item by the item, not before last currency has been
added.  In this traceback 'MXN' is added to 'pesos' while pesos is no longer a
list as the optimization was carried out too early.

    $ ./local/py3/bin/python searxng_extra/update/update_currencies.py
    Traceback (most recent call last):
      File "searxng_extra/update/update_currencies.py", line 164, in <module>
        main()
      File "searxng_extra/update/update_currencies.py", line 157, in main
        add_currency_name(db, "pesos", 'MXN')
      File "searxng_extra/update/update_currencies.py", line 89, in add_currency_name
        iso4217_set.insert(0, iso4217)
      AttributeError: 'str' object has no attribute 'insert'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-28 21:21:53 +02:00
Markus Heiser
efea962504 [fix] simple template: preferences - add missing icon_smal import
Related: https://github.com/searxng/searxng/commit/2149e88bdd64#r119535272
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-28 18:36:52 +02:00
Paolo Basso
401561cb58 [mod] engine torznab - refactor & option to hide links
- torznab engine using types and clearer code
- torznab option to hide torrent and magnet links.
- document the torznab engine
- add myself to authors

Closes: https://github.com/searxng/searxng/issues/1124
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-28 10:03:44 +02:00
Markus Heiser
da7c30291d [fix] Google API changed
It seems that Google is rolling out a modified WEB API [1][2].

In the past there was only the UI language in the `hl` argument but nowadays it
seems a combination of the UI language and the "search region" is mixed in this
argument and the `gl` argument has been removed.  I'm very surprised that google
is starting to mix the parameters of the UI with the parameters of the search
index.

This patch modifies the get_google_info(..) function.  Beside Google-WEB this
function is also used by other Google services, here are some examples to test
region & language of ..

- Google-WEB:    `!go dragon boat :en-CA`
- Google-News:   `!gon dragon boat :en-CA`
- Google-Videos: `!gov bmw :en-CA`
- Goolge-Images  `!goi bmw :en-CA`

- [1] https://github.com/searxng/searxng/issues/2515#issuecomment-1606294635
- [2] https://github.com/searxng/searxng/issues/2515#issuecomment-1607150817

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-26 18:28:09 +02:00
Markus Heiser
e8706fb738 [fix] engine & network issues / documentation and type annotations
This patch fixes some quirks and issues related to the engines and the network.
Each engine has its own network and this network was broken for the following
engines[1]:

- archlinux
- bing
- dailymotion
- duckduckgo
- google
- peertube
- startpage
- wikipedia

Since the files have been touched anyway, the type annotaions of the engine
modules has also been completed so that error messages from the type checker are
no longer reported.

Related and (partial) fixed issue:

- [1] https://github.com/searxng/searxng/issues/762#issuecomment-1605323861
- [2] https://github.com/searxng/searxng/issues/2513
- [3] https://github.com/searxng/searxng/issues/2515

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-25 13:58:26 +02:00
searxng-bot
2e4a435134 [translations] update from Weblate
9512b92a - 2023-06-23 - Coccocoas_Helper <coccocoahelper@gmail.com>
ca08c51e - 2023-06-23 - Coccocoas_Helper <coccocoahelper@gmail.com>
56ad4f21 - 2023-06-21 - return42 <markus.heiser@darmarit.de>
3ee419d6 - 2023-06-21 - return42 <markus.heiser@darmarit.de>
2023-06-23 09:34:46 +02:00