OwlBurst/Ponysearch
1
0
Fork 0
forked from Ponysearch/Ponysearch
Code Pull requests Activity
7270 commits 4 branches 17 tags 96 MiB
Commit graph

7270 commits

This branch
This branch
All branches
Author SHA1 Message Date
Markus Heiser
c1ae2ef57c [mod] qwant: fetch engine traits (data_type: traits_v1) 
Implements a fetch_traits function for the Qwant engines.

.. note::

   Includes migration of the request methode from 'supported_languages' to
   'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
fc0c775030 [mod] Dailymotion: fetch engine traits (data_type: supported_languages) 
Implements a fetch_traits function for the Dailymotion engine.

.. note::

   Does not include migration of the request methode from 'supported_languages'
   to 'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
61383edb27 [mod] Startpage: fetch engine traits (data_type: supported_languages) 
Implements a fetch_traits function for the Startpage engine.

.. note::

   Does not include migration of the request methode from 'supported_languages'
   to 'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
d3aa690a7a [mod] bing: fetch engine traits (data_type: supported_languages) 
Implements a fetch_traits function for the Bing engines.

.. note::

   Does not include migration of the request methode from 'supported_languages'
   to 'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
a7fe22770a [mod] Peertube: re-engineered & upgrade to data_type: traits_v1 
- fetch_traits(): Fetch languages from peertube's search-index source code.

  [mod] Include migration of the request methode from 'supported_languages'
        to 'traits' (EngineTraits) object.
  [fix] old supported_languages_url is no longer valid since the sources
        has been moved to a different path.

- fixed code to pass pylint
- request(): complete re-implementation based on the API docs [1]
- response(): complete re-implementation, adds serveral fields missed before
- add source code documentation

[1] https://docs.joinpeertube.org/api-rest-reference.html#tag/Search/operation/searchVideos

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
6e5f22e558 [mod] replace engines_languages.json by engines_traits.json 
Implementations of the *traits* of the engines.

Engine's traits are fetched from the origin engine and stored in a JSON file in
the *data folder*.  Most often traits are languages and region codes and their
mapping from SearXNG's representation to the representation in the origin search
engine.

To load traits from the persistence::

    searx.enginelib.traits.EngineTraitsMap.from_data()

For new traits new properties can be added to the class::

    searx.enginelib.traits.EngineTraits

.. hint::

   Implementation is downward compatible to the deprecated *supported_languages
   method* from the vintage implementation.

   The vintage code is tagged as *deprecated* an can be removed when all engines
   has been ported to the *traits method*.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
64fea2f9cb
Merge pull request #2280 from searxng/dependabot/pip/master/linuxdoc-20230321 
Bump linuxdoc from 20221127 to 20230321
 2023-03-24 10:22:08 +01:00
Markus Heiser
66ec5943e1
Merge pull request #2283 from searxng/revert-2279-dependabot/pip/master/redis-4.5.3 
Revert "Bump redis from 4.5.1 to 4.5.3"
 2023-03-24 10:18:40 +01:00
Markus Heiser
22a95f8771
Revert "Bump redis from 4.5.1 to 4.5.3" 2023-03-24 10:12:31 +01:00
Markus Heiser
183c084d36
Merge pull request #2279 from searxng/dependabot/pip/master/redis-4.5.3 
Bump redis from 4.5.1 to 4.5.3
 2023-03-24 10:05:03 +01:00
Markus Heiser
7b0aba8c51
Merge pull request #2281 from searxng/dependabot/pip/master/pylint-2.17.1 
Bump pylint from 2.17.0 to 2.17.1
 2023-03-24 09:43:04 +01:00
dependabot[bot]
76a432f0cd
Bump linuxdoc from 20221127 to 20230321 
Bumps [linuxdoc](https://github.com/return42/linuxdoc) from 20221127 to 20230321.
- [Release notes](https://github.com/return42/linuxdoc/releases)
- [Commits](https://github.com/return42/linuxdoc/commits)

---
updated-dependencies:
- dependency-name: linuxdoc
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-24 08:34:44 +00:00
Markus Heiser
ae62b25a8b
Merge pull request #2278 from searxng/dependabot/pip/master/yamllint-1.30.0 
Bump yamllint from 1.29.0 to 1.30.0
 2023-03-24 09:33:53 +01:00
dependabot[bot]
912f877dec
Bump ionicons from 6.1.3 to 7.1.0 in /searx/static/themes/simple 
Bumps [ionicons](https://github.com/ionic-team/ionicons) from 6.1.3 to 7.1.0.
- [Release notes](https://github.com/ionic-team/ionicons/releases)
- [Changelog](https://github.com/ionic-team/ionicons/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ionic-team/ionicons/compare/v6.1.3...v7.1.0)

---
updated-dependencies:
- dependency-name: ionicons
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-24 07:57:14 +00:00
dependabot[bot]
6de90867d0
Bump pylint from 2.17.0 to 2.17.1 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.17.0 to 2.17.1.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.17.0...v2.17.1)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-24 07:57:01 +00:00
dependabot[bot]
deb9b20bd2
Bump redis from 4.5.1 to 4.5.3 
Bumps [redis](https://github.com/redis/redis-py) from 4.5.1 to 4.5.3.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](https://github.com/redis/redis-py/compare/v4.5.1...v4.5.3)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-24 07:56:51 +00:00
dependabot[bot]
66289d8b72
Bump yamllint from 1.29.0 to 1.30.0 
Bumps [yamllint](https://github.com/adrienverge/yamllint) from 1.29.0 to 1.30.0.
- [Release notes](https://github.com/adrienverge/yamllint/releases)
- [Changelog](https://github.com/adrienverge/yamllint/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/adrienverge/yamllint/compare/v1.29.0...v1.30.0)

---
updated-dependencies:
- dependency-name: yamllint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-24 07:56:45 +00:00
Markus Heiser
0652eb420f
Merge pull request #2277 from searxng/translations_update 
Update translations
 2023-03-24 08:19:16 +01:00
searxng-bot
9f3a57c901 [translations] update from Weblate 
abfec8f4 - 2023-03-23 - return42 <markus.heiser@darmarit.de>
f02ea21c - 2023-03-23 - return42 <markus.heiser@darmarit.de>
3fc6c653 - 2023-03-20 - chenghui-lee <chlee9926@gmail.com>
342bbf46 - 2023-03-20 - return42 <markus.heiser@darmarit.de>
 2023-03-24 07:07:52 +00:00
Alexandre Flament
3841435587
Merge pull request #2274 from searxng/codespaces 
Add GitHub Codespaces configuration
 2023-03-22 08:17:34 +01:00
Markus Heiser
41823da1b2
Merge pull request #2272 from Solirs/Wikidata_desc 
wikidata: Add description for results
 2023-03-22 06:03:48 +01:00
Alexandre Flament
7b711cb15d Add GitHub Codespaces configuration 2023-03-21 20:58:13 +00:00
Solirs
ac169a0f75 Pass black formatting test 2023-03-21 00:41:36 +01:00
Solirs
e26bce33d4 WIKIDATA: Add description for results 2023-03-21 00:14:54 +01:00
Markus Heiser
9dc36965bb
Merge pull request #2268 from return42/editorconfig 
[enh] add .editorconfig
 2023-03-20 16:38:19 +01:00
Markus Heiser
60808fd3bd [enh] add .editorconfig 
EditorConfig [1] helps maintain consistent coding styles for multiple developers
working on the same project across various editors and IDEs.

The EditorConfig specification [2] support is pre installed in common IDEs [3]
and plugins for many others are available [4].

[1] https://editorconfig.org
[2] https://editorconfig-specification.readthedocs.io/
[3] https://editorconfig.org/#pre-installed
[4] https://editorconfig.org/#download

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-20 12:29:12 +01:00
Markus Heiser
b61b845951
Merge pull request #2266 from return42/shuffle-cipher 
[mod] Shuffle httpx's default ciphers of a SSL context randomly.
 2023-03-20 12:28:05 +01:00
Markus Heiser
70e3fa9441
Merge pull request #2204 from return42/readme 
[readme] reworked / partial merge of #2200
 2023-03-19 15:31:45 +01:00
Markus Heiser
94430e104c
Merge pull request #2238 from return42/fix-2027 
[fix] fix threshold in replace_auto_language
 2023-03-19 15:30:37 +01:00
Markus Heiser
f2962a2f4a
Merge pull request #2239 from return42/fix-eslintrc 
[fix] remove duplicate key in simple theme ESLint configuration
 2023-03-19 15:30:12 +01:00
Markus Heiser
8fa54ffddf [mod] Shuffle httpx's default ciphers of a SSL context randomly. 
From the analyse of @9Ninety [1] we know that DDG (and may be other engines / I
have startpage in mind) does some kind of TLS fingerprint to block bots.

This patch shuffles the default ciphers from httpx to avoid a cipher profile
that is known to httpx (and blocked by DDG).

[1] https://github.com/searxng/searxng/issues/2246#issuecomment-1467895556

----

From `What Is TLS Fingerprint and How to Bypass It`_

> When implementing TLS fingerprinting, servers can't operate based on a
> locked-in whitelist database of fingerprints.  New fingerprints appear
> when web clients or TLS libraries release new versions. So, they have to
> live off a blocklist database instead.
> ...
> It's safe to leave the first three as is but shuffle the remaining ciphers
> and you can bypass the TLS fingerprint check.

.. _What Is TLS Fingerprint and How to Bypass It:
   https://www.zenrows.com/blog/what-is-tls-fingerprint#how-to-bypass-tls-fingerprinting

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Closes: https://github.com/searxng/searxng/issues/2246
 2023-03-19 13:40:31 +01:00
Markus Heiser
677903c355
Merge pull request #2257 from Solirs/fix_bad_escape 
re.escape() the query in highlight_content to prevent a server side error.
 2023-03-17 08:54:54 +01:00
Solirs
fbb0e9d275 [fix] server side error: escape backslashes in the query highlight_content 
Any backslash escapes in the replacement are processed [1], backslashes should
be escaped [2].

[1] https://docs.python.org/3/library/re.html#re.sub
[2] https://docs.python.org/3/library/re.html#re.escape

closes:
- https://github.com/searxng/searxng/issues/2256
- https://github.com/searxng/searxng/issues/2250
 2023-03-17 08:46:00 +01:00
Markus Heiser
0e1010988d
Merge pull request #2262 from searxng/translations_update 
Update translations
 2023-03-17 08:41:29 +01:00
searxng-bot
86c3757872 [translations] update from Weblate 
32926a19 - 2023-03-15 - return42 <markus.heiser@darmarit.de>
7aabc876 - 2023-03-16 - Linerly <linerly@protonmail.com>
c0ed00f5 - 2023-03-14 - SonoAX <giovanniilgiovo@gmail.com>
6cf287f6 - 2023-03-13 - RhysJones <proladrhys123@outlook.com>
8c4c5f83 - 2023-03-12 - Cavemanly <k.adel.2m@protonmail.com>
dffe61fa - 2023-03-10 - return42 <markus.heiser@darmarit.de>
c7736cac - 2023-03-10 - BalkanMadman <zurabid2016@gmail.com>
e831b8e3 - 2023-03-10 - BalkanMadman <zurabid2016@gmail.com>
ef3c60af - 2023-03-10 - return42 <markus.heiser@darmarit.de>
c046a677 - 2023-03-07 - BalkanMadman <zurabid2016@gmail.com>
142041d6 - 2023-03-05 - return42 <markus.heiser@darmarit.de>
119b51df - 2023-03-05 - return42 <markus.heiser@darmarit.de>
 2023-03-17 07:07:53 +00:00
Alexandre Flament
3e9cddc606
rollback test 2023-03-15 19:55:20 +01:00
Alexandre Flament
41ed0ef0c7
test 2023-03-15 19:53:53 +01:00
Markus Heiser
f59ef7839a
Merge pull request #2214 from searxng/dependabot/pip/master/selenium-4.8.2 
Bump selenium from 4.8.0 to 4.8.2
 2023-03-15 18:50:51 +01:00
Markus Heiser
d4f328dacb
Merge pull request #2244 from searxng/dependabot/pip/master/myst-parser-1.0.0 
Bump myst-parser from 0.18.1 to 1.0.0
 2023-03-15 18:47:02 +01:00
dependabot[bot]
3f1762410c
Bump myst-parser from 0.18.1 to 1.0.0 
Bumps [myst-parser](https://github.com/executablebooks/MyST-Parser) from 0.18.1 to 1.0.0.
- [Release notes](https://github.com/executablebooks/MyST-Parser/releases)
- [Changelog](https://github.com/executablebooks/MyST-Parser/blob/master/CHANGELOG.md)
- [Commits](https://github.com/executablebooks/MyST-Parser/compare/v0.18.1...v1.0.0)

---
updated-dependencies:
- dependency-name: myst-parser
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-15 17:40:16 +00:00
Markus Heiser
097d092a7f
Merge pull request #2224 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2023-03-15 18:36:29 +01:00
Markus Heiser
85ef1af343
Merge pull request #2222 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2023-03-15 18:35:36 +01:00
Markus Heiser
60fdaba707
Merge pull request #2215 from searxng/dependabot/pip/master/markdown-it-py-2.2.0 
Bump markdown-it-py from 2.1.0 to 2.2.0
 2023-03-15 18:27:58 +01:00
Markus Heiser
e93e62582b
Merge pull request #2243 from searxng/dependabot/pip/master/pylint-2.17.0 
Bump pylint from 2.16.2 to 2.17.0
 2023-03-15 18:23:52 +01:00
Markus Heiser
2535e9d9ea
Merge pull request #2212 from searxng/translations_update 
Update translations
 2023-03-15 18:20:43 +01:00
dependabot[bot]
25fe7bf954
Bump pylint from 2.16.2 to 2.17.0 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.16.2 to 2.17.0.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.16.2...v2.17.0)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-10 07:56:36 +00:00
Markus Heiser
a7f1649190 [fix] remove duplicate key in simple theme ESLint configuration 
Partial merge of [PR-1736]

[PR-1736] https://github.com/searxng/searxng/pull/1736

Suggested-by: @FunctionalHacker in [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-05 09:12:13 +01:00
Markus Heiser
150a90c84e [fix] fix threshold in replace_auto_language 
[1] https://github.com/searxng/searxng/pull/2027#pullrequestreview-1322157677
[2] https://github.com/searxng/searxng/pull/1969#issuecomment-1345354529

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-05 08:29:58 +01:00
searxng-bot
1f36fc3a45 [translations] update from Weblate 
0d8ebfe1 - 2023-03-02 - AHOHNMYC <lqwh2h2cwa@protonmail.com>
1358dd6d - 2023-03-03 - mystery-z <07juwonc@kakao.com>
4d8c13db - 2023-03-01 - lhostfree951 <freeehost9191@gmail.com>
1ae581b6 - 2023-02-28 - tygyh <jonis9898@hotmail.com>
0003698f - 2023-02-28 - ewm <gnu.ewm@protonmail.com>
31c79617 - 2023-02-28 - gjveld <gjveld@gmail.com>
9015ec73 - 2023-02-28 - gallegonovato <fran-carro@hotmail.es>
03619a68 - 2023-02-25 - BalkanMadman <zurabid2016@gmail.com>
fa90585b - 2023-02-25 - BalkanMadman <zurabid2016@gmail.com>
c902c5e5 - 2023-02-26 - tentsbet <remendne@pentrens.jp>
 2023-03-03 07:08:08 +00:00
Alexandre Flament
abd2d8605e
Merge pull request #2231 from return42/fix-docs_prebuild 
[fix] make docs: NameError: name 'DOC' is not defined
 2023-03-02 18:37:25 +01:00