forked from Ponysearch/Ponysearch
[doc] rearranges Settings & Engines docs for better readability
We have built up detailed documentation of the *settings* and the *engines* over the past few years. However, this documentation was still spread over various chapters and was difficult to navigate in its entirety. This patch rearranges the Settings & Engines documentation for better readability. To review new ordered docs:: make docs.clean docs.live Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
Markus Heiser
Markus Heiser
parent
8e8d8dabe9
commit
5720844fcd
84 changed files with 1715 additions and 1414 deletions
|
|
@ -9,7 +9,7 @@ Buildhosts
|
||||||
If you have any contribution send us your :pull:`PR <../pulls>`, see
|
If you have any contribution send us your :pull:`PR <../pulls>`, see
|
||||||
:ref:`how to contribute`.
|
:ref:`how to contribute`.
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -1,79 +0,0 @@
|
||||||
.. _engine command:
|
|
||||||
|
|
||||||
====================
|
|
||||||
Command Line Engines
|
|
||||||
====================
|
|
||||||
|
|
||||||
.. sidebar:: info
|
|
||||||
|
|
||||||
- :origin:`command.py <searx/engines/command.py>`
|
|
||||||
- :ref:`offline engines`
|
|
||||||
|
|
||||||
With *command engines* administrators can run engines to integrate arbitrary
|
|
||||||
shell commands.
|
|
||||||
|
|
||||||
When creating and enabling a ``command`` engine on a public instance, you must
|
|
||||||
be careful to avoid leaking private data. The easiest solution is to limit the
|
|
||||||
access by setting ``tokens`` as described in section :ref:`private engines`.
|
|
||||||
|
|
||||||
The engine base is flexible. Only your imagination can limit the power of this
|
|
||||||
engine (and maybe security concerns). The following options are available:
|
|
||||||
|
|
||||||
``command``:
|
|
||||||
A comma separated list of the elements of the command. A special token
|
|
||||||
``{{QUERY}}`` tells where to put the search terms of the user. Example:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
['ls', '-l', '-h', '{{QUERY}}']
|
|
||||||
|
|
||||||
``delimiter``:
|
|
||||||
A mapping containing a delimiter ``char`` and the *titles* of each element in
|
|
||||||
``keys``.
|
|
||||||
|
|
||||||
``parse_regex``:
|
|
||||||
A dict containing the regular expressions for each result key.
|
|
||||||
|
|
||||||
``query_type``:
|
|
||||||
|
|
||||||
The expected type of user search terms. Possible values: ``path`` and
|
|
||||||
``enum``.
|
|
||||||
|
|
||||||
``path``:
|
|
||||||
Checks if the user provided path is inside the working directory. If not,
|
|
||||||
the query is not executed.
|
|
||||||
|
|
||||||
``enum``:
|
|
||||||
Is a list of allowed search terms. If the user submits something which is
|
|
||||||
not included in the list, the query returns an error.
|
|
||||||
|
|
||||||
``query_enum``:
|
|
||||||
A list containing allowed search terms if ``query_type`` is set to ``enum``.
|
|
||||||
|
|
||||||
``working_dir``:
|
|
||||||
|
|
||||||
The directory where the command has to be executed. Default: ``./``
|
|
||||||
|
|
||||||
``result_separator``:
|
|
||||||
The character that separates results. Default: ``\n``
|
|
||||||
|
|
||||||
The example engine below can be used to find files with a specific name in the
|
|
||||||
configured working directory:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
- name: find
|
|
||||||
engine: command
|
|
||||||
command: ['find', '.', '-name', '{{QUERY}}']
|
|
||||||
query_type: path
|
|
||||||
shortcut: fnd
|
|
||||||
delimiter:
|
|
||||||
chars: ' '
|
|
||||||
keys: ['line']
|
|
||||||
|
|
||||||
|
|
||||||
Acknowledgment
|
|
||||||
==============
|
|
||||||
|
|
||||||
This development was sponsored by `Search and Discovery Fund
|
|
||||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
|
||||||
|
|
@ -1,26 +0,0 @@
|
||||||
.. _engines and settings:
|
|
||||||
|
|
||||||
==================
|
|
||||||
Engines & Settings
|
|
||||||
==================
|
|
||||||
|
|
||||||
.. sidebar:: Further reading ..
|
|
||||||
|
|
||||||
- :ref:`settings engine`
|
|
||||||
- :ref:`engine settings` & :ref:`engine file`
|
|
||||||
|
|
||||||
.. toctree::
|
|
||||||
:maxdepth: 3
|
|
||||||
|
|
||||||
settings
|
|
||||||
|
|
||||||
.. toctree::
|
|
||||||
:maxdepth: 1
|
|
||||||
|
|
||||||
private-engines
|
|
||||||
recoll
|
|
||||||
sql-engines
|
|
||||||
nosql-engines
|
|
||||||
search-indexer-engines
|
|
||||||
command-line-engines
|
|
||||||
searx.engines.xpath
|
|
||||||
|
|
@ -1,49 +0,0 @@
|
||||||
.. _private engines:
|
|
||||||
|
|
||||||
============================
|
|
||||||
Private Engines (``tokens``)
|
|
||||||
============================
|
|
||||||
|
|
||||||
Administrators might find themselves wanting to limit access to some of the
|
|
||||||
enabled engines on their instances. It might be because they do not want to
|
|
||||||
expose some private information through :ref:`offline engines`. Or they would
|
|
||||||
rather share engines only with their trusted friends or colleagues.
|
|
||||||
|
|
||||||
To solve this issue the concept of *private engines* exists.
|
|
||||||
|
|
||||||
|
|
||||||
A new option was added to engines named `tokens`. It expects a list of
|
|
||||||
strings. If the user making a request presents one of the tokens of an engine,
|
|
||||||
they can access information about the engine and make search requests.
|
|
||||||
|
|
||||||
Example configuration to restrict access to the Arch Linux Wiki engine:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
- name: arch linux wiki
|
|
||||||
engine: archlinux
|
|
||||||
shortcut: al
|
|
||||||
tokens: [ 'my-secret-token' ]
|
|
||||||
|
|
||||||
|
|
||||||
Unless a user has configured the right token, the engine is going
|
|
||||||
to be hidden from him/her. It is not going to be included in the
|
|
||||||
list of engines on the Preferences page and in the output of
|
|
||||||
`/config` REST API call.
|
|
||||||
|
|
||||||
Tokens can be added to one's configuration on the Preferences page
|
|
||||||
under "Engine tokens". The input expects a comma separated list of
|
|
||||||
strings.
|
|
||||||
|
|
||||||
The distribution of the tokens from the administrator to the users
|
|
||||||
is not carved in stone. As providing access to such engines
|
|
||||||
implies that the admin knows and trusts the user, we do not see
|
|
||||||
necessary to come up with a strict process. Instead,
|
|
||||||
we would like to add guidelines to the documentation of the feature.
|
|
||||||
|
|
||||||
|
|
||||||
Acknowledgment
|
|
||||||
==============
|
|
||||||
|
|
||||||
This development was sponsored by `Search and Discovery Fund
|
|
||||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
|
||||||
|
|
@ -1,50 +0,0 @@
|
||||||
.. _engine recoll:
|
|
||||||
|
|
||||||
=============
|
|
||||||
Recoll Engine
|
|
||||||
=============
|
|
||||||
|
|
||||||
.. sidebar:: info
|
|
||||||
|
|
||||||
- `Recoll <https://www.lesbonscomptes.com/recoll/>`_
|
|
||||||
- `recoll-webui <https://framagit.org/medoc92/recollwebui.git>`_
|
|
||||||
- :origin:`searx/engines/recoll.py`
|
|
||||||
|
|
||||||
Recoll_ is a desktop full-text search tool based on Xapian. By itself Recoll_
|
|
||||||
does not offer WEB or API access, this can be achieved using recoll-webui_
|
|
||||||
|
|
||||||
|
|
||||||
Configuration
|
|
||||||
=============
|
|
||||||
|
|
||||||
You must configure the following settings:
|
|
||||||
|
|
||||||
``base_url``:
|
|
||||||
Location where recoll-webui can be reached.
|
|
||||||
|
|
||||||
``mount_prefix``:
|
|
||||||
Location where the file hierarchy is mounted on your *local* filesystem.
|
|
||||||
|
|
||||||
``dl_prefix``:
|
|
||||||
Location where the file hierarchy as indexed by recoll can be reached.
|
|
||||||
|
|
||||||
``search_dir``:
|
|
||||||
Part of the indexed file hierarchy to be search, if empty the full domain is
|
|
||||||
searched.
|
|
||||||
|
|
||||||
|
|
||||||
Example
|
|
||||||
=======
|
|
||||||
|
|
||||||
Scenario:
|
|
||||||
|
|
||||||
#. Recoll indexes a local filesystem mounted in ``/export/documents/reference``,
|
|
||||||
#. the Recoll search interface can be reached at https://recoll.example.org/ and
|
|
||||||
#. the contents of this filesystem can be reached though https://download.example.org/reference
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
base_url: https://recoll.example.org/
|
|
||||||
mount_prefix: /export/documents
|
|
||||||
dl_prefix: https://download.example.org
|
|
||||||
search_dir: ''
|
|
||||||
|
|
@ -1,136 +0,0 @@
|
||||||
====================
|
|
||||||
Local Search Engines
|
|
||||||
====================
|
|
||||||
|
|
||||||
.. sidebar:: further read
|
|
||||||
|
|
||||||
- `Comparison to alternatives
|
|
||||||
<https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html>`_
|
|
||||||
|
|
||||||
Administrators might find themselves wanting to integrate locally running search
|
|
||||||
engines. The following ones are supported for now:
|
|
||||||
|
|
||||||
* `Elasticsearch`_
|
|
||||||
* `Meilisearch`_
|
|
||||||
* `Solr`_
|
|
||||||
|
|
||||||
Each search engine is powerful, capable of full-text search. All of the engines
|
|
||||||
above are added to ``settings.yml`` just commented out, as you have to
|
|
||||||
``base_url`` for all them.
|
|
||||||
|
|
||||||
Please note that if you are not using HTTPS to access these engines, you have to enable
|
|
||||||
HTTP requests by setting ``enable_http`` to ``True``.
|
|
||||||
|
|
||||||
Furthermore, if you do not want to expose these engines on a public instance, you
|
|
||||||
can still add them and limit the access by setting ``tokens`` as described in
|
|
||||||
section :ref:`private engines`.
|
|
||||||
|
|
||||||
.. _engine meilisearch:
|
|
||||||
|
|
||||||
MeiliSearch
|
|
||||||
===========
|
|
||||||
|
|
||||||
.. sidebar:: info
|
|
||||||
|
|
||||||
- :origin:`meilisearch.py <searx/engines/meilisearch.py>`
|
|
||||||
- `MeiliSearch <https://www.meilisearch.com>`_
|
|
||||||
- `MeiliSearch Documentation <https://docs.meilisearch.com/>`_
|
|
||||||
- `Install MeiliSearch
|
|
||||||
<https://docs.meilisearch.com/learn/getting_started/installation.html>`_
|
|
||||||
|
|
||||||
MeiliSearch_ is aimed at individuals and small companies. It is designed for
|
|
||||||
small-scale (less than 10 million documents) data collections. E.g. it is great
|
|
||||||
for storing web pages you have visited and searching in the contents later.
|
|
||||||
|
|
||||||
The engine supports faceted search, so you can search in a subset of documents
|
|
||||||
of the collection. Furthermore, you can search in MeiliSearch_ instances that
|
|
||||||
require authentication by setting ``auth_token``.
|
|
||||||
|
|
||||||
Here is a simple example to query a Meilisearch instance:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
- name: meilisearch
|
|
||||||
engine: meilisearch
|
|
||||||
shortcut: mes
|
|
||||||
base_url: http://localhost:7700
|
|
||||||
index: my-index
|
|
||||||
enable_http: true
|
|
||||||
|
|
||||||
|
|
||||||
.. _engine elasticsearch:
|
|
||||||
|
|
||||||
Elasticsearch
|
|
||||||
=============
|
|
||||||
|
|
||||||
.. sidebar:: info
|
|
||||||
|
|
||||||
- :origin:`elasticsearch.py <searx/engines/elasticsearch.py>`
|
|
||||||
- `Elasticsearch <https://www.elastic.co/elasticsearch/>`_
|
|
||||||
- `Elasticsearch Guide
|
|
||||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html>`_
|
|
||||||
- `Install Elasticsearch
|
|
||||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html>`_
|
|
||||||
|
|
||||||
Elasticsearch_ supports numerous ways to query the data it is storing. At the
|
|
||||||
moment the engine supports the most popular search methods (``query_type``):
|
|
||||||
|
|
||||||
- ``match``,
|
|
||||||
- ``simple_query_string``,
|
|
||||||
- ``term`` and
|
|
||||||
- ``terms``.
|
|
||||||
|
|
||||||
If none of the methods fit your use case, you can select ``custom`` query type
|
|
||||||
and provide the JSON payload to submit to Elasticsearch in
|
|
||||||
``custom_query_json``.
|
|
||||||
|
|
||||||
The following is an example configuration for an Elasticsearch_ instance with
|
|
||||||
authentication configured to read from ``my-index`` index.
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
- name: elasticsearch
|
|
||||||
shortcut: es
|
|
||||||
engine: elasticsearch
|
|
||||||
base_url: http://localhost:9200
|
|
||||||
username: elastic
|
|
||||||
password: changeme
|
|
||||||
index: my-index
|
|
||||||
query_type: match
|
|
||||||
# custom_query_json: '{ ... }'
|
|
||||||
enable_http: true
|
|
||||||
|
|
||||||
.. _engine solr:
|
|
||||||
|
|
||||||
Solr
|
|
||||||
====
|
|
||||||
|
|
||||||
.. sidebar:: info
|
|
||||||
|
|
||||||
- :origin:`solr.py <searx/engines/solr.py>`
|
|
||||||
- `Solr <https://solr.apache.org>`_
|
|
||||||
- `Solr Resources <https://solr.apache.org/resources.html>`_
|
|
||||||
- `Install Solr <https://solr.apache.org/guide/installing-solr.html>`_
|
|
||||||
|
|
||||||
Solr_ is a popular search engine based on Lucene, just like Elasticsearch_. But
|
|
||||||
instead of searching in indices, you can search in collections.
|
|
||||||
|
|
||||||
This is an example configuration for searching in the collection
|
|
||||||
``my-collection`` and get the results in ascending order.
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
- name: solr
|
|
||||||
engine: solr
|
|
||||||
shortcut: slr
|
|
||||||
base_url: http://localhost:8983
|
|
||||||
collection: my-collection
|
|
||||||
sort: asc
|
|
||||||
enable_http: true
|
|
||||||
|
|
||||||
|
|
||||||
Acknowledgment
|
|
||||||
==============
|
|
||||||
|
|
||||||
This development was sponsored by `Search and Discovery Fund
|
|
||||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
|
||||||
|
|
@ -1,778 +0,0 @@
|
||||||
.. _settings.yml:
|
|
||||||
|
|
||||||
================
|
|
||||||
``settings.yml``
|
|
||||||
================
|
|
||||||
|
|
||||||
This page describe the options possibilities of the :origin:`searx/settings.yml`
|
|
||||||
file.
|
|
||||||
|
|
||||||
.. sidebar:: Further reading ..
|
|
||||||
|
|
||||||
- :ref:`use_default_settings.yml`
|
|
||||||
- :ref:`search API`
|
|
||||||
|
|
||||||
.. contents:: Contents
|
|
||||||
:depth: 2
|
|
||||||
:local:
|
|
||||||
:backlinks: entry
|
|
||||||
|
|
||||||
.. _settings location:
|
|
||||||
|
|
||||||
settings.yml location
|
|
||||||
=====================
|
|
||||||
|
|
||||||
The initial ``settings.yml`` we be load from these locations:
|
|
||||||
|
|
||||||
1. the full path specified in the ``SEARXNG_SETTINGS_PATH`` environment variable.
|
|
||||||
2. ``/etc/searxng/settings.yml``
|
|
||||||
|
|
||||||
If these files don't exist (or are empty or can't be read), SearXNG uses the
|
|
||||||
:origin:`searx/settings.yml` file. Read :ref:`settings use_default_settings` to
|
|
||||||
see how you can simplify your *user defined* ``settings.yml``.
|
|
||||||
|
|
||||||
|
|
||||||
.. _settings global:
|
|
||||||
|
|
||||||
Global Settings
|
|
||||||
===============
|
|
||||||
|
|
||||||
.. _settings brand:
|
|
||||||
|
|
||||||
``brand:``
|
|
||||||
----------
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
brand:
|
|
||||||
issue_url: https://github.com/searxng/searxng/issues
|
|
||||||
docs_url: https://docs.searxng.org
|
|
||||||
public_instances: https://searx.space
|
|
||||||
wiki_url: https://github.com/searxng/searxng/wiki
|
|
||||||
|
|
||||||
``issue_url`` :
|
|
||||||
If you host your own issue tracker change this URL.
|
|
||||||
|
|
||||||
``docs_url`` :
|
|
||||||
If you host your own documentation change this URL.
|
|
||||||
|
|
||||||
``public_instances`` :
|
|
||||||
If you host your own https://searx.space change this URL.
|
|
||||||
|
|
||||||
``wiki_url`` :
|
|
||||||
Link to your wiki (or ``false``)
|
|
||||||
|
|
||||||
.. _settings general:
|
|
||||||
|
|
||||||
``general:``
|
|
||||||
------------
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
general:
|
|
||||||
debug: false
|
|
||||||
instance_name: "SearXNG"
|
|
||||||
privacypolicy_url: false
|
|
||||||
donation_url: false
|
|
||||||
contact_url: false
|
|
||||||
enable_metrics: true
|
|
||||||
|
|
||||||
``debug`` : ``$SEARXNG_DEBUG``
|
|
||||||
Allow a more detailed log if you run SearXNG directly. Display *detailed* error
|
|
||||||
messages in the browser too, so this must be deactivated in production.
|
|
||||||
|
|
||||||
``donation_url`` :
|
|
||||||
Set value to ``true`` to use your own donation page written in the
|
|
||||||
:ref:`searx/info/en/donate.md <searx.infopage>` and use ``false`` to disable
|
|
||||||
the donation link altogether.
|
|
||||||
|
|
||||||
``privacypolicy_url``:
|
|
||||||
Link to privacy policy.
|
|
||||||
|
|
||||||
``contact_url``:
|
|
||||||
Contact ``mailto:`` address or WEB form.
|
|
||||||
|
|
||||||
``enable_metrics``:
|
|
||||||
Enabled by default. Record various anonymous metrics availabled at ``/stats``,
|
|
||||||
``/stats/errors`` and ``/preferences``.
|
|
||||||
|
|
||||||
.. _settings search:
|
|
||||||
|
|
||||||
``search:``
|
|
||||||
-----------
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
search:
|
|
||||||
safe_search: 0
|
|
||||||
autocomplete: ""
|
|
||||||
default_lang: ""
|
|
||||||
ban_time_on_fail: 5
|
|
||||||
max_ban_time_on_fail: 120
|
|
||||||
suspended_times:
|
|
||||||
SearxEngineAccessDenied: 86400
|
|
||||||
SearxEngineCaptcha: 86400
|
|
||||||
SearxEngineTooManyRequests: 3600
|
|
||||||
cf_SearxEngineCaptcha: 1296000
|
|
||||||
cf_SearxEngineAccessDenied: 86400
|
|
||||||
recaptcha_SearxEngineCaptcha: 604800
|
|
||||||
formats:
|
|
||||||
- html
|
|
||||||
|
|
||||||
``safe_search``:
|
|
||||||
Filter results.
|
|
||||||
|
|
||||||
- ``0``: None
|
|
||||||
- ``1``: Moderate
|
|
||||||
- ``2``: Strict
|
|
||||||
|
|
||||||
``autocomplete``:
|
|
||||||
Existing autocomplete backends, leave blank to turn it off.
|
|
||||||
|
|
||||||
- ``dbpedia``
|
|
||||||
- ``duckduckgo``
|
|
||||||
- ``google``
|
|
||||||
- ``startpage``
|
|
||||||
- ``swisscows``
|
|
||||||
- ``qwant``
|
|
||||||
- ``wikipedia``
|
|
||||||
|
|
||||||
``default_lang``:
|
|
||||||
Default search language - leave blank to detect from browser information or
|
|
||||||
use codes from :origin:`searx/languages.py`.
|
|
||||||
|
|
||||||
``languages``:
|
|
||||||
List of available languages - leave unset to use all codes from
|
|
||||||
:origin:`searx/languages.py`. Otherwise list codes of available languages.
|
|
||||||
The ``all`` value is shown as the ``Default language`` in the user interface
|
|
||||||
(in most cases, it is meant to send the query without a language parameter ;
|
|
||||||
in some cases, it means the English language) Example:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
languages:
|
|
||||||
- all
|
|
||||||
- en
|
|
||||||
- en-US
|
|
||||||
- de
|
|
||||||
- it-IT
|
|
||||||
- fr
|
|
||||||
- fr-BE
|
|
||||||
|
|
||||||
``ban_time_on_fail``:
|
|
||||||
Ban time in seconds after engine errors.
|
|
||||||
|
|
||||||
``max_ban_time_on_fail``:
|
|
||||||
Max ban time in seconds after engine errors.
|
|
||||||
|
|
||||||
``suspended_times``:
|
|
||||||
Engine suspension time after error (in seconds; set to 0 to disable)
|
|
||||||
|
|
||||||
``SearxEngineAccessDenied``: 86400
|
|
||||||
For error "Access denied" and "HTTP error [402, 403]"
|
|
||||||
|
|
||||||
``SearxEngineCaptcha``: 86400
|
|
||||||
For error "CAPTCHA"
|
|
||||||
|
|
||||||
``SearxEngineTooManyRequests``: 3600
|
|
||||||
For error "Too many request" and "HTTP error 429"
|
|
||||||
|
|
||||||
Cloudflare CAPTCHA:
|
|
||||||
- ``cf_SearxEngineCaptcha``: 1296000
|
|
||||||
- ``cf_SearxEngineAccessDenied``: 86400
|
|
||||||
|
|
||||||
Google CAPTCHA:
|
|
||||||
- ``recaptcha_SearxEngineCaptcha``: 604800
|
|
||||||
|
|
||||||
``formats``:
|
|
||||||
Result formats available from web, remove format to deny access (use lower
|
|
||||||
case).
|
|
||||||
|
|
||||||
- ``html``
|
|
||||||
- ``csv``
|
|
||||||
- ``json``
|
|
||||||
- ``rss``
|
|
||||||
|
|
||||||
|
|
||||||
.. _settings server:
|
|
||||||
|
|
||||||
``server:``
|
|
||||||
-----------
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
server:
|
|
||||||
base_url: http://example.org/location # change this!
|
|
||||||
port: 8888
|
|
||||||
bind_address: "127.0.0.1"
|
|
||||||
secret_key: "ultrasecretkey" # change this!
|
|
||||||
limiter: false
|
|
||||||
image_proxy: false
|
|
||||||
default_http_headers:
|
|
||||||
X-Content-Type-Options : nosniff
|
|
||||||
X-XSS-Protection : 1; mode=block
|
|
||||||
X-Download-Options : noopen
|
|
||||||
X-Robots-Tag : noindex, nofollow
|
|
||||||
Referrer-Policy : no-referrer
|
|
||||||
|
|
||||||
|
|
||||||
``base_url`` : ``$SEARXNG_URL`` :ref:`buildenv <make buildenv>`
|
|
||||||
The base URL where SearXNG is deployed. Used to create correct inbound links.
|
|
||||||
If you change the value, don't forget to rebuild instance's environment
|
|
||||||
(:ref:`utils/brand.env <make buildenv>`)
|
|
||||||
|
|
||||||
``port`` & ``bind_address``: ``$SEARXNG_PORT`` & ``$SEARXNG_BIND_ADDRESS`` :ref:`buildenv <make buildenv>`
|
|
||||||
Port number and *bind address* of the SearXNG web application if you run it
|
|
||||||
directly using ``python searx/webapp.py``. Doesn't apply to a SearXNG
|
|
||||||
services running behind a proxy and using socket communications. If you
|
|
||||||
change the value, don't forget to rebuild instance's environment
|
|
||||||
(:ref:`utils/brand.env <make buildenv>`)
|
|
||||||
|
|
||||||
``secret_key`` : ``$SEARXNG_SECRET``
|
|
||||||
Used for cryptography purpose.
|
|
||||||
|
|
||||||
.. _limiter:
|
|
||||||
|
|
||||||
``limiter`` :
|
|
||||||
Rate limit the number of request on the instance, block some bots. The
|
|
||||||
:ref:`limiter src` requires a :ref:`settings redis` database.
|
|
||||||
|
|
||||||
.. _image_proxy:
|
|
||||||
|
|
||||||
``image_proxy`` :
|
|
||||||
Allow your instance of SearXNG of being able to proxy images. Uses memory space.
|
|
||||||
|
|
||||||
.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
|
|
||||||
|
|
||||||
``default_http_headers`` :
|
|
||||||
Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__
|
|
||||||
|
|
||||||
|
|
||||||
.. _settings ui:
|
|
||||||
|
|
||||||
``ui:``
|
|
||||||
-------
|
|
||||||
|
|
||||||
.. _cache busting:
|
|
||||||
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#caching_static_assets_with_cache_busting
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
ui:
|
|
||||||
static_use_hash: false
|
|
||||||
default_locale: ""
|
|
||||||
query_in_title: false
|
|
||||||
infinite_scroll: false
|
|
||||||
center_alignment: false
|
|
||||||
cache_url: https://web.archive.org/web/
|
|
||||||
default_theme: simple
|
|
||||||
theme_args:
|
|
||||||
simple_style: auto
|
|
||||||
|
|
||||||
.. _static_use_hash:
|
|
||||||
|
|
||||||
``static_use_hash`` :
|
|
||||||
Enables `cache busting`_ of static files.
|
|
||||||
|
|
||||||
``default_locale`` :
|
|
||||||
SearXNG interface language. If blank, the locale is detected by using the
|
|
||||||
browser language. If it doesn't work, or you are deploying a language
|
|
||||||
specific instance of searx, a locale can be defined using an ISO language
|
|
||||||
code, like ``fr``, ``en``, ``de``.
|
|
||||||
|
|
||||||
``query_in_title`` :
|
|
||||||
When true, the result page's titles contains the query it decreases the
|
|
||||||
privacy, since the browser can records the page titles.
|
|
||||||
|
|
||||||
``infinite_scroll``:
|
|
||||||
When true, automatically loads the next page when scrolling to bottom of the current page.
|
|
||||||
|
|
||||||
``center_alignment`` : default ``false``
|
|
||||||
When enabled, the results are centered instead of being in the left (or RTL)
|
|
||||||
side of the screen. This setting only affects the *desktop layout*
|
|
||||||
(:origin:`min-width: @tablet <searx/static/themes/simple/src/less/definitions.less>`)
|
|
||||||
|
|
||||||
.. cache_url:
|
|
||||||
|
|
||||||
``cache_url`` : ``https://web.archive.org/web/``
|
|
||||||
URL prefix of the internet archive or cache, don't forgett trailing slash (if
|
|
||||||
needed). The default is https://web.archive.org/web/ alternatives are:
|
|
||||||
|
|
||||||
- https://webcache.googleusercontent.com/search?q=cache:
|
|
||||||
- https://archive.today/
|
|
||||||
|
|
||||||
``default_theme`` :
|
|
||||||
Name of the theme you want to use by default on your SearXNG instance.
|
|
||||||
|
|
||||||
``theme_args.simple_style``:
|
|
||||||
Style of simple theme: ``auto``, ``light``, ``dark``
|
|
||||||
|
|
||||||
``results_on_new_tab``:
|
|
||||||
Open result links in a new tab by default.
|
|
||||||
|
|
||||||
.. _settings redis:
|
|
||||||
|
|
||||||
``redis:``
|
|
||||||
----------
|
|
||||||
|
|
||||||
.. _Redis.from_url(url): https://redis-py.readthedocs.io/en/stable/connections.html#redis.client.Redis.from_url
|
|
||||||
|
|
||||||
A redis DB can be connected by an URL, in :py:obj:`searx.redisdb` you
|
|
||||||
will find a description to test your redis connection in SerXNG. When using
|
|
||||||
sockets, don't forget to check the access rights on the socket::
|
|
||||||
|
|
||||||
ls -la /usr/local/searxng-redis/run/redis.sock
|
|
||||||
srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock
|
|
||||||
|
|
||||||
In this example read/write access is given to the *searxng-redis* group. To get
|
|
||||||
access rights to redis instance (the socket), your SearXNG (or even your
|
|
||||||
developer) account needs to be added to the *searxng-redis* group.
|
|
||||||
|
|
||||||
``url`` : ``$SEARXNG_REDIS_URL``
|
|
||||||
URL to connect redis database, see `Redis.from_url(url)`_ & :ref:`redis db`::
|
|
||||||
|
|
||||||
redis://[[username]:[password]]@localhost:6379/0
|
|
||||||
rediss://[[username]:[password]]@localhost:6379/0
|
|
||||||
unix://[[username]:[password]]@/path/to/socket.sock?db=0
|
|
||||||
|
|
||||||
.. admonition:: Tip for developers
|
|
||||||
|
|
||||||
To set up a local redis instance, first set the socket path of the Redis DB
|
|
||||||
in your YAML setting:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
redis:
|
|
||||||
url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
|
|
||||||
|
|
||||||
Then use the following commands to install the redis instance ::
|
|
||||||
|
|
||||||
$ ./manage redis.build
|
|
||||||
$ sudo -H ./manage redis.install
|
|
||||||
$ sudo -H ./manage redis.addgrp "${USER}"
|
|
||||||
# don't forget to logout & login to get member of group
|
|
||||||
|
|
||||||
|
|
||||||
.. _settings outgoing:
|
|
||||||
|
|
||||||
``outgoing:``
|
|
||||||
-------------
|
|
||||||
|
|
||||||
Communication with search engines.
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
outgoing:
|
|
||||||
request_timeout: 2.0 # default timeout in seconds, can be override by engine
|
|
||||||
max_request_timeout: 10.0 # the maximum timeout in seconds
|
|
||||||
useragent_suffix: "" # information like an email address to the administrator
|
|
||||||
pool_connections: 100 # Maximum number of allowable connections, or null
|
|
||||||
# for no limits. The default is 100.
|
|
||||||
pool_maxsize: 10 # Number of allowable keep-alive connections, or null
|
|
||||||
# to always allow. The default is 10.
|
|
||||||
enable_http2: true # See https://www.python-httpx.org/http2/
|
|
||||||
# uncomment below section if you want to use a custom server certificate
|
|
||||||
# see https://www.python-httpx.org/advanced/#changing-the-verification-defaults
|
|
||||||
# and https://www.python-httpx.org/compatibility/#ssl-configuration
|
|
||||||
# verify: ~/.mitmproxy/mitmproxy-ca-cert.cer
|
|
||||||
#
|
|
||||||
# uncomment below section if you want to use a proxyq see: SOCKS proxies
|
|
||||||
# https://2.python-requests.org/en/latest/user/advanced/#proxies
|
|
||||||
# are also supported: see
|
|
||||||
# https://2.python-requests.org/en/latest/user/advanced/#socks
|
|
||||||
#
|
|
||||||
# proxies:
|
|
||||||
# all://:
|
|
||||||
# - http://proxy1:8080
|
|
||||||
# - http://proxy2:8080
|
|
||||||
#
|
|
||||||
# using_tor_proxy: true
|
|
||||||
#
|
|
||||||
# Extra seconds to add in order to account for the time taken by the proxy
|
|
||||||
#
|
|
||||||
# extra_proxy_timeout: 10.0
|
|
||||||
#
|
|
||||||
|
|
||||||
``request_timeout`` :
|
|
||||||
Global timeout of the requests made to others engines in seconds. A bigger
|
|
||||||
timeout will allow to wait for answers from slow engines, but in consequence
|
|
||||||
will slow SearXNG reactivity (the result page may take the time specified in the
|
|
||||||
timeout to load). Can be override by ``timeout`` in the :ref:`settings engine`.
|
|
||||||
|
|
||||||
``useragent_suffix`` :
|
|
||||||
Suffix to the user-agent SearXNG uses to send requests to others engines. If an
|
|
||||||
engine wish to block you, a contact info here may be useful to avoid that.
|
|
||||||
|
|
||||||
.. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration
|
|
||||||
|
|
||||||
``pool_maxsize``:
|
|
||||||
Number of allowable keep-alive connections, or ``null`` to always allow. The
|
|
||||||
default is 10. See ``max_keepalive_connections`` `Pool limit configuration`_.
|
|
||||||
|
|
||||||
``pool_connections`` :
|
|
||||||
Maximum number of allowable connections, or ``null`` # for no limits. The
|
|
||||||
default is 100. See ``max_connections`` `Pool limit configuration`_.
|
|
||||||
|
|
||||||
``keepalive_expiry`` :
|
|
||||||
Number of seconds to keep a connection in the pool. By default 5.0 seconds.
|
|
||||||
See ``keepalive_expiry`` `Pool limit configuration`_.
|
|
||||||
|
|
||||||
|
|
||||||
.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying
|
|
||||||
|
|
||||||
``proxies`` :
|
|
||||||
Define one or more proxies you wish to use, see `httpx proxies`_.
|
|
||||||
If there are more than one proxy for one protocol (http, https),
|
|
||||||
requests to the engines are distributed in a round-robin fashion.
|
|
||||||
|
|
||||||
``source_ips`` :
|
|
||||||
If you use multiple network interfaces, define from which IP the requests must
|
|
||||||
be made. Example:
|
|
||||||
|
|
||||||
* ``0.0.0.0`` any local IPv4 address.
|
|
||||||
* ``::`` any local IPv6 address.
|
|
||||||
* ``192.168.0.1``
|
|
||||||
* ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses
|
|
||||||
* ``fe80::60a2:1691:e5a2:ee1f``
|
|
||||||
* ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network.
|
|
||||||
* ``[ 192.168.0.1, fe80::/126 ]``
|
|
||||||
|
|
||||||
``retries`` :
|
|
||||||
Number of retry in case of an HTTP error. On each retry, SearXNG uses an
|
|
||||||
different proxy and source ip.
|
|
||||||
|
|
||||||
``enable_http2`` :
|
|
||||||
Enable by default. Set to ``false`` to disable HTTP/2.
|
|
||||||
|
|
||||||
.. _httpx verification defaults: https://www.python-httpx.org/advanced/#changing-the-verification-defaults
|
|
||||||
.. _httpx ssl configuration: https://www.python-httpx.org/compatibility/#ssl-configuration
|
|
||||||
|
|
||||||
``verify``: : ``$SSL_CERT_FILE``, ``$SSL_CERT_DIR``
|
|
||||||
Allow to specify a path to certificate.
|
|
||||||
see `httpx verification defaults`_.
|
|
||||||
|
|
||||||
In addition to ``verify``, SearXNG supports the ``$SSL_CERT_FILE`` (for a file) and
|
|
||||||
``$SSL_CERT_DIR`` (for a directory) OpenSSL variables.
|
|
||||||
see `httpx ssl configuration`_.
|
|
||||||
|
|
||||||
``max_redirects`` :
|
|
||||||
30 by default. Maximum redirect before it is an error.
|
|
||||||
|
|
||||||
``using_tor_proxy`` :
|
|
||||||
Using tor proxy (``true``) or not (``false``) for all engines. The default is
|
|
||||||
``false`` and can be overwritten in the :ref:`settings engine`
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. _settings categories_as_tabs:
|
|
||||||
|
|
||||||
``categories_as_tabs:``
|
|
||||||
-----------------------
|
|
||||||
|
|
||||||
A list of the categories that are displayed as tabs in the user interface.
|
|
||||||
Categories not listed here can still be searched with the :ref:`search-syntax`.
|
|
||||||
|
|
||||||
.. code-block:: yaml
|
|
||||||
|
|
||||||
categories_as_tabs:
|
|
||||||
general:
|
|
||||||
images:
|
|
||||||
videos:
|
|
||||||
news:
|
|
||||||
map:
|
|
||||||
music:
|
|
||||||
it:
|
|
||||||
science:
|
|
||||||
files:
|
|
||||||
social media:
|
|
||||||
|
|
||||||
Engines are added to ``categories:`` (compare :ref:`engine categories`), the
|
|
||||||
categories listed in ``categories_as_tabs`` are shown as tabs in the UI. If
|
|
||||||
there are no active engines in a category, the tab is not displayed (e.g. if a
|
|
||||||
user disables all engines in a category).
|
|
||||||
|
|
||||||
On the preferences page (``/preferences``) -- under *engines* -- there is an
|
|
||||||
additional tab, called *other*. In this tab are all engines listed that are not
|
|
||||||
in one of the UI tabs (not included in ``categories_as_tabs``).
|
|
||||||
|
|
||||||
.. _settings engine:
|
|
||||||
|
|
||||||
Engine settings
|
|
||||||
===============
|
|
||||||
|
|
||||||
.. sidebar:: Further reading ..
|
|
||||||
|
|
||||||
- :ref:`configured engines`
|
|
||||||
- :ref:`engines-dev`
|
|
||||||
|
|
||||||
In the code example below a *full fledged* example of a YAML setup from a dummy
|
|
||||||
engine is shown. Most of the options have a default value or even are optional.
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
- name: example engine
|
|
||||||
engine: example
|
|
||||||
shortcut: demo
|
|
||||||
base_url: 'https://{language}.example.com/'
|
|
||||||
send_accept_language_header: false
|
|
||||||
categories: general
|
|
||||||
timeout: 3.0
|
|
||||||
api_key: 'apikey'
|
|
||||||
disabled: false
|
|
||||||
language: en_US
|
|
||||||
tokens: [ 'my-secret-token' ]
|
|
||||||
weight: 1
|
|
||||||
display_error_messages: true
|
|
||||||
about:
|
|
||||||
website: https://example.com
|
|
||||||
wikidata_id: Q306656
|
|
||||||
official_api_documentation: https://example.com/api-doc
|
|
||||||
use_official_api: true
|
|
||||||
require_api_key: true
|
|
||||||
results: HTML
|
|
||||||
|
|
||||||
# overwrite values from section 'outgoing:'
|
|
||||||
enable_http2: false
|
|
||||||
retries: 1
|
|
||||||
max_connections: 100
|
|
||||||
max_keepalive_connections: 10
|
|
||||||
keepalive_expiry: 5.0
|
|
||||||
using_tor_proxy: false
|
|
||||||
proxies:
|
|
||||||
http:
|
|
||||||
- http://proxy1:8080
|
|
||||||
- http://proxy2:8080
|
|
||||||
https:
|
|
||||||
- http://proxy1:8080
|
|
||||||
- http://proxy2:8080
|
|
||||||
- socks5://user:password@proxy3:1080
|
|
||||||
- socks5h://user:password@proxy4:1080
|
|
||||||
|
|
||||||
# other network settings
|
|
||||||
enable_http: false
|
|
||||||
retry_on_http_error: true # or 403 or [404, 429]
|
|
||||||
|
|
||||||
|
|
||||||
``name`` :
|
|
||||||
Name that will be used across SearXNG to define this engine. In settings, on
|
|
||||||
the result page...
|
|
||||||
|
|
||||||
``engine`` :
|
|
||||||
Name of the python file used to handle requests and responses to and from this
|
|
||||||
search engine.
|
|
||||||
|
|
||||||
``shortcut`` :
|
|
||||||
Code used to execute bang requests (in this case using ``!bi``)
|
|
||||||
|
|
||||||
``base_url`` : optional
|
|
||||||
Part of the URL that should be stable across every request. Can be useful to
|
|
||||||
use multiple sites using only one engine, or updating the site URL without
|
|
||||||
touching at the code.
|
|
||||||
|
|
||||||
``send_accept_language_header`` :
|
|
||||||
Several engines that support languages (or regions) deal with the HTTP header
|
|
||||||
``Accept-Language`` to build a response that fits to the locale. When this
|
|
||||||
option is activated, the language (locale) that is selected by the user is used
|
|
||||||
to build and send a ``Accept-Language`` header in the request to the origin
|
|
||||||
search engine.
|
|
||||||
|
|
||||||
.. _engine categories:
|
|
||||||
|
|
||||||
``categories`` : optional
|
|
||||||
Specifies to which categories the engine should be added. Engines can be
|
|
||||||
assigned to multiple categories.
|
|
||||||
|
|
||||||
Categories can be shown as tabs (:ref:`settings categories_as_tabs`) in the
|
|
||||||
UI. A search in a tab (in the UI) will query all engines that are active in
|
|
||||||
this tab. In the preferences page (``/preferences``) -- under *engines* --
|
|
||||||
users can select what engine should be active when querying in this tab.
|
|
||||||
|
|
||||||
Alternatively, :ref:`\!bang <search-syntax>` can be used to search all engines
|
|
||||||
in a category, regardless of whether they are active or not, or whether they
|
|
||||||
are in a tab of the UI or not. For example, ``!dictionaries`` can be used to
|
|
||||||
query all search engines in that category (group).
|
|
||||||
|
|
||||||
``timeout`` : optional
|
|
||||||
Timeout of the search with the current search engine. Overwrites
|
|
||||||
``request_timeout`` from :ref:`settings outgoing`. **Be careful, it will
|
|
||||||
modify the global timeout of SearXNG.**
|
|
||||||
|
|
||||||
``api_key`` : optional
|
|
||||||
In a few cases, using an API needs the use of a secret key. How to obtain them
|
|
||||||
is described in the file.
|
|
||||||
|
|
||||||
``disabled`` : optional
|
|
||||||
To disable by default the engine, but not deleting it. It will allow the user
|
|
||||||
to manually activate it in the settings.
|
|
||||||
|
|
||||||
``inactive``: optional
|
|
||||||
Remove the engine from the settings (*disabled & removed*).
|
|
||||||
|
|
||||||
``language`` : optional
|
|
||||||
If you want to use another language for a specific engine, you can define it
|
|
||||||
by using the ISO code of language (and region), like ``fr``, ``en-US``,
|
|
||||||
``de-DE``.
|
|
||||||
|
|
||||||
``tokens`` : optional
|
|
||||||
A list of secret tokens to make this engine *private*, more details see
|
|
||||||
:ref:`private engines`.
|
|
||||||
|
|
||||||
``weight`` : default ``1``
|
|
||||||
Weighting of the results of this engine.
|
|
||||||
|
|
||||||
``display_error_messages`` : default ``true``
|
|
||||||
When an engine returns an error, the message is displayed on the user interface.
|
|
||||||
|
|
||||||
``network`` : optional
|
|
||||||
Use the network configuration from another engine.
|
|
||||||
In addition, there are two default networks:
|
|
||||||
|
|
||||||
- ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses)
|
|
||||||
- ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses)
|
|
||||||
|
|
||||||
``enable_http`` : optional
|
|
||||||
Enable HTTP for this engine (by default only HTTPS is enabled).
|
|
||||||
|
|
||||||
``retry_on_http_error`` : optional
|
|
||||||
Retry request on some HTTP status code.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
* ``true`` : on HTTP status code between 400 and 599.
|
|
||||||
* ``403`` : on HTTP status code 403.
|
|
||||||
* ``[403, 429]``: on HTTP status code 403 and 429.
|
|
||||||
|
|
||||||
``proxies`` :
|
|
||||||
Overwrites proxy settings from :ref:`settings outgoing`.
|
|
||||||
|
|
||||||
``using_tor_proxy`` :
|
|
||||||
Using tor proxy (``true``) or not (``false``) for this engine. The default is
|
|
||||||
taken from ``using_tor_proxy`` of the :ref:`settings outgoing`.
|
|
||||||
|
|
||||||
``max_keepalive_connection#s`` :
|
|
||||||
`Pool limit configuration`_, overwrites value ``pool_maxsize`` from
|
|
||||||
:ref:`settings outgoing` for this engine.
|
|
||||||
|
|
||||||
``max_connections`` :
|
|
||||||
`Pool limit configuration`_, overwrites value ``pool_connections`` from
|
|
||||||
:ref:`settings outgoing` for this engine.
|
|
||||||
|
|
||||||
``keepalive_expiry`` :
|
|
||||||
`Pool limit configuration`_, overwrites value ``keepalive_expiry`` from
|
|
||||||
:ref:`settings outgoing` for this engine.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
A few more options are possible, but they are pretty specific to some
|
|
||||||
engines, and so won't be described here.
|
|
||||||
|
|
||||||
|
|
||||||
Example: Multilingual Search
|
|
||||||
----------------------------
|
|
||||||
|
|
||||||
SearXNG does not support true multilingual search. You have to use the language
|
|
||||||
prefix in your search query when searching in a different language.
|
|
||||||
|
|
||||||
But there is a workaround: By adding a new search engine with a different
|
|
||||||
language, SearXNG will search in your default and other language.
|
|
||||||
|
|
||||||
Example configuration in settings.yml for a German and English speaker:
|
|
||||||
|
|
||||||
.. code-block:: yaml
|
|
||||||
|
|
||||||
search:
|
|
||||||
default_lang : "de"
|
|
||||||
...
|
|
||||||
|
|
||||||
engines:
|
|
||||||
- name : google english
|
|
||||||
engine : google
|
|
||||||
language : en
|
|
||||||
...
|
|
||||||
|
|
||||||
When searching, the default google engine will return German results and
|
|
||||||
"google english" will return English results.
|
|
||||||
|
|
||||||
|
|
||||||
.. _settings use_default_settings:
|
|
||||||
|
|
||||||
use_default_settings
|
|
||||||
====================
|
|
||||||
|
|
||||||
.. sidebar:: ``use_default_settings: true``
|
|
||||||
|
|
||||||
- :ref:`settings location`
|
|
||||||
- :ref:`use_default_settings.yml`
|
|
||||||
- :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
|
|
||||||
|
|
||||||
The user defined ``settings.yml`` is loaded from the :ref:`settings location`
|
|
||||||
and can relied on the default configuration :origin:`searx/settings.yml` using:
|
|
||||||
|
|
||||||
``use_default_settings: true``
|
|
||||||
|
|
||||||
``server:``
|
|
||||||
In the following example, the actual settings are the default settings defined
|
|
||||||
in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and
|
|
||||||
the ``bind_address``:
|
|
||||||
|
|
||||||
.. code-block:: yaml
|
|
||||||
|
|
||||||
use_default_settings: true
|
|
||||||
server:
|
|
||||||
secret_key: "ultrasecretkey" # change this!
|
|
||||||
bind_address: "0.0.0.0"
|
|
||||||
|
|
||||||
``engines:``
|
|
||||||
With ``use_default_settings: true``, each settings can be override in a
|
|
||||||
similar way, the ``engines`` section is merged according to the engine
|
|
||||||
``name``. In this example, SearXNG will load all the default engines, will
|
|
||||||
enable the ``bing`` engine and define a :ref:`token <private engines>` for
|
|
||||||
the arch linux engine:
|
|
||||||
|
|
||||||
.. code-block:: yaml
|
|
||||||
|
|
||||||
use_default_settings: true
|
|
||||||
server:
|
|
||||||
secret_key: "ultrasecretkey" # change this!
|
|
||||||
engines:
|
|
||||||
- name: arch linux wiki
|
|
||||||
tokens: ['$ecretValue']
|
|
||||||
- name: bing
|
|
||||||
disabled: false
|
|
||||||
|
|
||||||
|
|
||||||
``engines:`` / ``remove:``
|
|
||||||
It is possible to remove some engines from the default settings. The following
|
|
||||||
example is similar to the above one, but SearXNG doesn't load the the google
|
|
||||||
engine:
|
|
||||||
|
|
||||||
.. code-block:: yaml
|
|
||||||
|
|
||||||
use_default_settings:
|
|
||||||
engines:
|
|
||||||
remove:
|
|
||||||
- google
|
|
||||||
server:
|
|
||||||
secret_key: "ultrasecretkey" # change this!
|
|
||||||
engines:
|
|
||||||
- name: arch linux wiki
|
|
||||||
tokens: ['$ecretValue']
|
|
||||||
|
|
||||||
``engines:`` / ``keep_only:``
|
|
||||||
As an alternative, it is possible to specify the engines to keep. In the
|
|
||||||
following example, SearXNG has only two engines:
|
|
||||||
|
|
||||||
.. code-block:: yaml
|
|
||||||
|
|
||||||
use_default_settings:
|
|
||||||
engines:
|
|
||||||
keep_only:
|
|
||||||
- google
|
|
||||||
- duckduckgo
|
|
||||||
server:
|
|
||||||
secret_key: "ultrasecretkey" # change this!
|
|
||||||
engines:
|
|
||||||
- name: google
|
|
||||||
tokens: ['$ecretValue']
|
|
||||||
- name: duckduckgo
|
|
||||||
tokens: ['$ecretValue']
|
|
||||||
|
|
@ -4,8 +4,8 @@ Administrator documentation
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
:caption: Contents
|
|
||||||
|
|
||||||
|
settings/index
|
||||||
installation
|
installation
|
||||||
installation-docker
|
installation-docker
|
||||||
installation-scripts
|
installation-scripts
|
||||||
|
|
@ -15,7 +15,6 @@ Administrator documentation
|
||||||
installation-apache
|
installation-apache
|
||||||
update-searxng
|
update-searxng
|
||||||
answer-captcha
|
answer-captcha
|
||||||
engines/index
|
|
||||||
api
|
api
|
||||||
architecture
|
architecture
|
||||||
plugins
|
plugins
|
||||||
|
|
|
||||||
|
|
@ -61,7 +61,7 @@ section might give you some guidance.
|
||||||
- `Apache Fedora`_
|
- `Apache Fedora`_
|
||||||
- `Apache directives`_
|
- `Apache directives`_
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -41,7 +41,7 @@ section might give you some guidance.
|
||||||
- `uWSGI support from nginx`_
|
- `uWSGI support from nginx`_
|
||||||
|
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
Step by step installation
|
Step by step installation
|
||||||
=========================
|
=========================
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
@ -73,7 +73,7 @@ Configuration
|
||||||
|
|
||||||
.. sidebar:: ``use_default_settings: True``
|
.. sidebar:: ``use_default_settings: True``
|
||||||
|
|
||||||
- :ref:`settings global`
|
- :ref:`settings.yml`
|
||||||
- :ref:`settings location`
|
- :ref:`settings location`
|
||||||
- :ref:`settings use_default_settings`
|
- :ref:`settings use_default_settings`
|
||||||
- :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
|
- :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,7 @@ uWSGI
|
||||||
- `systemd.unit`_
|
- `systemd.unit`_
|
||||||
- `uWSGI Emperor`_
|
- `uWSGI Emperor`_
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
25
docs/admin/settings/index.rst
Normal file
25
docs/admin/settings/index.rst
Normal file
|
|
@ -0,0 +1,25 @@
|
||||||
|
========
|
||||||
|
Settings
|
||||||
|
========
|
||||||
|
|
||||||
|
.. sidebar:: Further reading ..
|
||||||
|
|
||||||
|
- :ref:`engine settings`
|
||||||
|
- :ref:`engine file`
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 2
|
||||||
|
|
||||||
|
settings
|
||||||
|
settings_engine
|
||||||
|
settings_brand
|
||||||
|
settings_general
|
||||||
|
settings_search
|
||||||
|
settings_server
|
||||||
|
settings_ui
|
||||||
|
settings_redis
|
||||||
|
settings_outgoing
|
||||||
|
settings_categories_as_tabs
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
117
docs/admin/settings/settings.rst
Normal file
117
docs/admin/settings/settings.rst
Normal file
|
|
@ -0,0 +1,117 @@
|
||||||
|
.. _settings.yml:
|
||||||
|
|
||||||
|
================
|
||||||
|
``settings.yml``
|
||||||
|
================
|
||||||
|
|
||||||
|
This page describe the options possibilities of the :origin:`searx/settings.yml`
|
||||||
|
file.
|
||||||
|
|
||||||
|
.. sidebar:: Further reading ..
|
||||||
|
|
||||||
|
- :ref:`use_default_settings.yml`
|
||||||
|
- :ref:`search API`
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
|
.. _settings location:
|
||||||
|
|
||||||
|
settings.yml location
|
||||||
|
=====================
|
||||||
|
|
||||||
|
The initial ``settings.yml`` we be load from these locations:
|
||||||
|
|
||||||
|
1. the full path specified in the ``SEARXNG_SETTINGS_PATH`` environment variable.
|
||||||
|
2. ``/etc/searxng/settings.yml``
|
||||||
|
|
||||||
|
If these files don't exist (or are empty or can't be read), SearXNG uses the
|
||||||
|
:origin:`searx/settings.yml` file. Read :ref:`settings use_default_settings` to
|
||||||
|
see how you can simplify your *user defined* ``settings.yml``.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. _settings use_default_settings:
|
||||||
|
|
||||||
|
use_default_settings
|
||||||
|
====================
|
||||||
|
|
||||||
|
.. sidebar:: ``use_default_settings: true``
|
||||||
|
|
||||||
|
- :ref:`settings location`
|
||||||
|
- :ref:`use_default_settings.yml`
|
||||||
|
- :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
|
||||||
|
|
||||||
|
The user defined ``settings.yml`` is loaded from the :ref:`settings location`
|
||||||
|
and can relied on the default configuration :origin:`searx/settings.yml` using:
|
||||||
|
|
||||||
|
``use_default_settings: true``
|
||||||
|
|
||||||
|
``server:``
|
||||||
|
In the following example, the actual settings are the default settings defined
|
||||||
|
in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and
|
||||||
|
the ``bind_address``:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
use_default_settings: true
|
||||||
|
server:
|
||||||
|
secret_key: "ultrasecretkey" # change this!
|
||||||
|
bind_address: "0.0.0.0"
|
||||||
|
|
||||||
|
``engines:``
|
||||||
|
With ``use_default_settings: true``, each settings can be override in a
|
||||||
|
similar way, the ``engines`` section is merged according to the engine
|
||||||
|
``name``. In this example, SearXNG will load all the default engines, will
|
||||||
|
enable the ``bing`` engine and define a :ref:`token <private engines>` for
|
||||||
|
the arch linux engine:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
use_default_settings: true
|
||||||
|
server:
|
||||||
|
secret_key: "ultrasecretkey" # change this!
|
||||||
|
engines:
|
||||||
|
- name: arch linux wiki
|
||||||
|
tokens: ['$ecretValue']
|
||||||
|
- name: bing
|
||||||
|
disabled: false
|
||||||
|
|
||||||
|
|
||||||
|
``engines:`` / ``remove:``
|
||||||
|
It is possible to remove some engines from the default settings. The following
|
||||||
|
example is similar to the above one, but SearXNG doesn't load the the google
|
||||||
|
engine:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
use_default_settings:
|
||||||
|
engines:
|
||||||
|
remove:
|
||||||
|
- google
|
||||||
|
server:
|
||||||
|
secret_key: "ultrasecretkey" # change this!
|
||||||
|
engines:
|
||||||
|
- name: arch linux wiki
|
||||||
|
tokens: ['$ecretValue']
|
||||||
|
|
||||||
|
``engines:`` / ``keep_only:``
|
||||||
|
As an alternative, it is possible to specify the engines to keep. In the
|
||||||
|
following example, SearXNG has only two engines:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
use_default_settings:
|
||||||
|
engines:
|
||||||
|
keep_only:
|
||||||
|
- google
|
||||||
|
- duckduckgo
|
||||||
|
server:
|
||||||
|
secret_key: "ultrasecretkey" # change this!
|
||||||
|
engines:
|
||||||
|
- name: google
|
||||||
|
tokens: ['$ecretValue']
|
||||||
|
- name: duckduckgo
|
||||||
|
tokens: ['$ecretValue']
|
||||||
25
docs/admin/settings/settings_brand.rst
Normal file
25
docs/admin/settings/settings_brand.rst
Normal file
|
|
@ -0,0 +1,25 @@
|
||||||
|
.. _settings brand:
|
||||||
|
|
||||||
|
==========
|
||||||
|
``brand:``
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
brand:
|
||||||
|
issue_url: https://github.com/searxng/searxng/issues
|
||||||
|
docs_url: https://docs.searxng.org
|
||||||
|
public_instances: https://searx.space
|
||||||
|
wiki_url: https://github.com/searxng/searxng/wiki
|
||||||
|
|
||||||
|
``issue_url`` :
|
||||||
|
If you host your own issue tracker change this URL.
|
||||||
|
|
||||||
|
``docs_url`` :
|
||||||
|
If you host your own documentation change this URL.
|
||||||
|
|
||||||
|
``public_instances`` :
|
||||||
|
If you host your own https://searx.space change this URL.
|
||||||
|
|
||||||
|
``wiki_url`` :
|
||||||
|
Link to your wiki (or ``false``)
|
||||||
31
docs/admin/settings/settings_categories_as_tabs.rst
Normal file
31
docs/admin/settings/settings_categories_as_tabs.rst
Normal file
|
|
@ -0,0 +1,31 @@
|
||||||
|
.. _settings categories_as_tabs:
|
||||||
|
|
||||||
|
=======================
|
||||||
|
``categories_as_tabs:``
|
||||||
|
=======================
|
||||||
|
|
||||||
|
A list of the categories that are displayed as tabs in the user interface.
|
||||||
|
Categories not listed here can still be searched with the :ref:`search-syntax`.
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
categories_as_tabs:
|
||||||
|
general:
|
||||||
|
images:
|
||||||
|
videos:
|
||||||
|
news:
|
||||||
|
map:
|
||||||
|
music:
|
||||||
|
it:
|
||||||
|
science:
|
||||||
|
files:
|
||||||
|
social media:
|
||||||
|
|
||||||
|
Engines are added to ``categories:`` (compare :ref:`engine categories`), the
|
||||||
|
categories listed in ``categories_as_tabs`` are shown as tabs in the UI. If
|
||||||
|
there are no active engines in a category, the tab is not displayed (e.g. if a
|
||||||
|
user disables all engines in a category).
|
||||||
|
|
||||||
|
On the preferences page (``/preferences``) -- under *engines* -- there is an
|
||||||
|
additional tab, called *other*. In this tab are all engines listed that are not
|
||||||
|
in one of the UI tabs (not included in ``categories_as_tabs``).
|
||||||
244
docs/admin/settings/settings_engine.rst
Normal file
244
docs/admin/settings/settings_engine.rst
Normal file
|
|
@ -0,0 +1,244 @@
|
||||||
|
.. _settings engine:
|
||||||
|
|
||||||
|
===========
|
||||||
|
``engine:``
|
||||||
|
===========
|
||||||
|
|
||||||
|
.. sidebar:: Further reading ..
|
||||||
|
|
||||||
|
- :ref:`configured engines`
|
||||||
|
- :ref:`engines-dev`
|
||||||
|
|
||||||
|
In the code example below a *full fledged* example of a YAML setup from a dummy
|
||||||
|
engine is shown. Most of the options have a default value or even are optional.
|
||||||
|
|
||||||
|
.. hint::
|
||||||
|
|
||||||
|
A few more options are possible, but they are pretty specific to some
|
||||||
|
engines (:ref:`engine implementations`).
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
- name: example engine
|
||||||
|
engine: example
|
||||||
|
shortcut: demo
|
||||||
|
base_url: 'https://{language}.example.com/'
|
||||||
|
send_accept_language_header: false
|
||||||
|
categories: general
|
||||||
|
timeout: 3.0
|
||||||
|
api_key: 'apikey'
|
||||||
|
disabled: false
|
||||||
|
language: en_US
|
||||||
|
tokens: [ 'my-secret-token' ]
|
||||||
|
weight: 1
|
||||||
|
display_error_messages: true
|
||||||
|
about:
|
||||||
|
website: https://example.com
|
||||||
|
wikidata_id: Q306656
|
||||||
|
official_api_documentation: https://example.com/api-doc
|
||||||
|
use_official_api: true
|
||||||
|
require_api_key: true
|
||||||
|
results: HTML
|
||||||
|
|
||||||
|
# overwrite values from section 'outgoing:'
|
||||||
|
enable_http2: false
|
||||||
|
retries: 1
|
||||||
|
max_connections: 100
|
||||||
|
max_keepalive_connections: 10
|
||||||
|
keepalive_expiry: 5.0
|
||||||
|
using_tor_proxy: false
|
||||||
|
proxies:
|
||||||
|
http:
|
||||||
|
- http://proxy1:8080
|
||||||
|
- http://proxy2:8080
|
||||||
|
https:
|
||||||
|
- http://proxy1:8080
|
||||||
|
- http://proxy2:8080
|
||||||
|
- socks5://user:password@proxy3:1080
|
||||||
|
- socks5h://user:password@proxy4:1080
|
||||||
|
|
||||||
|
# other network settings
|
||||||
|
enable_http: false
|
||||||
|
retry_on_http_error: true # or 403 or [404, 429]
|
||||||
|
|
||||||
|
|
||||||
|
``name`` :
|
||||||
|
Name that will be used across SearXNG to define this engine. In settings, on
|
||||||
|
the result page...
|
||||||
|
|
||||||
|
``engine`` :
|
||||||
|
Name of the python file used to handle requests and responses to and from this
|
||||||
|
search engine.
|
||||||
|
|
||||||
|
``shortcut`` :
|
||||||
|
Code used to execute bang requests (in this case using ``!bi``)
|
||||||
|
|
||||||
|
``base_url`` : optional
|
||||||
|
Part of the URL that should be stable across every request. Can be useful to
|
||||||
|
use multiple sites using only one engine, or updating the site URL without
|
||||||
|
touching at the code.
|
||||||
|
|
||||||
|
``send_accept_language_header`` :
|
||||||
|
Several engines that support languages (or regions) deal with the HTTP header
|
||||||
|
``Accept-Language`` to build a response that fits to the locale. When this
|
||||||
|
option is activated, the language (locale) that is selected by the user is used
|
||||||
|
to build and send a ``Accept-Language`` header in the request to the origin
|
||||||
|
search engine.
|
||||||
|
|
||||||
|
.. _engine categories:
|
||||||
|
|
||||||
|
``categories`` : optional
|
||||||
|
Specifies to which categories the engine should be added. Engines can be
|
||||||
|
assigned to multiple categories.
|
||||||
|
|
||||||
|
Categories can be shown as tabs (:ref:`settings categories_as_tabs`) in the
|
||||||
|
UI. A search in a tab (in the UI) will query all engines that are active in
|
||||||
|
this tab. In the preferences page (``/preferences``) -- under *engines* --
|
||||||
|
users can select what engine should be active when querying in this tab.
|
||||||
|
|
||||||
|
Alternatively, :ref:`\!bang <search-syntax>` can be used to search all engines
|
||||||
|
in a category, regardless of whether they are active or not, or whether they
|
||||||
|
are in a tab of the UI or not. For example, ``!dictionaries`` can be used to
|
||||||
|
query all search engines in that category (group).
|
||||||
|
|
||||||
|
``timeout`` : optional
|
||||||
|
Timeout of the search with the current search engine. Overwrites
|
||||||
|
``request_timeout`` from :ref:`settings outgoing`. **Be careful, it will
|
||||||
|
modify the global timeout of SearXNG.**
|
||||||
|
|
||||||
|
``api_key`` : optional
|
||||||
|
In a few cases, using an API needs the use of a secret key. How to obtain them
|
||||||
|
is described in the file.
|
||||||
|
|
||||||
|
``disabled`` : optional
|
||||||
|
To disable by default the engine, but not deleting it. It will allow the user
|
||||||
|
to manually activate it in the settings.
|
||||||
|
|
||||||
|
``inactive``: optional
|
||||||
|
Remove the engine from the settings (*disabled & removed*).
|
||||||
|
|
||||||
|
``language`` : optional
|
||||||
|
If you want to use another language for a specific engine, you can define it
|
||||||
|
by using the ISO code of language (and region), like ``fr``, ``en-US``,
|
||||||
|
``de-DE``.
|
||||||
|
|
||||||
|
``tokens`` : optional
|
||||||
|
A list of secret tokens to make this engine *private*, more details see
|
||||||
|
:ref:`private engines`.
|
||||||
|
|
||||||
|
``weight`` : default ``1``
|
||||||
|
Weighting of the results of this engine.
|
||||||
|
|
||||||
|
``display_error_messages`` : default ``true``
|
||||||
|
When an engine returns an error, the message is displayed on the user interface.
|
||||||
|
|
||||||
|
``network`` : optional
|
||||||
|
Use the network configuration from another engine.
|
||||||
|
In addition, there are two default networks:
|
||||||
|
|
||||||
|
- ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses)
|
||||||
|
- ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses)
|
||||||
|
|
||||||
|
``enable_http`` : optional
|
||||||
|
Enable HTTP for this engine (by default only HTTPS is enabled).
|
||||||
|
|
||||||
|
``retry_on_http_error`` : optional
|
||||||
|
Retry request on some HTTP status code.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
* ``true`` : on HTTP status code between 400 and 599.
|
||||||
|
* ``403`` : on HTTP status code 403.
|
||||||
|
* ``[403, 429]``: on HTTP status code 403 and 429.
|
||||||
|
|
||||||
|
``proxies`` :
|
||||||
|
Overwrites proxy settings from :ref:`settings outgoing`.
|
||||||
|
|
||||||
|
``using_tor_proxy`` :
|
||||||
|
Using tor proxy (``true``) or not (``false``) for this engine. The default is
|
||||||
|
taken from ``using_tor_proxy`` of the :ref:`settings outgoing`.
|
||||||
|
|
||||||
|
.. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration
|
||||||
|
|
||||||
|
``max_keepalive_connection#s`` :
|
||||||
|
`Pool limit configuration`_, overwrites value ``pool_maxsize`` from
|
||||||
|
:ref:`settings outgoing` for this engine.
|
||||||
|
|
||||||
|
``max_connections`` :
|
||||||
|
`Pool limit configuration`_, overwrites value ``pool_connections`` from
|
||||||
|
:ref:`settings outgoing` for this engine.
|
||||||
|
|
||||||
|
``keepalive_expiry`` :
|
||||||
|
`Pool limit configuration`_, overwrites value ``keepalive_expiry`` from
|
||||||
|
:ref:`settings outgoing` for this engine.
|
||||||
|
|
||||||
|
|
||||||
|
.. _private engines:
|
||||||
|
|
||||||
|
Private Engines (``tokens``)
|
||||||
|
============================
|
||||||
|
|
||||||
|
Administrators might find themselves wanting to limit access to some of the
|
||||||
|
enabled engines on their instances. It might be because they do not want to
|
||||||
|
expose some private information through :ref:`offline engines`. Or they would
|
||||||
|
rather share engines only with their trusted friends or colleagues.
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
Initial sponsored by `Search and Discovery Fund
|
||||||
|
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||||
|
|
||||||
|
To solve this issue the concept of *private engines* exists.
|
||||||
|
|
||||||
|
A new option was added to engines named `tokens`. It expects a list of strings.
|
||||||
|
If the user making a request presents one of the tokens of an engine, they can
|
||||||
|
access information about the engine and make search requests.
|
||||||
|
|
||||||
|
Example configuration to restrict access to the Arch Linux Wiki engine:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
- name: arch linux wiki
|
||||||
|
engine: archlinux
|
||||||
|
shortcut: al
|
||||||
|
tokens: [ 'my-secret-token' ]
|
||||||
|
|
||||||
|
Unless a user has configured the right token, the engine is going to be hidden
|
||||||
|
from him/her. It is not going to be included in the list of engines on the
|
||||||
|
Preferences page and in the output of `/config` REST API call.
|
||||||
|
|
||||||
|
Tokens can be added to one's configuration on the Preferences page under "Engine
|
||||||
|
tokens". The input expects a comma separated list of strings.
|
||||||
|
|
||||||
|
The distribution of the tokens from the administrator to the users is not carved
|
||||||
|
in stone. As providing access to such engines implies that the admin knows and
|
||||||
|
trusts the user, we do not see necessary to come up with a strict process.
|
||||||
|
Instead, we would like to add guidelines to the documentation of the feature.
|
||||||
|
|
||||||
|
|
||||||
|
Example: Multilingual Search
|
||||||
|
============================
|
||||||
|
|
||||||
|
SearXNG does not support true multilingual search. You have to use the language
|
||||||
|
prefix in your search query when searching in a different language.
|
||||||
|
|
||||||
|
But there is a workaround: By adding a new search engine with a different
|
||||||
|
language, SearXNG will search in your default and other language.
|
||||||
|
|
||||||
|
Example configuration in settings.yml for a German and English speaker:
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
search:
|
||||||
|
default_lang : "de"
|
||||||
|
...
|
||||||
|
|
||||||
|
engines:
|
||||||
|
- name : google english
|
||||||
|
engine : google
|
||||||
|
language : en
|
||||||
|
...
|
||||||
|
|
||||||
|
When searching, the default google engine will return German results and
|
||||||
|
"google english" will return English results.
|
||||||
|
|
||||||
34
docs/admin/settings/settings_general.rst
Normal file
34
docs/admin/settings/settings_general.rst
Normal file
|
|
@ -0,0 +1,34 @@
|
||||||
|
.. _settings general:
|
||||||
|
|
||||||
|
============
|
||||||
|
``general:``
|
||||||
|
============
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
general:
|
||||||
|
debug: false
|
||||||
|
instance_name: "SearXNG"
|
||||||
|
privacypolicy_url: false
|
||||||
|
donation_url: false
|
||||||
|
contact_url: false
|
||||||
|
enable_metrics: true
|
||||||
|
|
||||||
|
``debug`` : ``$SEARXNG_DEBUG``
|
||||||
|
Allow a more detailed log if you run SearXNG directly. Display *detailed* error
|
||||||
|
messages in the browser too, so this must be deactivated in production.
|
||||||
|
|
||||||
|
``donation_url`` :
|
||||||
|
Set value to ``true`` to use your own donation page written in the
|
||||||
|
:ref:`searx/info/en/donate.md <searx.infopage>` and use ``false`` to disable
|
||||||
|
the donation link altogether.
|
||||||
|
|
||||||
|
``privacypolicy_url``:
|
||||||
|
Link to privacy policy.
|
||||||
|
|
||||||
|
``contact_url``:
|
||||||
|
Contact ``mailto:`` address or WEB form.
|
||||||
|
|
||||||
|
``enable_metrics``:
|
||||||
|
Enabled by default. Record various anonymous metrics availabled at ``/stats``,
|
||||||
|
``/stats/errors`` and ``/preferences``.
|
||||||
110
docs/admin/settings/settings_outgoing.rst
Normal file
110
docs/admin/settings/settings_outgoing.rst
Normal file
|
|
@ -0,0 +1,110 @@
|
||||||
|
.. _settings outgoing:
|
||||||
|
|
||||||
|
=============
|
||||||
|
``outgoing:``
|
||||||
|
=============
|
||||||
|
|
||||||
|
Communication with search engines.
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
outgoing:
|
||||||
|
request_timeout: 2.0 # default timeout in seconds, can be override by engine
|
||||||
|
max_request_timeout: 10.0 # the maximum timeout in seconds
|
||||||
|
useragent_suffix: "" # information like an email address to the administrator
|
||||||
|
pool_connections: 100 # Maximum number of allowable connections, or null
|
||||||
|
# for no limits. The default is 100.
|
||||||
|
pool_maxsize: 10 # Number of allowable keep-alive connections, or null
|
||||||
|
# to always allow. The default is 10.
|
||||||
|
enable_http2: true # See https://www.python-httpx.org/http2/
|
||||||
|
# uncomment below section if you want to use a custom server certificate
|
||||||
|
# see https://www.python-httpx.org/advanced/#changing-the-verification-defaults
|
||||||
|
# and https://www.python-httpx.org/compatibility/#ssl-configuration
|
||||||
|
# verify: ~/.mitmproxy/mitmproxy-ca-cert.cer
|
||||||
|
#
|
||||||
|
# uncomment below section if you want to use a proxyq see: SOCKS proxies
|
||||||
|
# https://2.python-requests.org/en/latest/user/advanced/#proxies
|
||||||
|
# are also supported: see
|
||||||
|
# https://2.python-requests.org/en/latest/user/advanced/#socks
|
||||||
|
#
|
||||||
|
# proxies:
|
||||||
|
# all://:
|
||||||
|
# - http://proxy1:8080
|
||||||
|
# - http://proxy2:8080
|
||||||
|
#
|
||||||
|
# using_tor_proxy: true
|
||||||
|
#
|
||||||
|
# Extra seconds to add in order to account for the time taken by the proxy
|
||||||
|
#
|
||||||
|
# extra_proxy_timeout: 10.0
|
||||||
|
#
|
||||||
|
|
||||||
|
``request_timeout`` :
|
||||||
|
Global timeout of the requests made to others engines in seconds. A bigger
|
||||||
|
timeout will allow to wait for answers from slow engines, but in consequence
|
||||||
|
will slow SearXNG reactivity (the result page may take the time specified in the
|
||||||
|
timeout to load). Can be override by ``timeout`` in the :ref:`settings engine`.
|
||||||
|
|
||||||
|
``useragent_suffix`` :
|
||||||
|
Suffix to the user-agent SearXNG uses to send requests to others engines. If an
|
||||||
|
engine wish to block you, a contact info here may be useful to avoid that.
|
||||||
|
|
||||||
|
.. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration
|
||||||
|
|
||||||
|
``pool_maxsize``:
|
||||||
|
Number of allowable keep-alive connections, or ``null`` to always allow. The
|
||||||
|
default is 10. See ``max_keepalive_connections`` `Pool limit configuration`_.
|
||||||
|
|
||||||
|
``pool_connections`` :
|
||||||
|
Maximum number of allowable connections, or ``null`` # for no limits. The
|
||||||
|
default is 100. See ``max_connections`` `Pool limit configuration`_.
|
||||||
|
|
||||||
|
``keepalive_expiry`` :
|
||||||
|
Number of seconds to keep a connection in the pool. By default 5.0 seconds.
|
||||||
|
See ``keepalive_expiry`` `Pool limit configuration`_.
|
||||||
|
|
||||||
|
.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying
|
||||||
|
|
||||||
|
``proxies`` :
|
||||||
|
Define one or more proxies you wish to use, see `httpx proxies`_.
|
||||||
|
If there are more than one proxy for one protocol (http, https),
|
||||||
|
requests to the engines are distributed in a round-robin fashion.
|
||||||
|
|
||||||
|
``source_ips`` :
|
||||||
|
If you use multiple network interfaces, define from which IP the requests must
|
||||||
|
be made. Example:
|
||||||
|
|
||||||
|
* ``0.0.0.0`` any local IPv4 address.
|
||||||
|
* ``::`` any local IPv6 address.
|
||||||
|
* ``192.168.0.1``
|
||||||
|
* ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses
|
||||||
|
* ``fe80::60a2:1691:e5a2:ee1f``
|
||||||
|
* ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network.
|
||||||
|
* ``[ 192.168.0.1, fe80::/126 ]``
|
||||||
|
|
||||||
|
``retries`` :
|
||||||
|
Number of retry in case of an HTTP error. On each retry, SearXNG uses an
|
||||||
|
different proxy and source ip.
|
||||||
|
|
||||||
|
``enable_http2`` :
|
||||||
|
Enable by default. Set to ``false`` to disable HTTP/2.
|
||||||
|
|
||||||
|
.. _httpx verification defaults: https://www.python-httpx.org/advanced/#changing-the-verification-defaults
|
||||||
|
.. _httpx ssl configuration: https://www.python-httpx.org/compatibility/#ssl-configuration
|
||||||
|
|
||||||
|
``verify``: : ``$SSL_CERT_FILE``, ``$SSL_CERT_DIR``
|
||||||
|
Allow to specify a path to certificate.
|
||||||
|
see `httpx verification defaults`_.
|
||||||
|
|
||||||
|
In addition to ``verify``, SearXNG supports the ``$SSL_CERT_FILE`` (for a file) and
|
||||||
|
``$SSL_CERT_DIR`` (for a directory) OpenSSL variables.
|
||||||
|
see `httpx ssl configuration`_.
|
||||||
|
|
||||||
|
``max_redirects`` :
|
||||||
|
30 by default. Maximum redirect before it is an error.
|
||||||
|
|
||||||
|
``using_tor_proxy`` :
|
||||||
|
Using tor proxy (``true``) or not (``false``) for all engines. The default is
|
||||||
|
``false`` and can be overwritten in the :ref:`settings engine`
|
||||||
|
|
||||||
|
|
||||||
43
docs/admin/settings/settings_redis.rst
Normal file
43
docs/admin/settings/settings_redis.rst
Normal file
|
|
@ -0,0 +1,43 @@
|
||||||
|
.. _settings redis:
|
||||||
|
|
||||||
|
==========
|
||||||
|
``redis:``
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. _Redis.from_url(url): https://redis-py.readthedocs.io/en/stable/connections.html#redis.client.Redis.from_url
|
||||||
|
|
||||||
|
A redis DB can be connected by an URL, in :py:obj:`searx.redisdb` you
|
||||||
|
will find a description to test your redis connection in SerXNG. When using
|
||||||
|
sockets, don't forget to check the access rights on the socket::
|
||||||
|
|
||||||
|
ls -la /usr/local/searxng-redis/run/redis.sock
|
||||||
|
srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock
|
||||||
|
|
||||||
|
In this example read/write access is given to the *searxng-redis* group. To get
|
||||||
|
access rights to redis instance (the socket), your SearXNG (or even your
|
||||||
|
developer) account needs to be added to the *searxng-redis* group.
|
||||||
|
|
||||||
|
``url`` : ``$SEARXNG_REDIS_URL``
|
||||||
|
URL to connect redis database, see `Redis.from_url(url)`_ & :ref:`redis db`::
|
||||||
|
|
||||||
|
redis://[[username]:[password]]@localhost:6379/0
|
||||||
|
rediss://[[username]:[password]]@localhost:6379/0
|
||||||
|
unix://[[username]:[password]]@/path/to/socket.sock?db=0
|
||||||
|
|
||||||
|
.. admonition:: Tip for developers
|
||||||
|
|
||||||
|
To set up a local redis instance, first set the socket path of the Redis DB
|
||||||
|
in your YAML setting:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
redis:
|
||||||
|
url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
|
||||||
|
|
||||||
|
Then use the following commands to install the redis instance ::
|
||||||
|
|
||||||
|
$ ./manage redis.build
|
||||||
|
$ sudo -H ./manage redis.install
|
||||||
|
$ sudo -H ./manage redis.addgrp "${USER}"
|
||||||
|
# don't forget to logout & login to get member of group
|
||||||
|
|
||||||
97
docs/admin/settings/settings_search.rst
Normal file
97
docs/admin/settings/settings_search.rst
Normal file
|
|
@ -0,0 +1,97 @@
|
||||||
|
.. _settings search:
|
||||||
|
|
||||||
|
===========
|
||||||
|
``search:``
|
||||||
|
===========
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
search:
|
||||||
|
safe_search: 0
|
||||||
|
autocomplete: ""
|
||||||
|
default_lang: ""
|
||||||
|
ban_time_on_fail: 5
|
||||||
|
max_ban_time_on_fail: 120
|
||||||
|
suspended_times:
|
||||||
|
SearxEngineAccessDenied: 86400
|
||||||
|
SearxEngineCaptcha: 86400
|
||||||
|
SearxEngineTooManyRequests: 3600
|
||||||
|
cf_SearxEngineCaptcha: 1296000
|
||||||
|
cf_SearxEngineAccessDenied: 86400
|
||||||
|
recaptcha_SearxEngineCaptcha: 604800
|
||||||
|
formats:
|
||||||
|
- html
|
||||||
|
|
||||||
|
``safe_search``:
|
||||||
|
Filter results.
|
||||||
|
|
||||||
|
- ``0``: None
|
||||||
|
- ``1``: Moderate
|
||||||
|
- ``2``: Strict
|
||||||
|
|
||||||
|
``autocomplete``:
|
||||||
|
Existing autocomplete backends, leave blank to turn it off.
|
||||||
|
|
||||||
|
- ``dbpedia``
|
||||||
|
- ``duckduckgo``
|
||||||
|
- ``google``
|
||||||
|
- ``startpage``
|
||||||
|
- ``swisscows``
|
||||||
|
- ``qwant``
|
||||||
|
- ``wikipedia``
|
||||||
|
|
||||||
|
``default_lang``:
|
||||||
|
Default search language - leave blank to detect from browser information or
|
||||||
|
use codes from :origin:`searx/languages.py`.
|
||||||
|
|
||||||
|
``languages``:
|
||||||
|
List of available languages - leave unset to use all codes from
|
||||||
|
:origin:`searx/languages.py`. Otherwise list codes of available languages.
|
||||||
|
The ``all`` value is shown as the ``Default language`` in the user interface
|
||||||
|
(in most cases, it is meant to send the query without a language parameter ;
|
||||||
|
in some cases, it means the English language) Example:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
languages:
|
||||||
|
- all
|
||||||
|
- en
|
||||||
|
- en-US
|
||||||
|
- de
|
||||||
|
- it-IT
|
||||||
|
- fr
|
||||||
|
- fr-BE
|
||||||
|
|
||||||
|
``ban_time_on_fail``:
|
||||||
|
Ban time in seconds after engine errors.
|
||||||
|
|
||||||
|
``max_ban_time_on_fail``:
|
||||||
|
Max ban time in seconds after engine errors.
|
||||||
|
|
||||||
|
``suspended_times``:
|
||||||
|
Engine suspension time after error (in seconds; set to 0 to disable)
|
||||||
|
|
||||||
|
``SearxEngineAccessDenied``: 86400
|
||||||
|
For error "Access denied" and "HTTP error [402, 403]"
|
||||||
|
|
||||||
|
``SearxEngineCaptcha``: 86400
|
||||||
|
For error "CAPTCHA"
|
||||||
|
|
||||||
|
``SearxEngineTooManyRequests``: 3600
|
||||||
|
For error "Too many request" and "HTTP error 429"
|
||||||
|
|
||||||
|
Cloudflare CAPTCHA:
|
||||||
|
- ``cf_SearxEngineCaptcha``: 1296000
|
||||||
|
- ``cf_SearxEngineAccessDenied``: 86400
|
||||||
|
|
||||||
|
Google CAPTCHA:
|
||||||
|
- ``recaptcha_SearxEngineCaptcha``: 604800
|
||||||
|
|
||||||
|
``formats``:
|
||||||
|
Result formats available from web, remove format to deny access (use lower
|
||||||
|
case).
|
||||||
|
|
||||||
|
- ``html``
|
||||||
|
- ``csv``
|
||||||
|
- ``json``
|
||||||
|
- ``rss``
|
||||||
54
docs/admin/settings/settings_server.rst
Normal file
54
docs/admin/settings/settings_server.rst
Normal file
|
|
@ -0,0 +1,54 @@
|
||||||
|
.. _settings server:
|
||||||
|
|
||||||
|
===========
|
||||||
|
``server:``
|
||||||
|
===========
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
server:
|
||||||
|
base_url: http://example.org/location # change this!
|
||||||
|
port: 8888
|
||||||
|
bind_address: "127.0.0.1"
|
||||||
|
secret_key: "ultrasecretkey" # change this!
|
||||||
|
limiter: false
|
||||||
|
image_proxy: false
|
||||||
|
default_http_headers:
|
||||||
|
X-Content-Type-Options : nosniff
|
||||||
|
X-XSS-Protection : 1; mode=block
|
||||||
|
X-Download-Options : noopen
|
||||||
|
X-Robots-Tag : noindex, nofollow
|
||||||
|
Referrer-Policy : no-referrer
|
||||||
|
|
||||||
|
|
||||||
|
``base_url`` : ``$SEARXNG_URL`` :ref:`buildenv <make buildenv>`
|
||||||
|
The base URL where SearXNG is deployed. Used to create correct inbound links.
|
||||||
|
If you change the value, don't forget to rebuild instance's environment
|
||||||
|
(:ref:`utils/brand.env <make buildenv>`)
|
||||||
|
|
||||||
|
``port`` & ``bind_address``: ``$SEARXNG_PORT`` & ``$SEARXNG_BIND_ADDRESS`` :ref:`buildenv <make buildenv>`
|
||||||
|
Port number and *bind address* of the SearXNG web application if you run it
|
||||||
|
directly using ``python searx/webapp.py``. Doesn't apply to a SearXNG
|
||||||
|
services running behind a proxy and using socket communications. If you
|
||||||
|
change the value, don't forget to rebuild instance's environment
|
||||||
|
(:ref:`utils/brand.env <make buildenv>`)
|
||||||
|
|
||||||
|
``secret_key`` : ``$SEARXNG_SECRET``
|
||||||
|
Used for cryptography purpose.
|
||||||
|
|
||||||
|
.. _limiter:
|
||||||
|
|
||||||
|
``limiter`` :
|
||||||
|
Rate limit the number of request on the instance, block some bots. The
|
||||||
|
:ref:`limiter src` requires a :ref:`settings redis` database.
|
||||||
|
|
||||||
|
.. _image_proxy:
|
||||||
|
|
||||||
|
``image_proxy`` :
|
||||||
|
Allow your instance of SearXNG of being able to proxy images. Uses memory space.
|
||||||
|
|
||||||
|
.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
|
||||||
|
|
||||||
|
``default_http_headers`` :
|
||||||
|
Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__
|
||||||
|
|
||||||
62
docs/admin/settings/settings_ui.rst
Normal file
62
docs/admin/settings/settings_ui.rst
Normal file
|
|
@ -0,0 +1,62 @@
|
||||||
|
.. _settings ui:
|
||||||
|
|
||||||
|
=======
|
||||||
|
``ui:``
|
||||||
|
=======
|
||||||
|
|
||||||
|
.. _cache busting:
|
||||||
|
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#caching_static_assets_with_cache_busting
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
ui:
|
||||||
|
static_use_hash: false
|
||||||
|
default_locale: ""
|
||||||
|
query_in_title: false
|
||||||
|
infinite_scroll: false
|
||||||
|
center_alignment: false
|
||||||
|
cache_url: https://web.archive.org/web/
|
||||||
|
default_theme: simple
|
||||||
|
theme_args:
|
||||||
|
simple_style: auto
|
||||||
|
|
||||||
|
.. _static_use_hash:
|
||||||
|
|
||||||
|
``static_use_hash`` :
|
||||||
|
Enables `cache busting`_ of static files.
|
||||||
|
|
||||||
|
``default_locale`` :
|
||||||
|
SearXNG interface language. If blank, the locale is detected by using the
|
||||||
|
browser language. If it doesn't work, or you are deploying a language
|
||||||
|
specific instance of searx, a locale can be defined using an ISO language
|
||||||
|
code, like ``fr``, ``en``, ``de``.
|
||||||
|
|
||||||
|
``query_in_title`` :
|
||||||
|
When true, the result page's titles contains the query it decreases the
|
||||||
|
privacy, since the browser can records the page titles.
|
||||||
|
|
||||||
|
``infinite_scroll``:
|
||||||
|
When true, automatically loads the next page when scrolling to bottom of the current page.
|
||||||
|
|
||||||
|
``center_alignment`` : default ``false``
|
||||||
|
When enabled, the results are centered instead of being in the left (or RTL)
|
||||||
|
side of the screen. This setting only affects the *desktop layout*
|
||||||
|
(:origin:`min-width: @tablet <searx/static/themes/simple/src/less/definitions.less>`)
|
||||||
|
|
||||||
|
.. cache_url:
|
||||||
|
|
||||||
|
``cache_url`` : ``https://web.archive.org/web/``
|
||||||
|
URL prefix of the internet archive or cache, don't forgett trailing slash (if
|
||||||
|
needed). The default is https://web.archive.org/web/ alternatives are:
|
||||||
|
|
||||||
|
- https://webcache.googleusercontent.com/search?q=cache:
|
||||||
|
- https://archive.today/
|
||||||
|
|
||||||
|
``default_theme`` :
|
||||||
|
Name of the theme you want to use by default on your SearXNG instance.
|
||||||
|
|
||||||
|
``theme_args.simple_style``:
|
||||||
|
Style of simple theme: ``auto``, ``light``, ``dark``
|
||||||
|
|
||||||
|
``results_on_new_tab``:
|
||||||
|
Open result links in a new tab by default.
|
||||||
|
|
@ -9,7 +9,7 @@ SearXNG maintenance
|
||||||
- :ref:`toolboxing`
|
- :ref:`toolboxing`
|
||||||
- :ref:`uWSGI maintenance`
|
- :ref:`uWSGI maintenance`
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
How to contribute
|
How to contribute
|
||||||
=================
|
=================
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,11 @@
|
||||||
Demo Offline Engine
|
Demo Offline Engine
|
||||||
===================
|
===================
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
.. automodule:: searx.engines.demo_offline
|
.. automodule:: searx.engines.demo_offline
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
@ -4,6 +4,11 @@
|
||||||
Demo Online Engine
|
Demo Online Engine
|
||||||
==================
|
==================
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
.. automodule:: searx.engines.demo_online
|
.. automodule:: searx.engines.demo_online
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
@ -4,6 +4,11 @@
|
||||||
Engine Overview
|
Engine Overview
|
||||||
===============
|
===============
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 3
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
|
.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
|
||||||
|
|
||||||
.. sidebar:: Further reading ..
|
.. sidebar:: Further reading ..
|
||||||
|
|
@ -11,10 +16,6 @@ Engine Overview
|
||||||
- :ref:`configured engines`
|
- :ref:`configured engines`
|
||||||
- :ref:`settings engine`
|
- :ref:`settings engine`
|
||||||
|
|
||||||
.. contents::
|
|
||||||
:depth: 3
|
|
||||||
:backlinks: entry
|
|
||||||
|
|
||||||
SearXNG is a metasearch-engine_, so it uses different search engines to provide
|
SearXNG is a metasearch-engine_, so it uses different search engines to provide
|
||||||
better results.
|
better results.
|
||||||
|
|
||||||
|
|
@ -49,12 +50,12 @@ Engine File
|
||||||
categories list categories, in which the engine is working
|
categories list categories, in which the engine is working
|
||||||
paging boolean support multiple pages
|
paging boolean support multiple pages
|
||||||
time_range_support boolean support search time range
|
time_range_support boolean support search time range
|
||||||
engine_type str - ``online`` :ref:`[ref] <demo online engine>` by
|
engine_type str - ``online`` :ref:`[ref] <online engines>` by
|
||||||
default, other possibles values are:
|
default, other possibles values are:
|
||||||
- ``offline`` :ref:`[ref] <offline engines>`
|
- ``offline`` :ref:`[ref] <offline engines>`
|
||||||
- ``online_dictionary``
|
- ``online_dictionary`` :ref:`[ref] <online dictionary>`
|
||||||
- ``online_currency``
|
- ``online_currency`` :ref:`[ref] <online currency>`
|
||||||
- ``online_url_search``
|
- ``online_url_search`` :ref:`[ref] <online url search>`
|
||||||
======================= =========== ========================================================
|
======================= =========== ========================================================
|
||||||
|
|
||||||
.. _engine settings:
|
.. _engine settings:
|
||||||
|
|
@ -239,12 +240,18 @@ following parameters can be used to specify a search request:
|
||||||
.. _engine results:
|
.. _engine results:
|
||||||
.. _engine media types:
|
.. _engine media types:
|
||||||
|
|
||||||
Media Types
|
Result Types (``template``)
|
||||||
===========
|
===========================
|
||||||
|
|
||||||
Each result item of an engine can be of different media-types. Currently the
|
Each result item of an engine can be of different media-types. Currently the
|
||||||
following media-types are supported. To set another media-type as ``default``,
|
following media-types are supported. To set another media-type as
|
||||||
the parameter ``template`` must be set to the desired type.
|
:ref:`template default`, the parameter ``template`` must be set to the desired
|
||||||
|
type.
|
||||||
|
|
||||||
|
.. _template default:
|
||||||
|
|
||||||
|
``default``
|
||||||
|
-----------
|
||||||
|
|
||||||
.. table:: Parameter of the **default** media type:
|
.. table:: Parameter of the **default** media type:
|
||||||
:width: 100%
|
:width: 100%
|
||||||
|
|
@ -259,6 +266,11 @@ the parameter ``template`` must be set to the desired type.
|
||||||
========================= =====================================================
|
========================= =====================================================
|
||||||
|
|
||||||
|
|
||||||
|
.. _template images:
|
||||||
|
|
||||||
|
``images``
|
||||||
|
----------
|
||||||
|
|
||||||
.. table:: Parameter of the **images** media type:
|
.. table:: Parameter of the **images** media type:
|
||||||
:width: 100%
|
:width: 100%
|
||||||
|
|
||||||
|
|
@ -277,6 +289,11 @@ the parameter ``template`` must be set to the desired type.
|
||||||
========================= =====================================================
|
========================= =====================================================
|
||||||
|
|
||||||
|
|
||||||
|
.. _template videos:
|
||||||
|
|
||||||
|
``videos``
|
||||||
|
----------
|
||||||
|
|
||||||
.. table:: Parameter of the **videos** media type:
|
.. table:: Parameter of the **videos** media type:
|
||||||
:width: 100%
|
:width: 100%
|
||||||
|
|
||||||
|
|
@ -292,6 +309,12 @@ the parameter ``template`` must be set to the desired type.
|
||||||
thumbnail string, url to a small-preview image
|
thumbnail string, url to a small-preview image
|
||||||
========================= =====================================================
|
========================= =====================================================
|
||||||
|
|
||||||
|
|
||||||
|
.. _template torrent:
|
||||||
|
|
||||||
|
``torrent``
|
||||||
|
-----------
|
||||||
|
|
||||||
.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
|
.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
|
||||||
|
|
||||||
.. table:: Parameter of the **torrent** media type:
|
.. table:: Parameter of the **torrent** media type:
|
||||||
|
|
@ -315,6 +338,12 @@ the parameter ``template`` must be set to the desired type.
|
||||||
torrentfile string, torrentfile of the result
|
torrentfile string, torrentfile of the result
|
||||||
========================= =====================================================
|
========================= =====================================================
|
||||||
|
|
||||||
|
|
||||||
|
.. _template map:
|
||||||
|
|
||||||
|
``map``
|
||||||
|
-------
|
||||||
|
|
||||||
.. table:: Parameter of the **map** media type:
|
.. table:: Parameter of the **map** media type:
|
||||||
:width: 100%
|
:width: 100%
|
||||||
|
|
||||||
|
|
@ -342,6 +371,12 @@ the parameter ``template`` must be set to the desired type.
|
||||||
address.country country of object
|
address.country country of object
|
||||||
========================= =====================================================
|
========================= =====================================================
|
||||||
|
|
||||||
|
|
||||||
|
.. _template paper:
|
||||||
|
|
||||||
|
``paper``
|
||||||
|
---------
|
||||||
|
|
||||||
.. _BibTeX format: https://www.bibtex.com/g/bibtex-format/
|
.. _BibTeX format: https://www.bibtex.com/g/bibtex-format/
|
||||||
.. _BibTeX field types: https://en.wikipedia.org/wiki/BibTeX#Field_types
|
.. _BibTeX field types: https://en.wikipedia.org/wiki/BibTeX#Field_types
|
||||||
|
|
||||||
|
|
@ -430,3 +465,4 @@ the parameter ``template`` must be set to the desired type.
|
||||||
* - html_url
|
* - html_url
|
||||||
- :py:class:`str`
|
- :py:class:`str`
|
||||||
- URL to full article, HTML version
|
- URL to full article, HTML version
|
||||||
|
|
||||||
|
|
@ -1,15 +1,20 @@
|
||||||
.. _searx.enginelib:
|
.. _searx.enginelib:
|
||||||
|
|
||||||
============
|
==============
|
||||||
Engine model
|
Engine Library
|
||||||
============
|
==============
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
.. automodule:: searx.enginelib
|
.. automodule:: searx.enginelib
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
.. _searx.enginelib.traits:
|
.. _searx.enginelib.traits:
|
||||||
|
|
||||||
=============
|
|
||||||
Engine traits
|
Engine traits
|
||||||
=============
|
=============
|
||||||
|
|
||||||
9
docs/dev/engines/engines.rst
Normal file
9
docs/dev/engines/engines.rst
Normal file
|
|
@ -0,0 +1,9 @@
|
||||||
|
.. _searx.engines loader:
|
||||||
|
|
||||||
|
========================
|
||||||
|
SearXNG's engines loader
|
||||||
|
========================
|
||||||
|
|
||||||
|
.. automodule:: searx.engines
|
||||||
|
:members:
|
||||||
|
|
||||||
97
docs/dev/engines/index.rst
Normal file
97
docs/dev/engines/index.rst
Normal file
|
|
@ -0,0 +1,97 @@
|
||||||
|
.. _engine implementations:
|
||||||
|
|
||||||
|
======================
|
||||||
|
Engine Implementations
|
||||||
|
======================
|
||||||
|
|
||||||
|
Framework Components
|
||||||
|
====================
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 2
|
||||||
|
|
||||||
|
enginelib
|
||||||
|
engines
|
||||||
|
engine_overview
|
||||||
|
|
||||||
|
|
||||||
|
Engine Types
|
||||||
|
============
|
||||||
|
|
||||||
|
The :py:obj:`engine_type <searx.enginelib.Engine.engine_type>` of an engine
|
||||||
|
determines which :ref:`search processor <searx.search.processors>` is used by
|
||||||
|
the engine.
|
||||||
|
|
||||||
|
In this section a list of the enignes that are documented is given, a complete
|
||||||
|
list of the engines can be found in the source under: :origin:`searx/engines`.
|
||||||
|
|
||||||
|
.. _online engines:
|
||||||
|
|
||||||
|
Online Engines
|
||||||
|
--------------
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
- :py:obj:`processors.online <searx.search.processors.online>`
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 1
|
||||||
|
:glob:
|
||||||
|
|
||||||
|
demo/demo_online
|
||||||
|
xpath
|
||||||
|
online/*
|
||||||
|
|
||||||
|
.. _offline engines:
|
||||||
|
|
||||||
|
Offline Engines
|
||||||
|
---------------
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
- :py:obj:`processors.offline <searx.search.processors.offline>`
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 1
|
||||||
|
:glob:
|
||||||
|
|
||||||
|
offline_concept
|
||||||
|
demo/demo_offline
|
||||||
|
offline/*
|
||||||
|
|
||||||
|
.. _online url search:
|
||||||
|
|
||||||
|
Online URL Search
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
- :py:obj:`processors.online_url_search <searx.search.processors.online_url_search>`
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 1
|
||||||
|
:glob:
|
||||||
|
|
||||||
|
online_url_search/*
|
||||||
|
|
||||||
|
.. _online currency:
|
||||||
|
|
||||||
|
Online Currency
|
||||||
|
---------------
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
- :py:obj:`processors.online_currency <searx.search.processors.online_currency>`
|
||||||
|
|
||||||
|
*no engine of this type is documented yet / comming soon*
|
||||||
|
|
||||||
|
.. _online dictionary:
|
||||||
|
|
||||||
|
Online Dictionary
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
- :py:obj:`processors.online_dictionary <searx.search.processors.online_dictionary>`
|
||||||
|
|
||||||
|
*no engine of this type is documented yet / comming soon*
|
||||||
23
docs/dev/engines/offline/command-line-engines.rst
Normal file
23
docs/dev/engines/offline/command-line-engines.rst
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
.. _engine command:
|
||||||
|
|
||||||
|
====================
|
||||||
|
Command Line Engines
|
||||||
|
====================
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
- :origin:`command.py <searx/engines/command.py>`
|
||||||
|
- :ref:`offline engines`
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
Initial sponsored by `Search and Discovery Fund
|
||||||
|
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||||
|
|
||||||
|
.. automodule:: searx.engines.command
|
||||||
|
:members:
|
||||||
|
|
@ -1,3 +1,5 @@
|
||||||
|
.. _nosql engines:
|
||||||
|
|
||||||
===============
|
===============
|
||||||
NoSQL databases
|
NoSQL databases
|
||||||
===============
|
===============
|
||||||
|
|
@ -8,6 +10,16 @@ NoSQL databases
|
||||||
- `redis.io <https://redis.io/>`_
|
- `redis.io <https://redis.io/>`_
|
||||||
- `MongoDB <https://www.mongodb.com>`_
|
- `MongoDB <https://www.mongodb.com>`_
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
Initial sponsored by `Search and Discovery Fund
|
||||||
|
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||||
|
|
||||||
The following `NoSQL databases`_ are supported:
|
The following `NoSQL databases`_ are supported:
|
||||||
|
|
||||||
- :ref:`engine redis_server`
|
- :ref:`engine redis_server`
|
||||||
|
|
@ -30,15 +42,8 @@ can still add them and limit the access by setting ``tokens`` as described in
|
||||||
section :ref:`private engines`.
|
section :ref:`private engines`.
|
||||||
|
|
||||||
|
|
||||||
Configure the engines
|
|
||||||
=====================
|
|
||||||
|
|
||||||
`NoSQL databases`_ are used for storing arbitrary data without first defining
|
|
||||||
their structure.
|
|
||||||
|
|
||||||
|
|
||||||
Extra Dependencies
|
Extra Dependencies
|
||||||
------------------
|
==================
|
||||||
|
|
||||||
For using :ref:`engine redis_server` or :ref:`engine mongodb` you need to
|
For using :ref:`engine redis_server` or :ref:`engine mongodb` you need to
|
||||||
install additional packages in Python's Virtual Environment of your SearXNG
|
install additional packages in Python's Virtual Environment of your SearXNG
|
||||||
|
|
@ -49,6 +54,13 @@ instance. To switch into the environment (:ref:`searxng-src`) you can use
|
||||||
(searxng-pyenv)$ pip install ...
|
(searxng-pyenv)$ pip install ...
|
||||||
|
|
||||||
|
|
||||||
|
Configure the engines
|
||||||
|
=====================
|
||||||
|
|
||||||
|
`NoSQL databases`_ are used for storing arbitrary data without first defining
|
||||||
|
their structure.
|
||||||
|
|
||||||
|
|
||||||
.. _engine redis_server:
|
.. _engine redis_server:
|
||||||
|
|
||||||
Redis Server
|
Redis Server
|
||||||
|
|
@ -62,29 +74,9 @@ Redis Server
|
||||||
- redis.io_
|
- redis.io_
|
||||||
- :origin:`redis_server.py <searx/engines/redis_server.py>`
|
- :origin:`redis_server.py <searx/engines/redis_server.py>`
|
||||||
|
|
||||||
|
.. automodule:: searx.engines.redis_server
|
||||||
|
:members:
|
||||||
|
|
||||||
Redis is an open source (BSD licensed), in-memory data structure (key value
|
|
||||||
based) store. Before configuring the ``redis_server`` engine, you must install
|
|
||||||
the dependency redis_.
|
|
||||||
|
|
||||||
Select a database to search in and set its index in the option ``db``. You can
|
|
||||||
either look for exact matches or use partial keywords to find what you are
|
|
||||||
looking for by configuring ``exact_match_only``. You find an example
|
|
||||||
configuration below:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
# Required dependency: redis
|
|
||||||
|
|
||||||
- name: myredis
|
|
||||||
shortcut : rds
|
|
||||||
engine: redis_server
|
|
||||||
exact_match_only: false
|
|
||||||
host: '127.0.0.1'
|
|
||||||
port: 6379
|
|
||||||
enable_http: true
|
|
||||||
password: ''
|
|
||||||
db: 0
|
|
||||||
|
|
||||||
.. _engine mongodb:
|
.. _engine mongodb:
|
||||||
|
|
||||||
|
|
@ -99,37 +91,7 @@ MongoDB
|
||||||
- MongoDB_
|
- MongoDB_
|
||||||
- :origin:`mongodb.py <searx/engines/mongodb.py>`
|
- :origin:`mongodb.py <searx/engines/mongodb.py>`
|
||||||
|
|
||||||
MongoDB_ is a document based database program that handles JSON like data.
|
|
||||||
Before configuring the ``mongodb`` engine, you must install the dependency
|
|
||||||
redis_.
|
|
||||||
|
|
||||||
In order to query MongoDB_, you have to select a ``database`` and a
|
.. automodule:: searx.engines.mongodb
|
||||||
``collection``. Furthermore, you have to select a ``key`` that is going to be
|
:members:
|
||||||
searched. MongoDB_ also supports the option ``exact_match_only``, so configure
|
|
||||||
it as you wish. Below is an example configuration for using a MongoDB
|
|
||||||
collection:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
# MongoDB engine
|
|
||||||
# Required dependency: pymongo
|
|
||||||
|
|
||||||
- name: mymongo
|
|
||||||
engine: mongodb
|
|
||||||
shortcut: md
|
|
||||||
exact_match_only: false
|
|
||||||
host: '127.0.0.1'
|
|
||||||
port: 27017
|
|
||||||
enable_http: true
|
|
||||||
results_per_page: 20
|
|
||||||
database: 'business'
|
|
||||||
collection: 'reviews' # name of the db collection
|
|
||||||
key: 'name' # key in the collection to search for
|
|
||||||
|
|
||||||
|
|
||||||
Acknowledgment
|
|
||||||
==============
|
|
||||||
|
|
||||||
This development was sponsored by `Search and Discovery Fund
|
|
||||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
|
||||||
|
|
||||||
62
docs/dev/engines/offline/search-indexer-engines.rst
Normal file
62
docs/dev/engines/offline/search-indexer-engines.rst
Normal file
|
|
@ -0,0 +1,62 @@
|
||||||
|
=================
|
||||||
|
Local Search APIs
|
||||||
|
=================
|
||||||
|
|
||||||
|
.. sidebar:: further read
|
||||||
|
|
||||||
|
- `Comparison to alternatives
|
||||||
|
<https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html>`_
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 1
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
Initial sponsored by `Search and Discovery Fund
|
||||||
|
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||||
|
|
||||||
|
Administrators might find themselves wanting to integrate locally running search
|
||||||
|
engines. The following ones are supported for now:
|
||||||
|
|
||||||
|
* `Elasticsearch`_
|
||||||
|
* `Meilisearch`_
|
||||||
|
* `Solr`_
|
||||||
|
|
||||||
|
Each search engine is powerful, capable of full-text search. All of the engines
|
||||||
|
above are added to ``settings.yml`` just commented out, as you have to
|
||||||
|
``base_url`` for all them.
|
||||||
|
|
||||||
|
Please note that if you are not using HTTPS to access these engines, you have to
|
||||||
|
enable HTTP requests by setting ``enable_http`` to ``True``.
|
||||||
|
|
||||||
|
Furthermore, if you do not want to expose these engines on a public instance,
|
||||||
|
you can still add them and limit the access by setting ``tokens`` as described
|
||||||
|
in section :ref:`private engines`.
|
||||||
|
|
||||||
|
.. _engine meilisearch:
|
||||||
|
|
||||||
|
MeiliSearch
|
||||||
|
===========
|
||||||
|
|
||||||
|
.. automodule:: searx.engines.meilisearch
|
||||||
|
:members:
|
||||||
|
|
||||||
|
|
||||||
|
.. _engine elasticsearch:
|
||||||
|
|
||||||
|
Elasticsearch
|
||||||
|
=============
|
||||||
|
|
||||||
|
.. automodule:: searx.engines.elasticsearch
|
||||||
|
:members:
|
||||||
|
|
||||||
|
.. _engine solr:
|
||||||
|
|
||||||
|
Solr
|
||||||
|
====
|
||||||
|
|
||||||
|
.. automodule:: searx.engines.solr
|
||||||
|
:members:
|
||||||
|
|
||||||
|
|
@ -10,6 +10,16 @@ SQL Engines
|
||||||
- `PostgreSQL <https://www.postgresql.org>`_
|
- `PostgreSQL <https://www.postgresql.org>`_
|
||||||
- `MySQL <https://www.mysql.com>`_
|
- `MySQL <https://www.mysql.com>`_
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
|
.. sidebar:: info
|
||||||
|
|
||||||
|
Initial sponsored by `Search and Discovery Fund
|
||||||
|
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||||
|
|
||||||
With the *SQL engines* you can bind SQL databases into SearXNG. The following
|
With the *SQL engines* you can bind SQL databases into SearXNG. The following
|
||||||
Relational Database Management System (RDBMS) are supported:
|
Relational Database Management System (RDBMS) are supported:
|
||||||
|
|
||||||
|
|
@ -42,6 +52,18 @@ add them and limit the access by setting ``tokens`` as described in section
|
||||||
:ref:`private engines`.
|
:ref:`private engines`.
|
||||||
|
|
||||||
|
|
||||||
|
Extra Dependencies
|
||||||
|
==================
|
||||||
|
|
||||||
|
For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
|
||||||
|
install additional packages in Python's Virtual Environment of your SearXNG
|
||||||
|
instance. To switch into the environment (:ref:`searxng-src`) you can use
|
||||||
|
:ref:`searxng.sh`::
|
||||||
|
|
||||||
|
$ sudo utils/searxng.sh instance cmd bash
|
||||||
|
(searxng-pyenv)$ pip install ...
|
||||||
|
|
||||||
|
|
||||||
Configure the engines
|
Configure the engines
|
||||||
=====================
|
=====================
|
||||||
|
|
||||||
|
|
@ -64,45 +86,8 @@ SQLite
|
||||||
|
|
||||||
- :origin:`sqlite.py <searx/engines/sqlite.py>`
|
- :origin:`sqlite.py <searx/engines/sqlite.py>`
|
||||||
|
|
||||||
.. _MediathekView: https://mediathekview.de/
|
.. automodule:: searx.engines.sqlite
|
||||||
|
:members:
|
||||||
SQLite is a small, fast and reliable SQL database engine. It does not require
|
|
||||||
any extra dependency. To demonstrate the power of database engines, here is a
|
|
||||||
more complex example which reads from a MediathekView_ (DE) movie database. For
|
|
||||||
this example of the SQlite engine download the database:
|
|
||||||
|
|
||||||
- https://liste.mediathekview.de/filmliste-v2.db.bz2
|
|
||||||
|
|
||||||
and unpack into ``searx/data/filmliste-v2.db``. To search the database use e.g
|
|
||||||
Query to test: ``!mediathekview concert``
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
- name: mediathekview
|
|
||||||
engine: sqlite
|
|
||||||
disabled: False
|
|
||||||
categories: general
|
|
||||||
result_template: default.html
|
|
||||||
database: searx/data/filmliste-v2.db
|
|
||||||
query_str: >-
|
|
||||||
SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title,
|
|
||||||
COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url,
|
|
||||||
description AS content
|
|
||||||
FROM film
|
|
||||||
WHERE title LIKE :wildcard OR description LIKE :wildcard
|
|
||||||
ORDER BY duration DESC
|
|
||||||
|
|
||||||
|
|
||||||
Extra Dependencies
|
|
||||||
------------------
|
|
||||||
|
|
||||||
For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
|
|
||||||
install additional packages in Python's Virtual Environment of your SearXNG
|
|
||||||
instance. To switch into the environment (:ref:`searxng-src`) you can use
|
|
||||||
:ref:`searxng.sh`::
|
|
||||||
|
|
||||||
$ sudo utils/searxng.sh instance cmd bash
|
|
||||||
(searxng-pyenv)$ pip install ...
|
|
||||||
|
|
||||||
|
|
||||||
.. _engine postgresql:
|
.. _engine postgresql:
|
||||||
|
|
@ -115,20 +100,10 @@ PostgreSQL
|
||||||
.. sidebar:: info
|
.. sidebar:: info
|
||||||
|
|
||||||
- :origin:`postgresql.py <searx/engines/postgresql.py>`
|
- :origin:`postgresql.py <searx/engines/postgresql.py>`
|
||||||
- ``pip install`` psycopg2_
|
- ``pip install`` `psycopg2-binary <psycopg2>`_
|
||||||
|
|
||||||
PostgreSQL is a powerful and robust open source database. Before configuring
|
.. automodule:: searx.engines.postgresql
|
||||||
the PostgreSQL engine, you must install the dependency ``psychopg2``. You can
|
:members:
|
||||||
find an example configuration below:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
- name: my_database
|
|
||||||
engine: postgresql
|
|
||||||
database: my_database
|
|
||||||
username: searxng
|
|
||||||
password: password
|
|
||||||
query_str: 'SELECT * from my_table WHERE my_column = %(query)s'
|
|
||||||
|
|
||||||
.. _engine mysql_server:
|
.. _engine mysql_server:
|
||||||
|
|
||||||
|
|
@ -140,27 +115,7 @@ MySQL
|
||||||
- :origin:`mysql_server.py <searx/engines/mysql_server.py>`
|
- :origin:`mysql_server.py <searx/engines/mysql_server.py>`
|
||||||
- ``pip install`` :pypi:`mysql-connector-python <mysql-connector-python>`
|
- ``pip install`` :pypi:`mysql-connector-python <mysql-connector-python>`
|
||||||
|
|
||||||
MySQL is said to be the most popular open source database. Before enabling MySQL
|
|
||||||
engine, you must install the package ``mysql-connector-python``.
|
|
||||||
|
|
||||||
The authentication plugin is configurable by setting ``auth_plugin`` in the
|
.. automodule:: searx.engines.mysql_server
|
||||||
attributes. By default it is set to ``caching_sha2_password``. This is an
|
:members:
|
||||||
example configuration for querying a MySQL server:
|
|
||||||
|
|
||||||
.. code:: yaml
|
|
||||||
|
|
||||||
- name: my_database
|
|
||||||
engine: mysql_server
|
|
||||||
database: my_database
|
|
||||||
username: searxng
|
|
||||||
password: password
|
|
||||||
limit: 5
|
|
||||||
query_str: 'SELECT * from my_table WHERE my_column=%(query)s'
|
|
||||||
|
|
||||||
|
|
||||||
Acknowledgment
|
|
||||||
==============
|
|
||||||
|
|
||||||
This development was sponsored by `Search and Discovery Fund
|
|
||||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
|
||||||
|
|
||||||
|
|
@ -1,15 +1,14 @@
|
||||||
.. _offline engines:
|
|
||||||
|
|
||||||
===============
|
===============
|
||||||
Offline Engines
|
Offline Concept
|
||||||
===============
|
===============
|
||||||
|
|
||||||
.. sidebar:: offline engines
|
.. sidebar:: offline engines
|
||||||
|
|
||||||
- :ref:`demo offline engine`
|
- :ref:`demo offline engine`
|
||||||
- :ref:`sql engines`
|
|
||||||
- :ref:`engine command`
|
- :ref:`engine command`
|
||||||
- :origin:`Redis <searx/engines/redis_server.py>`
|
- :ref:`sql engines`
|
||||||
|
- :ref:`nosql engines`
|
||||||
|
- :py:obj:`searx.search.processors.offline`
|
||||||
|
|
||||||
To extend the functionality of SearXNG, offline engines are going to be
|
To extend the functionality of SearXNG, offline engines are going to be
|
||||||
introduced. An offline engine is an engine which does not need Internet
|
introduced. An offline engine is an engine which does not need Internet
|
||||||
|
|
@ -31,7 +30,6 @@ Programming Interface
|
||||||
in advance.
|
in advance.
|
||||||
|
|
||||||
:py:func:`search(query, params) <searx.engines.demo_offline.searc>`
|
:py:func:`search(query, params) <searx.engines.demo_offline.searc>`
|
||||||
|
|
||||||
Each offline engine has a function named ``search``. This function is
|
Each offline engine has a function named ``search``. This function is
|
||||||
responsible to perform a search and return the results in a presentable
|
responsible to perform a search and return the results in a presentable
|
||||||
format. (Where *presentable* means presentable by the selected result
|
format. (Where *presentable* means presentable by the selected result
|
||||||
|
|
@ -69,10 +67,3 @@ administrators can set token(s) for each of the :ref:`private engines`. If a
|
||||||
query contains a valid token, then SearXNG performs the requested private
|
query contains a valid token, then SearXNG performs the requested private
|
||||||
search. If not, requests from an offline engines return errors.
|
search. If not, requests from an offline engines return errors.
|
||||||
|
|
||||||
|
|
||||||
Acknowledgement
|
|
||||||
===============
|
|
||||||
|
|
||||||
This development was sponsored by `Search and Discovery Fund
|
|
||||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_ .
|
|
||||||
|
|
||||||
13
docs/dev/engines/online/annas_archive.rst
Normal file
13
docs/dev/engines/online/annas_archive.rst
Normal file
|
|
@ -0,0 +1,13 @@
|
||||||
|
.. _annas_archive engine:
|
||||||
|
|
||||||
|
==============
|
||||||
|
Anna's Archive
|
||||||
|
==============
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
|
.. automodule:: searx.engines.annas_archive
|
||||||
|
:members:
|
||||||
|
|
@ -4,6 +4,11 @@
|
||||||
Arch Linux
|
Arch Linux
|
||||||
==========
|
==========
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
.. automodule:: searx.engines.archlinux
|
.. automodule:: searx.engines.archlinux
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
Bing Engines
|
Bing Engines
|
||||||
============
|
============
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
@ -4,5 +4,10 @@
|
||||||
Dailymotion
|
Dailymotion
|
||||||
===========
|
===========
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
.. automodule:: searx.engines.dailymotion
|
.. automodule:: searx.engines.dailymotion
|
||||||
:members:
|
:members:
|
||||||
|
|
@ -1,10 +1,10 @@
|
||||||
.. _duckduckgo engines:
|
.. _duckduckgo engines:
|
||||||
|
|
||||||
=================
|
=================
|
||||||
DukcDukGo engines
|
DukcDukGo Engines
|
||||||
=================
|
=================
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
Google Engines
|
Google Engines
|
||||||
==============
|
==============
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
Peertube Engines
|
Peertube Engines
|
||||||
================
|
================
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
13
docs/dev/engines/online/recoll.rst
Normal file
13
docs/dev/engines/online/recoll.rst
Normal file
|
|
@ -0,0 +1,13 @@
|
||||||
|
.. _engine recoll:
|
||||||
|
|
||||||
|
=============
|
||||||
|
Recoll Engine
|
||||||
|
=============
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
|
.. automodule:: searx.engines.recoll
|
||||||
|
:members:
|
||||||
|
|
@ -1,10 +1,10 @@
|
||||||
.. _startpage engines:
|
.. _startpage engines:
|
||||||
|
|
||||||
=================
|
=================
|
||||||
Startpage engines
|
Startpage Engines
|
||||||
=================
|
=================
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
13
docs/dev/engines/online/torznab.rst
Normal file
13
docs/dev/engines/online/torznab.rst
Normal file
|
|
@ -0,0 +1,13 @@
|
||||||
|
.. _torznab engine:
|
||||||
|
|
||||||
|
==============
|
||||||
|
Torznab WebAPI
|
||||||
|
==============
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
|
.. automodule:: searx.engines.torznab
|
||||||
|
:members:
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
Wikimedia
|
Wikimedia
|
||||||
=========
|
=========
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
@ -4,5 +4,10 @@
|
||||||
Yahoo Engine
|
Yahoo Engine
|
||||||
============
|
============
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
.. automodule:: searx.engines.yahoo
|
.. automodule:: searx.engines.yahoo
|
||||||
:members:
|
:members:
|
||||||
|
|
@ -4,6 +4,11 @@
|
||||||
Tineye
|
Tineye
|
||||||
======
|
======
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
.. automodule:: searx.engines.tineye
|
.. automodule:: searx.engines.tineye
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
@ -4,6 +4,11 @@
|
||||||
XPath Engine
|
XPath Engine
|
||||||
============
|
============
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:depth: 2
|
||||||
|
:local:
|
||||||
|
:backlinks: entry
|
||||||
|
|
||||||
.. automodule:: searx.engines.xpath
|
.. automodule:: searx.engines.xpath
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
@ -4,12 +4,10 @@ Developer documentation
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
:caption: Contents
|
|
||||||
|
|
||||||
quickstart
|
quickstart
|
||||||
contribution_guide
|
contribution_guide
|
||||||
engine_overview
|
engines/index
|
||||||
offline_engines
|
|
||||||
search_api
|
search_api
|
||||||
plugins
|
plugins
|
||||||
translation
|
translation
|
||||||
|
|
|
||||||
|
|
@ -22,7 +22,7 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in
|
||||||
section :ref:`internet connectivity docker`.
|
section :ref:`internet connectivity docker`.
|
||||||
|
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -22,7 +22,7 @@ Calling the ``help`` target gives a first overview (``make help``):
|
||||||
|
|
||||||
.. program-output:: bash -c "cd ..; make --no-print-directory help"
|
.. program-output:: bash -c "cd ..; make --no-print-directory help"
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
@ -243,14 +243,14 @@ calling ``make clean`` stop all processes using the :ref:`make install` or
|
||||||
We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
|
We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
|
||||||
Documentation <contrib docs>` section. If you want to edit the documentation
|
Documentation <contrib docs>` section. If you want to edit the documentation
|
||||||
read our :ref:`make docs.live` section. If you are working in your own brand,
|
read our :ref:`make docs.live` section. If you are working in your own brand,
|
||||||
adjust your :ref:`settings global`.
|
adjust your :ref:`settings brand`.
|
||||||
|
|
||||||
.. _make docs.gh-pages:
|
.. _make docs.gh-pages:
|
||||||
|
|
||||||
``make docs.gh-pages``
|
``make docs.gh-pages``
|
||||||
======================
|
======================
|
||||||
|
|
||||||
To deploy on github.io first adjust your :ref:`settings global`. For any
|
To deploy on github.io first adjust your :ref:`settings brand`. For any
|
||||||
further read :ref:`deploy on github.io`.
|
further read :ref:`deploy on github.io`.
|
||||||
|
|
||||||
.. _make test:
|
.. _make test:
|
||||||
|
|
|
||||||
|
|
@ -37,7 +37,7 @@ docs.live <make docs.live>` to build HTML while editing.
|
||||||
- DOT_, `Graphviz's dot`_, Graphviz_
|
- DOT_, `Graphviz's dot`_, Graphviz_
|
||||||
|
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 3
|
:depth: 3
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,6 @@ developers.
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
:caption: Contents
|
|
||||||
|
|
||||||
update
|
update
|
||||||
standalone_searx.py
|
standalone_searx.py
|
||||||
|
|
|
||||||
|
|
@ -43,7 +43,6 @@ If you don't trust anyone, you can set up your own, see :ref:`installation`.
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
:caption: Contents
|
|
||||||
|
|
||||||
user/index
|
user/index
|
||||||
own-instance
|
own-instance
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,7 @@ Why use a private instance?
|
||||||
\.\. is a common question among SearXNG users. Before answering this
|
\.\. is a common question among SearXNG users. Before answering this
|
||||||
question, see what options a SearXNG user has.
|
question, see what options a SearXNG user has.
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -8,7 +8,6 @@ every item from the source code, but we will add documentation when requested.
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
:caption: Contents
|
|
||||||
:glob:
|
:glob:
|
||||||
|
|
||||||
searx.*
|
searx.*
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
Bot Detection
|
Bot Detection
|
||||||
=============
|
=============
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -1,2 +0,0 @@
|
||||||
.. automodule:: searx.engines.annas_archive
|
|
||||||
:members:
|
|
||||||
|
|
@ -1,8 +0,0 @@
|
||||||
.. _searx.engines:
|
|
||||||
|
|
||||||
=================
|
|
||||||
SearXNG's engines
|
|
||||||
=================
|
|
||||||
|
|
||||||
.. automodule:: searx.engines
|
|
||||||
:members:
|
|
||||||
|
|
@ -1,2 +0,0 @@
|
||||||
.. automodule:: searx.engines.torznab
|
|
||||||
:members:
|
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
Locales
|
Locales
|
||||||
=======
|
=======
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
Search processors
|
Search processors
|
||||||
=================
|
=================
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
@ -34,7 +34,7 @@ Online currency processor
|
||||||
.. automodule:: searx.search.processors.online_currency
|
.. automodule:: searx.search.processors.online_currency
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Online Dictionary processor
|
Online dictionary processor
|
||||||
===========================
|
===========================
|
||||||
|
|
||||||
.. automodule:: searx.search.processors.online_dictionary
|
.. automodule:: searx.search.processors.online_dictionary
|
||||||
|
|
|
||||||
|
|
@ -22,7 +22,7 @@ Configured Engines
|
||||||
called *tabs*), engines can be queried by their name or the categories they
|
called *tabs*), engines can be queried by their name or the categories they
|
||||||
belong to, by using a :ref:`\!bing syntax <search-syntax>`.
|
belong to, by using a :ref:`\!bing syntax <search-syntax>`.
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
User information
|
User information
|
||||||
================
|
================
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 3
|
:depth: 3
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -10,7 +10,6 @@ and developers.
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
:caption: Contents
|
|
||||||
|
|
||||||
searxng.sh
|
searxng.sh
|
||||||
lxc.sh
|
lxc.sh
|
||||||
|
|
|
||||||
|
|
@ -26,7 +26,7 @@ to care about*).
|
||||||
- `LXC/LXD Image Server`_
|
- `LXC/LXD Image Server`_
|
||||||
- `LXD@github`_
|
- `LXD@github`_
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,7 @@ script :origin:`utils/searxng.sh`.
|
||||||
- :ref:`installation nginx`
|
- :ref:`installation nginx`
|
||||||
- :ref:`installation apache`
|
- :ref:`installation apache`
|
||||||
|
|
||||||
.. contents:: Contents
|
.. contents::
|
||||||
:depth: 2
|
:depth: 2
|
||||||
:local:
|
:local:
|
||||||
:backlinks: entry
|
:backlinks: entry
|
||||||
|
|
|
||||||
|
|
@ -1,18 +1,15 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
"""Engine related implementations
|
"""Implementations of the framework for the SearXNG engines.
|
||||||
|
|
||||||
.. note::
|
.. hint::
|
||||||
|
|
||||||
The long term goal is to modularize all relevant implementations to the
|
The long term goal is to modularize all implementations of the engine
|
||||||
engines here in this Python package. In addition to improved modularization,
|
framework here in this Python package. ToDo:
|
||||||
this will also be necessary in part because the probability of circular
|
|
||||||
imports will increase due to the increased typification of implementations in
|
|
||||||
the future.
|
|
||||||
|
|
||||||
ToDo:
|
- move implementations of the :ref:`searx.engines loader` to a new module in
|
||||||
|
the :py:obj:`searx.enginelib` namespace.
|
||||||
|
|
||||||
- move :py:obj:`searx.engines.load_engine` to a new module `searx.enginelib`.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -36,7 +33,7 @@ class Engine: # pylint: disable=too-few-public-methods
|
||||||
# Common options in the engine module
|
# Common options in the engine module
|
||||||
|
|
||||||
engine_type: str
|
engine_type: str
|
||||||
"""Type of the engine (:origin:`searx/search/processors`)"""
|
"""Type of the engine (:ref:`searx.search.processors`)"""
|
||||||
|
|
||||||
paging: bool
|
paging: bool
|
||||||
"""Engine supports multiple pages."""
|
"""Engine supports multiple pages."""
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,6 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
"""This module implements the engine loader.
|
"""Load and initialize the ``engines``, see :py:func:`load_engines` and register
|
||||||
|
|
||||||
Load and initialize the ``engines``, see :py:func:`load_engines` and register
|
|
||||||
:py:obj:`engine_shortcuts`.
|
:py:obj:`engine_shortcuts`.
|
||||||
|
|
||||||
usage::
|
usage::
|
||||||
|
|
|
||||||
|
|
@ -1,24 +1,12 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
""".. _annas_archive engine:
|
"""`Anna's Archive`_ is a free non-profit online shadow library metasearch
|
||||||
|
engine providing access to a variety of book resources (also via IPFS), created
|
||||||
==============
|
by a team of anonymous archivists (AnnaArchivist_).
|
||||||
Anna's Archive
|
|
||||||
==============
|
|
||||||
|
|
||||||
.. _Anna's Archive: https://annas-archive.org/
|
.. _Anna's Archive: https://annas-archive.org/
|
||||||
.. _AnnaArchivist: https://annas-software.org/AnnaArchivist/annas-archive
|
.. _AnnaArchivist: https://annas-software.org/AnnaArchivist/annas-archive
|
||||||
|
|
||||||
`Anna's Archive`_ is a free non-profit online shadow library metasearch engine
|
|
||||||
providing access to a variety of book resources (also via IPFS), created by a
|
|
||||||
team of anonymous archivists (AnnaArchivist_).
|
|
||||||
|
|
||||||
.. contents:: Contents
|
|
||||||
:depth: 2
|
|
||||||
:local:
|
|
||||||
:backlinks: entry
|
|
||||||
|
|
||||||
|
|
||||||
Configuration
|
Configuration
|
||||||
=============
|
=============
|
||||||
|
|
||||||
|
|
@ -41,7 +29,6 @@ for *newest* articles and journals (PDF) / by shortcut ``!aaa <search-term>``.
|
||||||
aa_ext: 'pdf'
|
aa_ext: 'pdf'
|
||||||
aa_sort: 'newest'
|
aa_sort: 'newest'
|
||||||
|
|
||||||
|
|
||||||
Implementations
|
Implementations
|
||||||
===============
|
===============
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,77 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
"""
|
"""With *command engines* administrators can run engines to integrate arbitrary
|
||||||
Command (offline)
|
shell commands.
|
||||||
|
|
||||||
|
.. attention::
|
||||||
|
|
||||||
|
When creating and enabling a ``command`` engine on a public instance, you
|
||||||
|
must be careful to avoid leaking private data.
|
||||||
|
|
||||||
|
The easiest solution is to limit the access by setting ``tokens`` as described
|
||||||
|
in section :ref:`private engines`. The engine base is flexible. Only your
|
||||||
|
imagination can limit the power of this engine (and maybe security concerns).
|
||||||
|
|
||||||
|
Configuration
|
||||||
|
=============
|
||||||
|
|
||||||
|
The following options are available:
|
||||||
|
|
||||||
|
``command``:
|
||||||
|
A comma separated list of the elements of the command. A special token
|
||||||
|
``{{QUERY}}`` tells where to put the search terms of the user. Example:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
['ls', '-l', '-h', '{{QUERY}}']
|
||||||
|
|
||||||
|
``delimiter``:
|
||||||
|
A mapping containing a delimiter ``char`` and the *titles* of each element in
|
||||||
|
``keys``.
|
||||||
|
|
||||||
|
``parse_regex``:
|
||||||
|
A dict containing the regular expressions for each result key.
|
||||||
|
|
||||||
|
``query_type``:
|
||||||
|
|
||||||
|
The expected type of user search terms. Possible values: ``path`` and
|
||||||
|
``enum``.
|
||||||
|
|
||||||
|
``path``:
|
||||||
|
Checks if the user provided path is inside the working directory. If not,
|
||||||
|
the query is not executed.
|
||||||
|
|
||||||
|
``enum``:
|
||||||
|
Is a list of allowed search terms. If the user submits something which is
|
||||||
|
not included in the list, the query returns an error.
|
||||||
|
|
||||||
|
``query_enum``:
|
||||||
|
A list containing allowed search terms if ``query_type`` is set to ``enum``.
|
||||||
|
|
||||||
|
``working_dir``:
|
||||||
|
The directory where the command has to be executed. Default: ``./``.
|
||||||
|
|
||||||
|
``result_separator``:
|
||||||
|
The character that separates results. Default: ``\\n``.
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
The example engine below can be used to find files with a specific name in the
|
||||||
|
configured working directory:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
- name: find
|
||||||
|
engine: command
|
||||||
|
command: ['find', '.', '-name', '{{QUERY}}']
|
||||||
|
query_type: path
|
||||||
|
shortcut: fnd
|
||||||
|
delimiter:
|
||||||
|
chars: ' '
|
||||||
|
keys: ['line']
|
||||||
|
|
||||||
|
Implementations
|
||||||
|
===============
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import re
|
import re
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,44 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
"""
|
""".. sidebar:: info
|
||||||
Elasticsearch
|
|
||||||
|
- :origin:`elasticsearch.py <searx/engines/elasticsearch.py>`
|
||||||
|
- `Elasticsearch <https://www.elastic.co/elasticsearch/>`_
|
||||||
|
- `Elasticsearch Guide
|
||||||
|
<https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html>`_
|
||||||
|
- `Install Elasticsearch
|
||||||
|
<https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html>`_
|
||||||
|
|
||||||
|
Elasticsearch_ supports numerous ways to query the data it is storing. At the
|
||||||
|
moment the engine supports the most popular search methods (``query_type``):
|
||||||
|
|
||||||
|
- ``match``,
|
||||||
|
- ``simple_query_string``,
|
||||||
|
- ``term`` and
|
||||||
|
- ``terms``.
|
||||||
|
|
||||||
|
If none of the methods fit your use case, you can select ``custom`` query type
|
||||||
|
and provide the JSON payload to submit to Elasticsearch in
|
||||||
|
``custom_query_json``.
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
The following is an example configuration for an Elasticsearch_ instance with
|
||||||
|
authentication configured to read from ``my-index`` index.
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
- name: elasticsearch
|
||||||
|
shortcut: es
|
||||||
|
engine: elasticsearch
|
||||||
|
base_url: http://localhost:9200
|
||||||
|
username: elastic
|
||||||
|
password: changeme
|
||||||
|
index: my-index
|
||||||
|
query_type: match
|
||||||
|
# custom_query_json: '{ ... }'
|
||||||
|
enable_http: true
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from json import loads, dumps
|
from json import loads, dumps
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,35 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
"""
|
""".. sidebar:: info
|
||||||
Meilisearch
|
|
||||||
|
- :origin:`meilisearch.py <searx/engines/meilisearch.py>`
|
||||||
|
- `MeiliSearch <https://www.meilisearch.com>`_
|
||||||
|
- `MeiliSearch Documentation <https://docs.meilisearch.com/>`_
|
||||||
|
- `Install MeiliSearch
|
||||||
|
<https://docs.meilisearch.com/learn/getting_started/installation.html>`_
|
||||||
|
|
||||||
|
MeiliSearch_ is aimed at individuals and small companies. It is designed for
|
||||||
|
small-scale (less than 10 million documents) data collections. E.g. it is great
|
||||||
|
for storing web pages you have visited and searching in the contents later.
|
||||||
|
|
||||||
|
The engine supports faceted search, so you can search in a subset of documents
|
||||||
|
of the collection. Furthermore, you can search in MeiliSearch_ instances that
|
||||||
|
require authentication by setting ``auth_token``.
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
Here is a simple example to query a Meilisearch instance:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
- name: meilisearch
|
||||||
|
engine: meilisearch
|
||||||
|
shortcut: mes
|
||||||
|
base_url: http://localhost:7700
|
||||||
|
index: my-index
|
||||||
|
enable_http: true
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# pylint: disable=global-statement
|
# pylint: disable=global-statement
|
||||||
|
|
|
||||||
|
|
@ -1,11 +1,53 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
"""MongoDB engine (Offline)
|
"""MongoDB_ is a document based database program that handles JSON like data.
|
||||||
|
Before configuring the ``mongodb`` engine, you must install the dependency
|
||||||
|
pymongo_.
|
||||||
|
|
||||||
|
Configuration
|
||||||
|
=============
|
||||||
|
|
||||||
|
In order to query MongoDB_, you have to select a ``database`` and a
|
||||||
|
``collection``. Furthermore, you have to select a ``key`` that is going to be
|
||||||
|
searched. MongoDB_ also supports the option ``exact_match_only``, so configure
|
||||||
|
it as you wish.
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
Below is an example configuration for using a MongoDB collection:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
# MongoDB engine
|
||||||
|
# Required dependency: pymongo
|
||||||
|
|
||||||
|
- name: mymongo
|
||||||
|
engine: mongodb
|
||||||
|
shortcut: md
|
||||||
|
exact_match_only: false
|
||||||
|
host: '127.0.0.1'
|
||||||
|
port: 27017
|
||||||
|
enable_http: true
|
||||||
|
results_per_page: 20
|
||||||
|
database: 'business'
|
||||||
|
collection: 'reviews' # name of the db collection
|
||||||
|
key: 'name' # key in the collection to search for
|
||||||
|
|
||||||
|
Implementations
|
||||||
|
===============
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import re
|
import re
|
||||||
from pymongo import MongoClient # pyright: ignore # pylint: disable=import-error
|
|
||||||
|
try:
|
||||||
|
from pymongo import MongoClient # type: ignore
|
||||||
|
except ImportError:
|
||||||
|
# import error is ignored because the admin has to install pymongo manually
|
||||||
|
# to use the engine
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
engine_type = 'offline'
|
engine_type = 'offline'
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,37 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
"""MySQL database (offline)
|
"""MySQL is said to be the most popular open source database. Before enabling
|
||||||
|
MySQL engine, you must install the package ``mysql-connector-python``.
|
||||||
|
|
||||||
|
The authentication plugin is configurable by setting ``auth_plugin`` in the
|
||||||
|
attributes. By default it is set to ``caching_sha2_password``.
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
This is an example configuration for querying a MySQL server:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
- name: my_database
|
||||||
|
engine: mysql_server
|
||||||
|
database: my_database
|
||||||
|
username: searxng
|
||||||
|
password: password
|
||||||
|
limit: 5
|
||||||
|
query_str: 'SELECT * from my_table WHERE my_column=%(query)s'
|
||||||
|
|
||||||
|
Implementations
|
||||||
|
===============
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# import error is ignored because the admin has to install mysql manually to use
|
try:
|
||||||
# the engine
|
import mysql.connector # type: ignore
|
||||||
import mysql.connector # pyright: ignore # pylint: disable=import-error
|
except ImportError:
|
||||||
|
# import error is ignored because the admin has to install mysql manually to use
|
||||||
|
# the engine
|
||||||
|
pass
|
||||||
|
|
||||||
engine_type = 'offline'
|
engine_type = 'offline'
|
||||||
auth_plugin = 'caching_sha2_password'
|
auth_plugin = 'caching_sha2_password'
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,33 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
"""PostgreSQL database (offline)
|
"""PostgreSQL is a powerful and robust open source database. Before configuring
|
||||||
|
the PostgreSQL engine, you must install the dependency ``psychopg2``.
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
Below is an example configuration:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
- name: my_database
|
||||||
|
engine: postgresql
|
||||||
|
database: my_database
|
||||||
|
username: searxng
|
||||||
|
password: password
|
||||||
|
query_str: 'SELECT * from my_table WHERE my_column = %(query)s'
|
||||||
|
|
||||||
|
Implementations
|
||||||
|
===============
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# import error is ignored because the admin has to install mysql manually to use
|
try:
|
||||||
# the engine
|
import psycopg2 # type: ignore
|
||||||
import psycopg2 # pyright: ignore # pylint: disable=import-error
|
except ImportError:
|
||||||
|
# import error is ignored because the admin has to install postgresql
|
||||||
|
# manually to use the engine.
|
||||||
|
pass
|
||||||
|
|
||||||
engine_type = 'offline'
|
engine_type = 'offline'
|
||||||
host = "127.0.0.1"
|
host = "127.0.0.1"
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,51 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
"""
|
# lint: pylint
|
||||||
Recoll (local search engine)
|
""".. sidebar:: info
|
||||||
|
|
||||||
|
- `Recoll <https://www.lesbonscomptes.com/recoll/>`_
|
||||||
|
- `recoll-webui <https://framagit.org/medoc92/recollwebui.git>`_
|
||||||
|
- :origin:`searx/engines/recoll.py`
|
||||||
|
|
||||||
|
Recoll_ is a desktop full-text search tool based on Xapian. By itself Recoll_
|
||||||
|
does not offer WEB or API access, this can be achieved using recoll-webui_
|
||||||
|
|
||||||
|
Configuration
|
||||||
|
=============
|
||||||
|
|
||||||
|
You must configure the following settings:
|
||||||
|
|
||||||
|
``base_url``:
|
||||||
|
Location where recoll-webui can be reached.
|
||||||
|
|
||||||
|
``mount_prefix``:
|
||||||
|
Location where the file hierarchy is mounted on your *local* filesystem.
|
||||||
|
|
||||||
|
``dl_prefix``:
|
||||||
|
Location where the file hierarchy as indexed by recoll can be reached.
|
||||||
|
|
||||||
|
``search_dir``:
|
||||||
|
Part of the indexed file hierarchy to be search, if empty the full domain is
|
||||||
|
searched.
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
Scenario:
|
||||||
|
|
||||||
|
#. Recoll indexes a local filesystem mounted in ``/export/documents/reference``,
|
||||||
|
#. the Recoll search interface can be reached at https://recoll.example.org/ and
|
||||||
|
#. the contents of this filesystem can be reached though https://download.example.org/reference
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
base_url: https://recoll.example.org/
|
||||||
|
mount_prefix: /export/documents
|
||||||
|
dl_prefix: https://download.example.org
|
||||||
|
search_dir: ''
|
||||||
|
|
||||||
|
Implementations
|
||||||
|
===============
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from datetime import date, timedelta
|
from datetime import date, timedelta
|
||||||
|
|
@ -33,7 +78,7 @@ embedded_url = '<{ttype} controls height="166px" ' + 'src="{url}" type="{mtype}"
|
||||||
|
|
||||||
# helper functions
|
# helper functions
|
||||||
def get_time_range(time_range):
|
def get_time_range(time_range):
|
||||||
sw = {'day': 1, 'week': 7, 'month': 30, 'year': 365}
|
sw = {'day': 1, 'week': 7, 'month': 30, 'year': 365} # pylint: disable=invalid-name
|
||||||
|
|
||||||
offset = sw.get(time_range, 0)
|
offset = sw.get(time_range, 0)
|
||||||
if not offset:
|
if not offset:
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,37 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
"""Redis engine (offline)
|
"""Redis is an open source (BSD licensed), in-memory data structure (key value
|
||||||
|
based) store. Before configuring the ``redis_server`` engine, you must install
|
||||||
|
the dependency redis_.
|
||||||
|
|
||||||
|
Configuration
|
||||||
|
=============
|
||||||
|
|
||||||
|
Select a database to search in and set its index in the option ``db``. You can
|
||||||
|
either look for exact matches or use partial keywords to find what you are
|
||||||
|
looking for by configuring ``exact_match_only``.
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
Below is an example configuration:
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
# Required dependency: redis
|
||||||
|
|
||||||
|
- name: myredis
|
||||||
|
shortcut : rds
|
||||||
|
engine: redis_server
|
||||||
|
exact_match_only: false
|
||||||
|
host: '127.0.0.1'
|
||||||
|
port: 6379
|
||||||
|
enable_http: true
|
||||||
|
password: ''
|
||||||
|
db: 0
|
||||||
|
|
||||||
|
Implementations
|
||||||
|
===============
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,31 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
"""
|
""".. sidebar:: info
|
||||||
Solr
|
|
||||||
|
- :origin:`solr.py <searx/engines/solr.py>`
|
||||||
|
- `Solr <https://solr.apache.org>`_
|
||||||
|
- `Solr Resources <https://solr.apache.org/resources.html>`_
|
||||||
|
- `Install Solr <https://solr.apache.org/guide/installing-solr.html>`_
|
||||||
|
|
||||||
|
Solr_ is a popular search engine based on Lucene, just like Elasticsearch_. But
|
||||||
|
instead of searching in indices, you can search in collections.
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
This is an example configuration for searching in the collection
|
||||||
|
``my-collection`` and get the results in ascending order.
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
- name: solr
|
||||||
|
engine: solr
|
||||||
|
shortcut: slr
|
||||||
|
base_url: http://localhost:8983
|
||||||
|
collection: my-collection
|
||||||
|
sort: asc
|
||||||
|
enable_http: true
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# pylint: disable=global-statement
|
# pylint: disable=global-statement
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,40 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
|
"""SQLite is a small, fast and reliable SQL database engine. It does not require
|
||||||
|
any extra dependency.
|
||||||
|
|
||||||
"""SQLite database (Offline)
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
.. _MediathekView: https://mediathekview.de/
|
||||||
|
|
||||||
|
To demonstrate the power of database engines, here is a more complex example
|
||||||
|
which reads from a MediathekView_ (DE) movie database. For this example of the
|
||||||
|
SQlite engine download the database:
|
||||||
|
|
||||||
|
- https://liste.mediathekview.de/filmliste-v2.db.bz2
|
||||||
|
|
||||||
|
and unpack into ``searx/data/filmliste-v2.db``. To search the database use e.g
|
||||||
|
Query to test: ``!mediathekview concert``
|
||||||
|
|
||||||
|
.. code:: yaml
|
||||||
|
|
||||||
|
- name: mediathekview
|
||||||
|
engine: sqlite
|
||||||
|
disabled: False
|
||||||
|
categories: general
|
||||||
|
result_template: default.html
|
||||||
|
database: searx/data/filmliste-v2.db
|
||||||
|
query_str: >-
|
||||||
|
SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title,
|
||||||
|
COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url,
|
||||||
|
description AS content
|
||||||
|
FROM film
|
||||||
|
WHERE title LIKE :wildcard OR description LIKE :wildcard
|
||||||
|
ORDER BY duration DESC
|
||||||
|
|
||||||
|
Implementations
|
||||||
|
===============
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
@ -26,14 +59,15 @@ def init(engine_settings):
|
||||||
|
|
||||||
@contextlib.contextmanager
|
@contextlib.contextmanager
|
||||||
def sqlite_cursor():
|
def sqlite_cursor():
|
||||||
"""Implements a `Context Manager`_ for a :py:obj:`sqlite3.Cursor`.
|
"""Implements a :py:obj:`Context Manager <contextlib.contextmanager>` for a
|
||||||
|
:py:obj:`sqlite3.Cursor`.
|
||||||
|
|
||||||
Open database in read only mode: if the database doesn't exist.
|
Open database in read only mode: if the database doesn't exist. The default
|
||||||
The default mode creates an empty file on the file system.
|
mode creates an empty file on the file system. See:
|
||||||
|
|
||||||
see:
|
|
||||||
* https://docs.python.org/3/library/sqlite3.html#sqlite3.connect
|
* https://docs.python.org/3/library/sqlite3.html#sqlite3.connect
|
||||||
* https://www.sqlite.org/uri.html
|
* https://www.sqlite.org/uri.html
|
||||||
|
|
||||||
"""
|
"""
|
||||||
uri = 'file:' + database + '?mode=ro'
|
uri = 'file:' + database + '?mode=ro'
|
||||||
with contextlib.closing(sqlite3.connect(uri, uri=True)) as connect:
|
with contextlib.closing(sqlite3.connect(uri, uri=True)) as connect:
|
||||||
|
|
|
||||||
|
|
@ -1,17 +1,6 @@
|
||||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
""".. _torznab engine:
|
"""Torznab_ is an API specification that provides a standardized way to query
|
||||||
|
|
||||||
==============
|
|
||||||
Torznab WebAPI
|
|
||||||
==============
|
|
||||||
|
|
||||||
.. contents:: Contents
|
|
||||||
:depth: 2
|
|
||||||
:local:
|
|
||||||
:backlinks: entry
|
|
||||||
|
|
||||||
Torznab_ is an API specification that provides a standardized way to query
|
|
||||||
torrent site for content. It is used by a number of torrent applications,
|
torrent site for content. It is used by a number of torrent applications,
|
||||||
including Prowlarr_ and Jackett_.
|
including Prowlarr_ and Jackett_.
|
||||||
|
|
||||||
|
|
@ -55,7 +44,6 @@ The engine has the following settings:
|
||||||
.. _Jackett-categories:
|
.. _Jackett-categories:
|
||||||
https://github.com/Jackett/Jackett/wiki/Jackett-Categories
|
https://github.com/Jackett/Jackett/wiki/Jackett-Categories
|
||||||
|
|
||||||
|
|
||||||
Implementations
|
Implementations
|
||||||
===============
|
===============
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -3,8 +3,55 @@
|
||||||
"""The XPath engine is a *generic* engine with which it is possible to configure
|
"""The XPath engine is a *generic* engine with which it is possible to configure
|
||||||
engines in the settings.
|
engines in the settings.
|
||||||
|
|
||||||
Here is a simple example of a XPath engine configured in the
|
.. _XPath selector: https://quickref.me/xpath.html#xpath-selectors
|
||||||
:ref:`settings engine` section, further read :ref:`engines-dev`.
|
|
||||||
|
Configuration
|
||||||
|
=============
|
||||||
|
|
||||||
|
Request:
|
||||||
|
|
||||||
|
- :py:obj:`search_url`
|
||||||
|
- :py:obj:`lang_all`
|
||||||
|
- :py:obj:`soft_max_redirects`
|
||||||
|
- :py:obj:`cookies`
|
||||||
|
- :py:obj:`headers`
|
||||||
|
|
||||||
|
Paging:
|
||||||
|
|
||||||
|
- :py:obj:`paging`
|
||||||
|
- :py:obj:`page_size`
|
||||||
|
- :py:obj:`first_page_num`
|
||||||
|
|
||||||
|
Time Range:
|
||||||
|
|
||||||
|
- :py:obj:`time_range_support`
|
||||||
|
- :py:obj:`time_range_url`
|
||||||
|
- :py:obj:`time_range_map`
|
||||||
|
|
||||||
|
Safe-Search:
|
||||||
|
|
||||||
|
- :py:obj:`safe_search_support`
|
||||||
|
- :py:obj:`safe_search_map`
|
||||||
|
|
||||||
|
Response:
|
||||||
|
|
||||||
|
- :py:obj:`no_result_for_http_status`
|
||||||
|
|
||||||
|
`XPath selector`_:
|
||||||
|
|
||||||
|
- :py:obj:`results_xpath`
|
||||||
|
- :py:obj:`url_xpath`
|
||||||
|
- :py:obj:`title_xpath`
|
||||||
|
- :py:obj:`content_xpath`
|
||||||
|
- :py:obj:`thumbnail_xpath`
|
||||||
|
- :py:obj:`suggestion_xpath`
|
||||||
|
|
||||||
|
|
||||||
|
Example
|
||||||
|
=======
|
||||||
|
|
||||||
|
Here is a simple example of a XPath engine configured in the :ref:`settings
|
||||||
|
engine` section, further read :ref:`engines-dev`.
|
||||||
|
|
||||||
.. code:: yaml
|
.. code:: yaml
|
||||||
|
|
||||||
|
|
@ -16,6 +63,9 @@ Here is a simple example of a XPath engine configured in the
|
||||||
title_xpath : //article[@class="repo-summary"]//a[@class="repo-link"]
|
title_xpath : //article[@class="repo-summary"]//a[@class="repo-link"]
|
||||||
content_xpath : //article[@class="repo-summary"]/p
|
content_xpath : //article[@class="repo-summary"]/p
|
||||||
|
|
||||||
|
Implementations
|
||||||
|
===============
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from urllib.parse import urlencode
|
from urllib.parse import urlencode
|
||||||
|
|
@ -74,30 +124,33 @@ soft_max_redirects = 0
|
||||||
'''Maximum redirects, soft limit. Record an error but don't stop the engine'''
|
'''Maximum redirects, soft limit. Record an error but don't stop the engine'''
|
||||||
|
|
||||||
results_xpath = ''
|
results_xpath = ''
|
||||||
'''XPath selector for the list of result items'''
|
'''`XPath selector`_ for the list of result items'''
|
||||||
|
|
||||||
url_xpath = None
|
url_xpath = None
|
||||||
'''XPath selector of result's ``url``.'''
|
'''`XPath selector`_ of result's ``url``.'''
|
||||||
|
|
||||||
content_xpath = None
|
content_xpath = None
|
||||||
'''XPath selector of result's ``content``.'''
|
'''`XPath selector`_ of result's ``content``.'''
|
||||||
|
|
||||||
title_xpath = None
|
title_xpath = None
|
||||||
'''XPath selector of result's ``title``.'''
|
'''`XPath selector`_ of result's ``title``.'''
|
||||||
|
|
||||||
thumbnail_xpath = False
|
thumbnail_xpath = False
|
||||||
'''XPath selector of result's ``img_src``.'''
|
'''`XPath selector`_ of result's ``img_src``.'''
|
||||||
|
|
||||||
suggestion_xpath = ''
|
suggestion_xpath = ''
|
||||||
'''XPath selector of result's ``suggestion``.'''
|
'''`XPath selector`_ of result's ``suggestion``.'''
|
||||||
|
|
||||||
cached_xpath = ''
|
cached_xpath = ''
|
||||||
cached_url = ''
|
cached_url = ''
|
||||||
|
|
||||||
cookies = {}
|
cookies = {}
|
||||||
|
'''Some engines might offer different result based on cookies.
|
||||||
|
Possible use-case: To set safesearch cookie.'''
|
||||||
|
|
||||||
headers = {}
|
headers = {}
|
||||||
'''Some engines might offer different result based on cookies or headers.
|
'''Some engines might offer different result based headers. Possible use-case:
|
||||||
Possible use-case: To set safesearch cookie or header to moderate.'''
|
To set header to moderate.'''
|
||||||
|
|
||||||
paging = False
|
paging = False
|
||||||
'''Engine supports paging [True or False].'''
|
'''Engine supports paging [True or False].'''
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue