OwlBurst/Ponysearch
1
0
Fork 0
forked from Ponysearch/Ponysearch
Code Pull requests Activity

Merge pull request #2450 from return42/lxc-doc

Browse source 
[doc] update & fix documentation of the "SearXNG LXC suite"
This commit is contained in:
Markus Heiser 2023-05-21 18:26:14 +02:00 committed by GitHub
commit ddbab4b3c9
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
6 changed files with 380 additions and 225 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

341
docs/dev/lxcdev.rst
View file

@ -16,6 +16,12 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in
should have a serious meaning about the terms: *distributed*, *merge* and should have a serious meaning about the terms: *distributed*, *merge* and
*linux container*. *linux container*.
**hint**
If you have issues with the internet connectivity of your containers read
section :ref:`internet connectivity docker`.
.. contents:: Contents .. contents:: Contents
:depth: 2 :depth: 2
:local: :local:
@ -25,38 +31,38 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in
Motivation Motivation
========== ==========
Usually in our development cycle, we edit the sources and run some test and/or Most often in our development cycle, we edit the sources and run some test
builds by using ``make`` :ref:`[ref] <makefile>` before we commit. This cycle and/or builds by using ``make`` :ref:`[ref] <makefile>` before we commit. This
is simple and perfect but might fail in some aspects we should not overlook. cycle is simple and perfect but might fail in some aspects we should not
overlook.
**The environment in which we run all our development processes matters!** **The environment in which we run all our development processes matters!**
The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but
they do not have access to all prerequisites. For example, there may have these tools do not have access to all prerequisites. For example, there may
dependencies on packages that are installed on the developer's desktop, but have dependencies on packages that are installed on developer's desktop, but
usually are not preinstalled on a server or client system. Another example is; usually are not preinstalled on a server or client system. Another example is;
settings have been made to the software on developer's desktop that would never settings have been made to the software on developer's desktop that would never
be set on a *production* system. be set on a *production* system.
**Linux Containers are isolate environments and not to mix up all the **Linux Containers are isolate environments**, we use them to not mix up all
prerequisites from various projects on developer's desktop is always a good the prerequisites from various projects on developer's desktop.
choice.**
The scripts from :ref:`searx_utils` can divide in those to install and maintain The scripts from :ref:`searx_utils` can divide in those to install and maintain
software: software
- :ref:`searxng.sh` - :ref:`searxng.sh`
and the script :ref:`lxc.sh`, with we can scale our installation, maintenance or and the script
even development tasks over a stack of isolated containers / what we call the:
**SearXNG LXC suite** - :ref:`lxc.sh`
.. hint:: with we can scale our installation, maintenance or even development tasks over a
stack of isolated containers / what we call the:
If you see any problems with the internet connectivity of your - :ref:`searxng lxc suite`
containers read section :ref:`internet connectivity docker`.
.. _lxcdev install searxng:
Gentlemen, start your engines! Gentlemen, start your engines!
============================== ==============================
@ -69,7 +75,7 @@ once:
.. tabs:: .. tabs::
.. group-tab:: desktop .. group-tab:: desktop (HOST)
.. code:: bash .. code:: bash
@ -81,7 +87,7 @@ fork:
.. tabs:: .. tabs::
.. group-tab:: desktop .. group-tab:: desktop (HOST)
.. code:: bash .. code:: bash
@ -89,76 +95,61 @@ fork:
$ git clone https://github.com/searxng/searxng.git searxng $ git clone https://github.com/searxng/searxng.git searxng
$ cd searxng $ cd searxng
.. sidebar:: The ``searxng-archlinux`` container
is the base of all our exercises here.
The :ref:`lxc-searxng.env` consists of several images, see ``export The :ref:`lxc-searxng.env` consists of several images, see ``export
LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`. For this blog post LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`.
we exercise on a archlinux_ image. The container of this image is named For this blog post we exercise on a archlinux_ image. The container of this
``searxng-archlinux``. Lets build the container, but be sure that this container image is named ``searxng-archlinux``.
does not already exists, so first lets remove possible old one:
Lets build the container, but be sure that this container does not already
exists, so first lets remove possible old one:
.. tabs:: .. tabs::
.. group-tab:: desktop .. group-tab:: desktop (HOST)
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh remove searxng-archlinux $ sudo -H ./utils/lxc.sh remove searxng-archlinux
$ sudo -H ./utils/lxc.sh build searxng-archlinux $ sudo -H ./utils/lxc.sh build searxng-archlinux
.. sidebar:: The ``searxng-archlinux`` container
is the base of all our exercises here. .. sidebar:: further read
In this container we install all services :ref:`including searx, morty & filtron - :ref:`lxc.sh install suite`
<lxc.sh install suite>` in once: - :ref:`installation nginx`
To install the complete :ref:`SearXNG suite <searxng lxc suite>` and the HTTP
proxy :ref:`installation nginx` into the archlinux container run:
.. tabs:: .. tabs::
.. group-tab:: desktop .. group-tab:: desktop (HOST)
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux $ sudo -H ./utils/lxc.sh install suite searxng-archlinux
$ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
To proxy HTTP from filtron and morty in the container to the outside of the $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
container, install nginx into the container. Once for the bot blocker filtron:
.. tabs::
.. group-tab:: desktop
.. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
./utils/filtron.sh nginx install
... ...
INFO: got 429 from http://10.174.184.156/searx [searxng-archlinux] SEARXNG_URL : http://n.n.n.140/searxng
and once for the content sanitizer (content proxy morty):
.. tabs::
.. group-tab:: desktop
.. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
./utils/morty.sh nginx install
...
INFO: got 200 from http://10.174.184.156/morty/
.. sidebar:: Fully functional SearXNG suite .. sidebar:: Fully functional SearXNG suite
From here on you have a fully functional SearXNG suite running with bot From here on you have a fully functional SearXNG suite (including a
blocker (filtron) and WEB content sanitizer (content proxy morty), both are :ref:`redis db`).
needed for a *privacy protecting* search engine.
On your system, the IP of your ``searxng-archlinux`` container differs from In such a SearXNG suite admins can maintain and access the debug log of the
http://10.174.184.156/searx, just open the URL reported in your installation services quite easy.
protocol in your WEB browser from the desktop to test the instance from outside
of the container.
In such a earXNG suite admins can maintain and access the debug log of the In the example above the SearXNG instance in the container is wrapped to
different services quite easy. ``http://n.n.n.140/searxng`` to the HOST system. Note, on your HOST system, the
IP of your ``searxng-archlinux`` container is different to this example. To
test the instance in the conatiner from outside of the container, in your WEB
browser on your desktop just open the URL reported in your installation
.. _working in containers: .. _working in containers:
@ -166,77 +157,76 @@ In containers, work as usual
============================ ============================
Usually you open a root-bash using ``sudo -H bash``. In case of LXC containers Usually you open a root-bash using ``sudo -H bash``. In case of LXC containers
open the root-bash in the container using ``./utils/lxc.sh cmd open the root-bash in the container is done by the ``./utils/lxc.sh cmd
searxng-archlinux``: searxng-archlinux`` command:
.. tabs:: .. tabs::
.. group-tab:: desktop .. group-tab:: desktop (HOST)
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
INFO: [searxng-archlinux] bash INFO: [searxng-archlinux] bash
[root@searxng-archlinux searx]# pwd [root@searxng-archlinux SearXNG]$
/share/searxng
The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user in The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user
the searxng-container. To debug the running SearXNG instance use: in the container (GUEST). To debug the running SearXNG instance use:
.. tabs:: .. tabs::
.. group-tab:: root@searxng-archlinux .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
.. code:: bash .. code:: bash
$ ./utils/searx.sh inspect service $ ./utils/searxng.sh instance inspect
... ...
use [CTRL-C] to stop monitoring the log use [CTRL-C] to stop monitoring the log
... ...
Back in the browser on your desktop open the service http://10.174.184.156/searx .. group-tab:: desktop (HOST)
.. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance inspect
...
use [CTRL-C] to stop monitoring the log
...
Back in the browser on your desktop open the service http://n.n.n.140/searxng
and run your application tests while the debug log is shown in the terminal from and run your application tests while the debug log is shown in the terminal from
above. You can stop monitoring using ``CTRL-C``, this also disables the *"debug above. You can stop monitoring using ``CTRL-C``, this also disables the *"debug
option"* in SearXNG's settings file and restarts the SearXNG uwsgi application. option"* in SearXNG's settings file and restarts the SearXNG uwsgi application.
To debug services from filtron and morty analogous use:
Another point we have to notice is that the service (:ref:`SearXNG <searxng.sh>` Another point we have to notice is that the service :ref:`SearXNG <searxng.sh>`
runs under dedicated system user account with the same name (compare runs under dedicated system user account with the same name (compare
:ref:`create searxng user`). To get a shell from these accounts, simply call: :ref:`create searxng user`). To get a login shell from these accounts, simply
call:
.. tabs:: .. tabs::
.. group-tab:: root@searxng-archlinux .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
.. code:: bash .. code:: bash
$ ./utils/searxng.sh instance cmd bash $ ./utils/searxng.sh instance cmd bash -l
(searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
/usr/local/searxng
To get in touch, open a shell from the service user (searxng@searxng-archlinux): .. group-tab:: desktop (HOST)
.. tabs::
.. group-tab:: desktop
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash -l
INFO: [searxng-archlinux] ./utils/searxng.sh instance cmd bash INFO: [searxng-archlinux] ./utils/searxng.sh instance cmd bash -l
[searxng@searxng-archlinux ~]$ (searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
/usr/local/searxng
The prompt ``[searxng@searxng-archlinux]`` signals that you are logged in as system The prompt ``[searxng@searxng-archlinux]`` signals that you are logged in as system
user ``searx`` in the ``searxng-archlinux`` container and the python *virtualenv* user ``searxng`` in the ``searxng-archlinux`` container and the python *virtualenv*
``(searxng-pyenv)`` environment is activated. ``(searxng-pyenv)`` environment is activated.
.. tabs::
.. group-tab:: searxng@searxng-archlinux
.. code:: bash
(searxng-pyenv) [searxng@searxng-archlinux ~]$ pwd
/usr/local/searxng
Wrap production into developer suite Wrap production into developer suite
==================================== ====================================
@ -256,12 +246,11 @@ With the use of the :ref:`searxng.sh` the SearXNG service was installed as
.. tabs:: .. tabs::
.. group-tab:: desktop .. group-tab:: uwsgi@searxng
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl stop uwsgi@searxng
systemctl stop uwsgi@searxng
With the command above, we stopped the SearXNG uWSGI-App in the archlinux With the command above, we stopped the SearXNG uWSGI-App in the archlinux
container. container.
@ -278,17 +267,25 @@ least you should attend the settings of ``uid``, ``chdir``, ``env`` and
virtualenv = /usr/local/searxng/searxng-pyenv virtualenv = /usr/local/searxng/searxng-pyenv
pythonpath = /usr/local/searxng/searxng-src pythonpath = /usr/local/searxng/searxng-src
If you have read the :ref:`"Good to know section" <lxc.sh>` you remember, that If you have read the :ref:`Good to know` you remember, that each container
each container shares the root folder of the repository and the command shares the root folder of the repository and the command ``utils/lxc.sh cmd``
``utils/lxc.sh cmd`` handles relative path names **transparent**. To wrap the handles relative path names **transparent**.
SearXNG installation into a developer one, we simple have to create a smylink to
the **transparent** reposetory from the desktop. Now lets replace the To wrap the SearXNG installation in the container into a developer one, we
repository at ``searxng-src`` in the container with the working tree from outside simple have to create a smylink to the **transparent** reposetory from the
of the container: desktop. Now lets replace the repository at ``searxng-src`` in the container
with the working tree from outside of the container:
.. tabs:: .. tabs::
.. group-tab:: container becomes a developer suite .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
.. code:: bash
$ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
$ ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
.. group-tab:: desktop (HOST)
.. code:: bash .. code:: bash
@ -296,7 +293,7 @@ of the container:
mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
ln -s /share/searx/ /usr/local/searxng/searxng-src ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
Now we can develop as usual in the working tree of our desktop system. Every Now we can develop as usual in the working tree of our desktop system. Every
time the software was changed, you have to restart the SearXNG service (in the time the software was changed, you have to restart the SearXNG service (in the
@ -304,48 +301,83 @@ container):
.. tabs:: .. tabs::
.. group-tab:: desktop .. group-tab:: uwsgi@searxng
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
systemctl restart uwsgi@searx
Remember: :ref:`working in containers` .. here are just some examples from my Remember: :ref:`working in containers` .. here are just some examples from my
daily usage: daily usage:
To *inspect* the SearXNG instance (already described above):
.. tabs:: .. tabs::
.. group-tab:: desktop .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
To *inspect* the SearXNG instance (already described above):
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ $ ./utils/searx.sh inspect service
./utils/searx.sh inspect service
.. group-tab:: desktop (HOST)
.. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searx.sh inspect service
Run :ref:`makefile`, e.g. to test inside the container: Run :ref:`makefile`, e.g. to test inside the container:
.. tabs::
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ $ make test
make test
.. group-tab:: desktop (HOST)
.. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux make test
To install all prerequisites needed for a :ref:`buildhosts`: To install all prerequisites needed for a :ref:`buildhosts`:
.. tabs::
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ $ ./utils/searxng.sh install buildhost
./utils/searxng.sh install buildhost
.. group-tab:: desktop (HOST)
.. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost
To build the docs on a buildhost :ref:`buildhosts`: To build the docs on a buildhost :ref:`buildhosts`:
.. tabs::
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ $ make docs.html
make docs.html
.. group-tab:: desktop (HOST)
.. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.html
.. _lxcdev summary: .. _lxcdev summary:
@ -356,48 +388,51 @@ We build up a fully functional SearXNG suite in a archlinux container:
.. code:: bash .. code:: bash
$ sudo -H ./utils/lxc.sh build searxng-archlinux
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux $ sudo -H ./utils/lxc.sh install suite searxng-archlinux
...
Developer install? (wraps source from HOST into the running instance) [YES/no]
To wrap the suite into a developer one answer ``YES`` (or press Enter).
.. code:: text
link SearXNG's sources to: /share/SearXNG
=========================================
mv -f "/usr/local/searxng/searxng-src" "/usr/local/searxng/searxng-src.backup"
ln -s "/share/SearXNG" "/usr/local/searxng/searxng-src"
ls -ld /usr/local/searxng/searxng-src
|searxng| lrwxrwxrwx 1 searxng searxng ... /usr/local/searxng/searxng-src -> /share/SearXNG
On code modification the instance has to be restarted (see :ref:`uWSGI
maintenance`):
.. code:: bash
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
To access HTTP from the desktop we installed nginx for the services inside the To access HTTP from the desktop we installed nginx for the services inside the
container: container:
.. tabs::
.. group-tab:: [root@searxng-archlinux]
.. code:: bash .. code:: bash
$ ./utils/filtron.sh nginx install $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
$ ./utils/morty.sh nginx install
To wrap the suite into a developer one, we created a symbolic link to the To get information about the SearxNG suite in the archlinux container we can
repository which is shared **transparent** from the desktop's file system into
the container :
.. tabs::
.. group-tab:: [root@searxng-archlinux]
.. code:: bash
$ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
$ ln -s /share/searx/ /usr/local/searxng/searxng-src
$ systemctl restart uwsgi@searx
To get information about the searxNG suite in the archlinux container we can
use: use:
.. tabs:: .. code:: text
.. group-tab:: desktop
.. code:: bash
$ sudo -H ./utils/lxc.sh show suite searxng-archlinux $ sudo -H ./utils/lxc.sh show suite searxng-archlinux
... [searxng-archlinux] INFO: (eth0) docs-live: http:///n.n.n.140:8080/
[searxng-archlinux] INFO: (eth0) filtron: http://10.174.184.156:4004/ http://10.174.184.156/searx [searxng-archlinux] INFO: (eth0) IPv6: http://[fd42:555b:2af9:e121:216:3eff:fe5b:1744]
[searxng-archlinux] INFO: (eth0) morty: http://10.174.184.156:3000/ [searxng-archlinux] uWSGI:
[searxng-archlinux] INFO: (eth0) docs.live: http://10.174.184.156:8080/ [searxng-archlinux] SEARXNG_UWSGI_SOCKET : /usr/local/searxng/run/socket
[searxng-archlinux] INFO: (eth0) IPv6: http://[fd42:573b:e0b3:e97e:216:3eff:fea5:9b65] [searxng-archlinux] environment /usr/local/searxng/searxng-src/utils/brand.env:
... [searxng-archlinux] GIT_URL : https://github.com/searxng/searxng
[searxng-archlinux] GIT_BRANCH : master
[searxng-archlinux] SEARXNG_URL : http:///n.n.n.140/searxng
[searxng-archlinux] SEARXNG_PORT : 8888
[searxng-archlinux] SEARXNG_BIND_ADDRESS : 127.0.0.1

2
docs/utils/index.rst
View file

@ -20,6 +20,8 @@ Common command environments
The scripts in our tooling box often dispose of common environments: The scripts in our tooling box often dispose of common environments:
.. _FORCE_TIMEOUT:
``FORCE_TIMEOUT`` : environment ``FORCE_TIMEOUT`` : environment
Sets timeout for interactive prompts. If you want to run a script in batch Sets timeout for interactive prompts. If you want to run a script in batch
job, with defaults choices, set ``FORCE_TIMEOUT=0``. By example; to install a job, with defaults choices, set ``FORCE_TIMEOUT=0``. By example; to install a

215
docs/utils/lxc.sh.rst
View file

@ -14,6 +14,11 @@
``utils/lxc.sh`` ``utils/lxc.sh``
================ ================
With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of
containers, what we call the: *lxc suite*. The :ref:`lxc-searxng.env` is
loaded by default, every time you start the ``lxc.sh`` script (*you do not need
to care about*).
.. sidebar:: further reading .. sidebar:: further reading
- snap_, `snapcraft LXD`_ - snap_, `snapcraft LXD`_
@ -21,10 +26,16 @@
- `LXC/LXD Image Server`_ - `LXC/LXD Image Server`_
- `LXD@github`_ - `LXD@github`_
With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of .. contents:: Contents
containers, what we call the: *lxc suite*. The *SearXNG suite* :depth: 2
(:origin:`lxc-searxng.env <utils/lxc-searxng.env>`) is loaded by default, every time :local:
you start the ``lxc.sh`` script (*you do not need to care about*). :backlinks: entry
.. _lxd install:
Install LXD
===========
Before you can start with containers, you need to install and initiate LXD_ Before you can start with containers, you need to install and initiate LXD_
once:: once::
@ -38,44 +49,19 @@ take some time**::
$ sudo -H ./utils/lxc.sh build $ sudo -H ./utils/lxc.sh build
.. sidebar:: hint
If you have issues with the internet connectivity of your containers read
section :ref:`internet connectivity docker`.
A cup of coffee later, your LXC suite is build up and you can run whatever task A cup of coffee later, your LXC suite is build up and you can run whatever task
you want / in a selected or even in all :ref:`LXC suite containers <lxc.sh you want / in a selected or even in all :ref:`LXC suite containers <lxc.sh
help>`. help>`.
.. hint::
If you see any problems with the internet connectivity of your
containers read section :ref:`internet connectivity docker`.
If you do not want to build all containers, **you can build just one**::
$ sudo -H ./utils/lxc.sh build searxng-archlinux
*Good to know ...*
Each container shares the root folder of the repository and the command
``utils/lxc.sh cmd`` **handles relative path names transparent**, compare output
of::
$ sudo -H ./utils/lxc.sh cmd -- ls -la Makefile
...
In the containers, you can run what ever you want, e.g. to start a bash use::
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
INFO: [searxng-archlinux] bash
[root@searxng-archlinux SearXNG]#
If there comes the time you want to **get rid off all** the containers and
**clean up local images** just type::
$ sudo -H ./utils/lxc.sh remove
$ sudo -H ./utils/lxc.sh remove images
.. _internet connectivity docker: .. _internet connectivity docker:
Internet Connectivity & Docker Internet Connectivity & Docker
============================== ------------------------------
.. sidebar:: further read .. sidebar:: further read
@ -115,21 +101,132 @@ Reboot your system and check the iptables rules::
:FORWARD ACCEPT [7048:7851230] :FORWARD ACCEPT [7048:7851230]
.. _searxng lxc suite:
SearXNG LXC suite
=================
The intention of the *SearXNG LXC suite* is to build up a suite of containers
for development tasks or :ref:`buildhosts <Setup SearXNG buildhost>` with a very
small set of simple commands. At the end of the ``--help`` output the SearXNG
suite from the :ref:`lxc-searxng.env` is introduced::
$ sudo -H ./utils/lxc.sh --help
...
LXC suite: searxng
Suite includes installation of SearXNG
images: ubu2004 ubu2204 fedora35 archlinux
containers: searxng-ubu2004 searxng-ubu2204 searxng-fedora35 searxng-archlinux
As shown above there are images and containers build up on this images. To show
more info about the containers in the *SearXNG LXC suite* call ``show suite``.
If this is the first time you make use of the SearXNG LXC suite, no containers
are installed and the output is::
$ sudo -H ./utils/lxc.sh show suite
LXC suite (searxng-*)
=====================
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+
WARN: container searxng-ubu2004 does not yet exists
WARN: container searxng-ubu2204 does not yet exists
WARN: container searxng-fedora35 does not yet exists
WARN: container searxng-archlinux does not yet exists
If you do not want to run a command or a build in all containers, **you can
build just one**. Here by example in the container that is build upon the
*archlinux* image::
$ sudo -H ./utils/lxc.sh build searxng-archlinux
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd
Otherwise, to apply a command to all containers you can use::
$ sudo -H ./utils/lxc.sh build
$ sudo -H ./utils/lxc.sh cmd -- ls -la .
Running commands
----------------
**Inside containers, you can run scripts** from the :ref:`toolboxing` or run
what ever command you need. By example, to start a bash use::
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
INFO: [searxng-archlinux] bash
[root@searxng-archlinux SearXNG]#
.. _Good to know:
Good to know
------------
Each container shares the root folder of the repository and the command
``utils/lxc.sh cmd`` **handle relative path names transparent**::
$ pwd
/share/SearXNG
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd
INFO: [searxng-archlinux] pwd
/share/SearXNG
The path ``/share/SearXNG`` will be different on your HOST system. The commands
in the conatiner are executed by the ``root`` inside of the container. Compare
output of::
$ ls -li Makefile
47712402 -rw-rw-r-- 1 markus markus 2923 Apr 19 13:52 Makefile
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ls -li Makefile
INFO: [searxng-archlinux] ls -li Makefile
47712402 -rw-rw-r-- 1 root root 2923 Apr 19 11:52 Makefile
...
Since the path ``/share/SearXNG`` of the HOST system is wrapped into the
container under the same name, the shown ``Makefile`` (inode ``47712402``) in
the ouput is always the identical ``/share/SearXNG/Makefile`` from the HOST
system. In the example shown above the owner of the path in the container is
the ``root`` user of the conatiner (and the timezone in the container is
different to HOST system).
.. _lxc.sh install suite: .. _lxc.sh install suite:
Install suite Install suite
============= -------------
To install the complete :ref:`SearXNG suite (includes searx, morty & filtron) .. sidebar:: further read
<lxc-searxng.env>` into all LXC_ use::
- :ref:`working in containers`
- :ref:`FORCE_TIMEOUT <FORCE_TIMEOUT>`
To install the complete :ref:`SearXNG suite <lxc-searxng.env>` into **all** LXC_
containers leave the container argument empty and run::
$ sudo -H ./utils/lxc.sh build
$ sudo -H ./utils/lxc.sh install suite $ sudo -H ./utils/lxc.sh install suite
To *build & install* suite only in one container you can use by example::
$ sudo -H ./utils/lxc.sh build searxng-archlinux
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux
The command above installs a SearXNG suite (see :ref:`installation scripts`). The command above installs a SearXNG suite (see :ref:`installation scripts`).
To :ref:`install a nginx <installation nginx>` reverse proxy (or alternatively To :ref:`install a nginx <installation nginx>` reverse proxy (or alternatively
use :ref:`apache <installation apache>`):: use :ref:`apache <installation apache>`)::
sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
Same operation just in one container of the suite::
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
The :ref:`FORCE_TIMEOUT <FORCE_TIMEOUT>` environment is set to zero to run the
script without user interaction.
To get the IP (URL) of the SearXNG service in the containers use ``show suite`` To get the IP (URL) of the SearXNG service in the containers use ``show suite``
command. To test instances from containers just open the URLs in your command. To test instances from containers just open the URLs in your
@ -137,22 +234,22 @@ WEB-Browser::
$ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
[searxng-ubu2110] SEARXNG_URL : http://n.n.n.147/searxng [searxng-ubu2110] SEARXNG_URL : http://n.n.n.170/searxng
[searxng-ubu2004] SEARXNG_URL : http://n.n.n.246/searxng [searxng-ubu2004] SEARXNG_URL : http://n.n.n.160/searxng
[searxnggfedora35] SEARXNG_URL : http://n.n.n.140/searxng [searxnggfedora35] SEARXNG_URL : http://n.n.n.150/searxng
[searxng-archlinux] SEARXNG_URL : http://n.n.n.165/searxng [searxng-archlinux] SEARXNG_URL : http://n.n.n.140/searxng
Clean up
--------
If there comes the time you want to **get rid off all** the containers and
**clean up local images** just type::
$ sudo -H ./utils/lxc.sh remove
$ sudo -H ./utils/lxc.sh remove images
Running commands .. _Setup SearXNG buildhost:
================
**Inside containers, you can use make or run scripts** from the
:ref:`toolboxing`. By example: to setup a :ref:`buildhosts` and run the
Makefile target ``test`` in the archlinux_ container::
sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost
sudo -H ./utils/lxc.sh cmd searxng-archlinux make test
Setup SearXNG buildhost Setup SearXNG buildhost
======================= =======================
@ -174,13 +271,13 @@ To get IP of the container and the port number *live docs* is listening::
$ sudo ./utils/lxc.sh show suite | grep docs.live $ sudo ./utils/lxc.sh show suite | grep docs.live
... ...
[searxng-archlinux] INFO: (eth0) docs.live: http://n.n.n.12:8080/ [searxng-archlinux] INFO: (eth0) docs.live: http://n.n.n.140:8080/
.. _lxc.sh help: .. _lxc.sh help:
Overview Command Help
======== ============
The ``--help`` output of the script is largely self-explanatory: The ``--help`` output of the script is largely self-explanatory:
@ -189,8 +286,10 @@ The ``--help`` output of the script is largely self-explanatory:
.. _lxc-searxng.env: .. _lxc-searxng.env:
SearXNG suite SearXNG suite config
============= ====================
The SearXNG suite is defined in the file :origin:`utils/lxc-searxng.env`:
.. literalinclude:: ../../utils/lxc-searxng.env .. literalinclude:: ../../utils/lxc-searxng.env
:language: bash :language: bash

14
docs/utils/searxng.sh.rst
View file

@ -5,6 +5,9 @@
``utils/searxng.sh`` ``utils/searxng.sh``
==================== ====================
To simplify the installation and maintenance of a SearXNG instance you can use the
script :origin:`utils/searxng.sh`.
.. sidebar:: further reading .. sidebar:: further reading
- :ref:`architecture` - :ref:`architecture`
@ -12,8 +15,11 @@
- :ref:`installation nginx` - :ref:`installation nginx`
- :ref:`installation apache` - :ref:`installation apache`
To simplify the installation and maintenance of a SearXNG instance you can use the .. contents:: Contents
script :origin:`utils/searxng.sh`. :depth: 2
:local:
:backlinks: entry
Install Install
======= =======
@ -28,8 +34,8 @@ The installation is described in chapter :ref:`installation basic`.
.. _searxng.sh overview: .. _searxng.sh overview:
Overview Command Help
======== ============
The ``--help`` output of the script is largely self-explanatory: The ``--help`` output of the script is largely self-explanatory:

9
utils/lxc-searxng.env
View file

@ -42,10 +42,11 @@ EOF
lxc_suite_install() { lxc_suite_install() {
( (
lxc_set_suite_env lxc_set_suite_env
FORCE_TIMEOUT=0 FORCE_TIMEOUT=0 "${LXC_REPO_ROOT}/utils/searxng.sh" install all
export FORCE_TIMEOUT rst_title "Suite installation finished ($(hostname))" part
"${LXC_REPO_ROOT}/utils/searxng.sh" install all if ask_yn "Developer install? (wraps source from HOST into the running instance)" Yn; then
rst_title "suite installation finished ($(hostname))" part "${LXC_REPO_ROOT}/utils/searxng.sh" searxng.install.link_src "$(pwd)"
fi
lxc_suite_info lxc_suite_info
echo echo
) )

12
utils/searxng.sh
View file

@ -467,6 +467,18 @@ EOF
popd > /dev/null popd > /dev/null
} }
searxng.install.link_src() {
rst_title "link SearXNG's sources to: $2" chapter
echo
tee_stderr 0.1 <<EOF | sudo -H -u "${SERVICE_USER}" -i 2>&1 | prefix_stdout "$_service_prefix"
mv -f "${SEARXNG_SRC}" "${SEARXNG_SRC}.backup"
ln -s "${2}" "${SEARXNG_SRC}"
ls -ld /usr/local/searxng/searxng-src
EOF
echo
uWSGI_restart "$SEARXNG_UWSGI_APP"
}
searxng.install.pyenv() { searxng.install.pyenv() {
rst_title "Create virtualenv (python)" section rst_title "Create virtualenv (python)" section
echo echo