Difference between revisions of "Kobo Docker Setup"

Step by step how-to instructions on installing and running KoBo Toolbox on a standalone server.
Jump to: navigation, search
m
(Tag: visualeditor-switched)
Line 1: Line 1:
−
[https://github.com/kobotoolbox/kobo-docker|Kobo Docker setup at Github]
+
=== Decide upon the location ===
−
<pre>
 
−
curl -L https://github.com/docker/compose/releases/download/1.7.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
 
−
chmod +x /usr/local/bin/docker-compose
 
−
visudo
 
−
./wait_for_postgres.bash
 
−
docker-compose up -d
 
−
</pre>
 
  
−
<pre>
+
$ cd /home/ate
−
sudo apt-get update
+
$ git clone https://github.com/kobotoolbox/kobo-docker.git
−
sudo apt-get install git
 
−
sudo apt-get remove --purge resolvconf && sudo apt-get install --reinstall resolvconf
 
−
sudo apt-get install --reinstall resolvconf
 
−
sudo ethtool eth0
 
−
vi /etc/resolvconf/resolv.conf.d/base
 
−
sudo apt-get install --reinstall resolvconf
 
−
sudo vi /etc/resolv.conf
 
−
git clone https://github.com/kobotoolbox/kobo-docker.git
 
−
cd kobo-docker/
 
−
ln -s docker-compose.server.yml docker-compose.yml
 
−
sudo docker-compose pull
 
−
sudo apt-get install docker-engine
 
−
uname -r
 
−
sudo apt-get install apt-transport-https ca-certificates
 
−
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
 
−
sudo vi /etc/apt/sources.list.d/docker.list
 
−
sudo apt-get purge lxc-docker
 
−
sudo apt-get install linux-image-extra-$(uname -r)
 
−
apt-get install apparmor
 
−
sudo apt-get install apparmor
 
−
sudo apt-get install docker-engine
 
−
sudo service docker start
 
−
sudo docker run hello-world
 
−
docker-compose pull
 
−
sudo apt-get install docker-compose
 
−
sudo curl -L https://github.com/docker/compose/releases/download/1.7.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
 
−
docker-compose --version
 
−
sudo docker-compose pull
 
−
docker-compose build
 
−
vi envfile.server.txt
 
−
mkdir secrets
 
−
cd secrets/
 
−
vi ssl.crt
 
−
vi ssl.key
 
−
sudo docker-compose stop; sudo docker-compose rm
 
−
sudo rm -rf .vols/ log/.
 
−
sudo docker-compose up -d
 
−
sudo docker ps
 
−
cd kobo-docker/
 
−
vi docker-compose.yml
 
−
vi envfile.server.txt
 
−
less envfile.server.txt
 
−
sudo docker-compose up -d
 
−
sudo docker ps
 
−
sudo docker-compose down
 
−
sudo docker-compose up -d
 
−
sudo docker ps
 
−
sudo docker-compose stop nginx && sudo docker-compose up -d && sudo docker-compose logs nginx
 
  
−
cd kobo-docker/
+
The working tree repository will be created in the directory kobo.docker, i.e. /home/ate/kobo-docker. If required, the folder can be renamed:
−
git fetch
 
−
git stash
 
−
git pull
 
−
git stash pop
 
−
vim envfile.server.txt
 
−
docker-compose pull
 
−
cat /etc/group
 
−
cd kobo-docker/
 
−
docker-compose pull
 
−
docker exec -it kobodocker_nginx_1 bash
 
−
tail -f log/nginx/access.log
 
−
tail -f log/nginx/kobocat.access.log
 
−
tail -f log/nginx/kpi.access.log
 
−
tail -f log/nginx/kpi.error.log
 
−
less log/nginx/kobocat.access.log
 
−
su esmail -c 'sudo du -h -d 1 .vols'
 
−
docker-compose logs -f
 
−
docker-compose up -d && docker-compose logs -f
 
−
sudo ls backups/postgres/
 
−
git fetch
 
−
git stash
 
−
git status
 
−
git add envfile.server.txt
 
−
git status
 
−
git reset HEAD envfile.server.txt
 
−
docker-compose stop
 
−
vim base_images/postgres/init_postgres.bash
 
−
docker ps -a
 
−
mkdir postgres
 
−
cd postgres
 
−
docker export kobodocker_postgres_1 | tar x
 
−
cd ..
 
−
rm -rf postgres/
 
−
mkdir postgres
 
−
docker export kobodocker_postgres_1 | tar x /srv
 
−
cd postgres
 
−
docker export kobodocker_postgres_1 | tar x /srv
 
−
docker export kobodocker_postgres_1 | tar x srv
 
−
cd srv/
 
−
du -h -d 1
 
−
docker export kobodocker_postgres_1 | tar x srv
 
−
du -h -d 1
 
−
mkdir postgres && cd postgres
 
−
docker export kobodocker_postgres_1 | tar x srv
 
−
vim base_images/postgres/init_postgres.bash
 
−
docker-compose up -d && docker-compose logs -f
 
−
docker-compose stop kpi
 
−
docker-compose rm -fv kpi
 
−
docker-compose up -d && docker-compose logs -f
 
−
cd base_images/
 
−
cd base
 
−
docker build -t kobotoolbox/base:latest .
 
−
cd ../base-kobos/
 
−
docker build -t kobotoolbox/base-koobos:latest .
 
−
#docker build -t kobotoolbox/base-kobos:latest .
 
−
git pull
 
−
git stash
 
−
git pull
 
−
git stash pop
 
−
git status
 
−
git reset HEAD ../postgres/init_postgres.bash
 
−
git status
 
−
git checkout -- ../postgres/init_postgres.bash
 
−
git status
 
−
git merge --continue
 
−
git status
 
−
git reset HEAD ../../envfile.server.txt
 
−
git status
 
−
docker build -t kobotoolbox/base-kobos:latest .
 
−
docker-compose pull
 
−
docker pull koboform_base
 
−
docker pull kobotoolbox/koboform_base
 
−
docker-compose pull
 
−
docker-compose up -d && docker-compose logs
 
−
docker-compose up -d && docker-compose logs -f
 
−
cd kobo-docker/
 
−
docker-compose pull
 
−
docker-compose up -d && docker-compose logs -f
 
−
docker exec -it kobodocker_kobocat_1 rm -rf /etc/service/wsgi
 
−
docker-compose logs -f kobocat
 
−
docker-compose restart kobocat && docker-compose logs -f kobocat
 
−
sudo top
 
−
docker-compose logs -f
 
−
cd kobo-docker/
 
−
vim docker-compose.yml
 
−
vim envfiles/smtp.txt
 
−
git stash -l
 
−
git stash lsit
 
−
git stash list
 
  
−
git stash list
+
$ mv kobo-docker new-name
−
git stash show -p stash@{1}
 
−
vim envfiles/aws.txt
 
−
vim envfiles/smtp.txt
 
−
vim envfile.server.txt
 
−
vim docker-compose.yml
 
−
ifconfig | grep docker
 
−
vim docker-compose.yml
 
−
git diff
 
−
docker-compose up -d && docker exec -it kobodocker_kobocat_1 rm -rf /etc/service/wsgi && docker-compose logs -f
 
−
docker-compose up -d --remove-orphans && docker exec -it kobodocker_kobocat_1 rm -rf /etc/service/wsgi && docker-compose logs -f
 
−
docker-compose up -d && docker exec -it kobodocker_kobocat_1 rm -rf /etc/service/wsgi && docker-compose logs -f
 
−
docker-compose logs -f
 
−
docker ps
 
−
docker rm 971d080c505b
 
−
docker rm 53ed0ff8e8fc
 
−
docker exec -it 53ed0ff8e8fc bash
 
−
docker rm -f 53ed0ff8e8fc
 
−
bg
 
−
vim envfile.server.txt
 
−
vim envfiles/smtp.txt
 
−
vim envfiles/aws.txt
 
−
cd kobo-docker/
 
−
docker-compose up
 
−
ssh ate@192.168.50.118
 
−
telnet 192.168.50.119 22
 
−
ssh 192.168.50.118
 
−
cd kobo-docker/
 
−
ssh root@192.168.50.118
 
−
cd kobo-docker/
 
−
more envfile.server.txt
 
−
more envfile.local.txt
 
−
cd envfiles
 
−
cat kobocat.txt
 
−
cd kobo-docker/
 
−
cd postgres/
 
−
cd srv
 
−
cd kobo-docker/scripts/
 
−
./wait_for_kpi.bash
 
−
./wait_for_postgres.bash
 
−
sudo service docker status
 
−
sudo service docker restart
 
−
sudo ./wait_for_postgres.bash
 
−
docker logs b98aa6bfaa6b
 
−
</pre>
 
  
−
<pre>
+
...but we will leave it at /home/ate/kobo-docker/. Systemwide variable:
−
git clone https://github.com/kobotoolbox/kpi.git
 
−
docker pull kobotoolbox/koboform_base:latest
 
−
cd kpi
 
−
git checkout origin/master
 
−
docker build .
 
−
docker build -t kobotoolbox/kpi:latest .
 
−
sudo chmod --reference .vols/db postgres/srv/db_9.3
 
−
sudo chown --reference .vols/db postgres/srv/db_9.3
 
−
docker-compose run --rm postgres
 
−
docker-compose run --rm postgres bash
 
−
sudo mv postgres/srv/db_9.3 .vols/db
 
−
docker-compose run --rm postgres
 
−
sudo rm -rf postgres/
 
−
vim base_images/postgres/init_postgres.bash
 
−
sudo chown --reference .vols/db postgres/srv/db_9.3
 
−
sudo chmod --reference .vols/db postgres/srv/db_9.3
 
−
sudo rm -rf .vols/db
 
−
sudo mv postgres/srv/db_9.3 .vols/db
 
−
docker-compose run --rm postgres
 
−
docker-compose run --rm postgres bash
 
−
sudo -c 'rm -rf postgres && mkdir postgres && cd postgres && docker export kobodocker_postgres_1 | tar x srv/db_9.3 && chmod --reference ../.vols/db srv/db_9.3 && chown --reference ../.vols/db srv/db_9.3'
 
−
sudo bash -c 'rm -rf postgres && mkdir postgres && cd postgres && docker export kobodocker_postgres_1 | tar x srv/db_9.3 && chmod --reference ../.vols/db srv/db_9.3 && chown --reference ../.vols/db srv/db_9.3'
 
−
sudo bash -c 'rm -rf postgres && mkdir postgres && cd postgres && docker export kobodocker_postgres_1 | tar x srv/db_9.3 && chmod --reference ../.vols/db srv/db_9.3 && chown --reference ../.vols/db srv/db_9.3 && rm -rf ../.vols/db && mv srv/db_9.3 ../.vols/db'
 
  
−
docker-compose run --rm postgres
+
$ sudo nano /etc/profile.d/kobo-docker.sh
−
docker-compose run --rm postgres bash
+
<blockquote><pre>export KOBO_DOCKER="/home/ate/kobo-docker"</pre></blockquote>
−
docker-compose up -d && docker-compose logs
+
 
−
docker-compose stop
+
This will define the system-wide (environment) variable, available to all users.
−
docker-compose run --rm postgres bash
+
 
−
docker-compose rm -fv postgres
+
$ cd $KOBO_DOCKER && pwd
−
docker-compose run --rm postgres
+
<blockquote><pre>/home/ate/kobo-docker</pre></blockquote>
−
docker-compose up -d && docker-compose logs -f
+
 
−
docker-compose stop
+
{{KOBO_DOCKER}}
−
git clone https://github.com/kobotoolbox/kobocat.git
+
 
−
cd kobocat
+
=== Configure KoBo Docker ===
−
docker build -f Dockerfile.kobocat_base .
+
$ ln -s docker-compose.server.yml docker-compose.yml
−
passwd
+
$ sudo docker-compose pull
−
sudo pstree
+
$ docker-compose build
−
byobu
+
$ nano envfile.server.txt
−
sudo ifconfig docker0
+
<blockquote><pre># The publicly-accessible domain where your KoBo Toolbox instance will be reached.
−
byobu
+
PUBLIC_DOMAIN_NAME=msf.org
−
</pre>
+
# The publicly-accessible subdomain for the KoBoForm form building and management interface.
 +
KOBOFORM_PUBLIC_SUBDOMAIN=kobo
 +
# The publicly-accessible subdomain for the KoBoCAT data collection and project management interface.
 +
KOBOCAT_PUBLIC_SUBDOMAIN=kc
 +
# The publicly-accessible subdomain for the Enketo Express web forms.
 +
ENKETO_EXPRESS_PUBLIC_SUBDOMAIN=ee
 +
# See "api key" here: https://github.com/kobotoolbox/enketo-express/tree/master/config#linked-form-and-data-server.
 +
ENKETO_API_TOKEN=fAvPevhvWb1w79XQxSabMKN9mW
 +
# Canonically a 50-character random string. See https://docs.djangoproject.com/en/1.8/ref/settings/#secret-key and https://docs.djangoproject.com/en/1.8/ref/settings/#secret-key.
 +
DJANGO_SECRET_KEY=QE7Vwf7uqYJ8LgLaEZLC9f3AebTCX8DPukue46tYAU7ZyVTbXX
 +
# The initial superuser's username.
 +
KOBO_SUPERUSER_USERNAME=msfuser
 +
# The initial superuser's password.
 +
KOBO_SUPERUSER_PASSWORD=msfOCB1971
 +
 
 +
######################
 +
# Optional variables #
 +
######################
 +
 
 +
# For help customizing backup schedules, use a `cron` schedule generator (e.g.
 +
#  crontab.guru). To enable a backup, uncomment the relevant schedule.
 +
# Default KoBoCAT media backup schedule is weekly at 12:00 AM UTC on Sunday.
 +
KOBOCAT_MEDIA_BACKUP_SCHEDULE=59 2 * * *
 +
# Default MongoDB backup schedule is weekly at 01:00 AM UTC on Sunday.
 +
MONGO_BACKUP_SCHEDULE=57 2 * * *
 +
# Default Postgres backup schedule is weekly at 02:00 AM UTC on Sunday.
 +
POSTGRES_BACKUP_SCHEDULE=52 2 * * *</pre></blockquote>
 +
 
 +
$ sudo nano envfiles/smtp.txt
 +
 
 +
<blockquote><pre>EMAIL_BACKEND=django.core.mail.backends.smtp.EmailBackend
 +
EMAIL_HOST=192.168.60.4
 +
EMAIL_PORT=25
 +
#EMAIL_HOST_USER=
 +
#EMAIL_HOST_PASSWORD=
 +
#EMAIL_USE_TLS=
 +
#DEFAULT_FROM_EMAIL=</pre></blockquote>
 +
 
 +
=== Configure SSL Certificates ===
 +
 
 +
$ mkdir secrets && cd secrets/
 +
$ nano ssl.crt
 +
$ nano ssl.key
 +
 
 +
Please refer to the article [[SSL Setup]] for more information.
 +
 
 +
=== Start the system! ===
 +
 
 +
$ sudo docker-compose up -d
 +
$ sudo docker ps
 +
 
 +
<blockquote><pre>          Name                        Command            State                    Ports<
 +
------------------------------------------------------------------------------------------------------------
 +
kobodocker_nginx_1            /sbin/my_init                Up      0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
 +
kobodocker_enketo_express_1  /sbin/my_init                Up      8005/tcp
 +
kobodocker_kobocat_1          /sbin/my_init                Up      8000/tcp
 +
kobodocker_kpi_1              /sbin/my_init                Up      8000/tcp
 +
kobodocker_mongo_1            /sbin/my_init                Up      27017/tcp
 +
kobodocker_postgres_1        /sbin/my_init                Up      5432/tcp
 +
kobodocker_rabbit_1          /sbin/my_init                Up      5672/tcp
 +
kobodocker_redis_cache_1      /entrypoint.sh redis-server  Up      6379/tcp
 +
kobodocker_redis_main_1      /entrypoint.sh redis-server  Up      6379/tcp</pre></blockquote>
 +
 
 +
[[File:KoBo-Toolbox-Diagram.png|thumb|150x150px|left]] Are the names on the list familiar? They should be - it is our (living and breathing) instance of the system laid out in the article [[KoBo Toolbox Architecture]]. 
 +
 
 +
'''Well done!'''
 +
 
 +
Your KoBo Toolbox has now been set up, and you should be able to log into the system with the initial username and password, as provided in <code>envfile.server.txt</code>.
 +
 
 +
Should you require to stop KoBo Toolbox, issue the following command:
 +
$ sudo docker-compose down
 +
==== Make Kobo start with the OS ====
 +
To make the system start with OS, i.e. restart on every reboot, the following steps are necessary
 +
$ sudo nano /etc/systemd/system/kobo-toolbox.service
 +
 
 +
<blockquote><pre>[Unit]
 +
Description=Kobo Toolbox containers
 +
Requires=docker.service
 +
After=docker.service
 +
 
 +
[Service]
 +
Restart=always
 +
ExecStart=/usr/local/bin/docker-compose -f /home/ate/kobo-docker/docker-compose.server.yml up
 +
ExecStop=/usr/local/bin/docker-compose -f /home/ate/kobo-docker/docker-compose.server.yml stop
 +
 
 +
[Install]
 +
WantedBy=default.target</pre></blockquote>
 +
 
 +
$ sudo ln -s /etc/systemd/system/kobo-toolbox.service /etc/systemd/system/default.target.wants/kobo-toolbox.service
 +
$ sudo systemctl enable kobo-toolbox.service
 +
 
 +
...and we are ready for system reboot.
 +
 
 +
For additional information, please familiarise yourself woth [https://github.com/kobotoolbox/kobo-docker|Kobo Docker setup at Github]
 +
 
 +
Raw bash history can be found [[Kobo Docker Setup Raw BASH history|here]].

Revision as of 09:19, 21 December 2016

Decide upon the location

$ cd /home/ate
$ git clone https://github.com/kobotoolbox/kobo-docker.git

The working tree repository will be created in the directory kobo.docker, i.e. /home/ate/kobo-docker. If required, the folder can be renamed: 

$ mv kobo-docker new-name

...but we will leave it at /home/ate/kobo-docker/. Systemwide variable: 

$ sudo nano /etc/profile.d/kobo-docker.sh

export KOBO_DOCKER="/home/ate/kobo-docker"

This will define the system-wide (environment) variable, available to all users. 

$ cd $KOBO_DOCKER && pwd

/home/ate/kobo-docker
KOBO_DOCKER=/home/ate/kobo-docker/

Configure KoBo Docker

$ ln -s docker-compose.server.yml docker-compose.yml
$ sudo docker-compose pull
$ docker-compose build
$ nano envfile.server.txt

# The publicly-accessible domain where your KoBo Toolbox instance will be reached.
PUBLIC_DOMAIN_NAME=msf.org
# The publicly-accessible subdomain for the KoBoForm form building and management interface.
KOBOFORM_PUBLIC_SUBDOMAIN=kobo
# The publicly-accessible subdomain for the KoBoCAT data collection and project management interface.
KOBOCAT_PUBLIC_SUBDOMAIN=kc
# The publicly-accessible subdomain for the Enketo Express web forms.
ENKETO_EXPRESS_PUBLIC_SUBDOMAIN=ee
# See "api key" here: https://github.com/kobotoolbox/enketo-express/tree/master/config#linked-form-and-data-server.
ENKETO_API_TOKEN=fAvPevhvWb1w79XQxSabMKN9mW
# Canonically a 50-character random string. See https://docs.djangoproject.com/en/1.8/ref/settings/#secret-key and https://docs.djangoproject.com/en/1.8/ref/settings/#secret-key.
DJANGO_SECRET_KEY=QE7Vwf7uqYJ8LgLaEZLC9f3AebTCX8DPukue46tYAU7ZyVTbXX
# The initial superuser's username.
KOBO_SUPERUSER_USERNAME=msfuser
# The initial superuser's password.
KOBO_SUPERUSER_PASSWORD=msfOCB1971

######################
# Optional variables #
######################

# For help customizing backup schedules, use a `cron` schedule generator (e.g.
#   crontab.guru). To enable a backup, uncomment the relevant schedule.
# Default KoBoCAT media backup schedule is weekly at 12:00 AM UTC on Sunday.
KOBOCAT_MEDIA_BACKUP_SCHEDULE=59 2 * * *
# Default MongoDB backup schedule is weekly at 01:00 AM UTC on Sunday.
MONGO_BACKUP_SCHEDULE=57 2 * * *
# Default Postgres backup schedule is weekly at 02:00 AM UTC on Sunday.
POSTGRES_BACKUP_SCHEDULE=52 2 * * *
$ sudo nano envfiles/smtp.txt

EMAIL_BACKEND=django.core.mail.backends.smtp.EmailBackend
EMAIL_HOST=192.168.60.4
EMAIL_PORT=25
#EMAIL_HOST_USER=
#EMAIL_HOST_PASSWORD=
#EMAIL_USE_TLS=
#DEFAULT_FROM_EMAIL=

Configure SSL Certificates

$ mkdir secrets && cd secrets/
$ nano ssl.crt
$ nano ssl.key

Please refer to the article SSL Setup for more information.

Start the system!

$ sudo docker-compose up -d
$ sudo docker ps

           Name                         Command             State                    Ports<
------------------------------------------------------------------------------------------------------------
kobodocker_nginx_1            /sbin/my_init                 Up      0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
kobodocker_enketo_express_1   /sbin/my_init                 Up      8005/tcp
kobodocker_kobocat_1          /sbin/my_init                 Up      8000/tcp
kobodocker_kpi_1              /sbin/my_init                 Up      8000/tcp
kobodocker_mongo_1            /sbin/my_init                 Up      27017/tcp
kobodocker_postgres_1         /sbin/my_init                 Up      5432/tcp
kobodocker_rabbit_1           /sbin/my_init                 Up      5672/tcp
kobodocker_redis_cache_1      /entrypoint.sh redis-server   Up      6379/tcp
kobodocker_redis_main_1       /entrypoint.sh redis-server   Up      6379/tcp
KoBo-Toolbox-Diagram.png
Are the names on the list familiar? They should be - it is our (living and breathing) instance of the system laid out in the article KoBo Toolbox Architecture.

Well done!

Your KoBo Toolbox has now been set up, and you should be able to log into the system with the initial username and password, as provided in envfile.server.txt.

Should you require to stop KoBo Toolbox, issue the following command: 

$ sudo docker-compose down

Make Kobo start with the OS

To make the system start with OS, i.e. restart on every reboot, the following steps are necessary 

$ sudo nano /etc/systemd/system/kobo-toolbox.service

[Unit]
Description=Kobo Toolbox containers
Requires=docker.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/local/bin/docker-compose -f /home/ate/kobo-docker/docker-compose.server.yml up
ExecStop=/usr/local/bin/docker-compose -f /home/ate/kobo-docker/docker-compose.server.yml stop

[Install]
WantedBy=default.target
$ sudo ln -s /etc/systemd/system/kobo-toolbox.service /etc/systemd/system/default.target.wants/kobo-toolbox.service
$ sudo systemctl enable kobo-toolbox.service

...and we are ready for system reboot.

For additional information, please familiarise yourself woth Docker setup at Github

Raw bash history can be found here.

Retrieved from "https://www.kobowiki.org/index.php?title=Kobo_Docker_Setup&oldid=75"