Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WiP: Move mainpage to docker #7

Open
wants to merge 38 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
ee3ec17
Deleted old python requirements, updated only the absolutly necessery…
rasovica Jul 13, 2017
1b80713
Added docker file to git.
rasovica Jul 13, 2017
a5c5cf2
Added env variable to fix django.
rasovica Jul 13, 2017
90bddb6
Adding uwsgi service.
rasovica Jul 13, 2017
da0a2f5
Update readme.
rasovica Jul 13, 2017
a9d7aa0
Adding code dir, ngnix and uwsgi.
rasovica Jul 18, 2017
fa4cc69
Adding docker compose.
Jul 18, 2017
462e333
Adding docker compose.
Jul 20, 2017
c06404b
Trying to fix broken django dependencies.
rasovica Jul 26, 2017
80cef88
Temporarily adding nodewatcher v2 to the project before replacing it …
rasovica Jul 26, 2017
3797734
Adding the nodewatcher correctly.
rasovica Jul 26, 2017
7d6a8e1
Updating readme.
rasovica Jul 26, 2017
c8b1bda
Fixing requirements to be installed properly.
rasovica Jul 26, 2017
c7a9ce5
Added twitter plugin, changed the settings to production, made a secr…
rasovica Jul 29, 2017
c85e9a8
Adding secrets module.
rasovica Jul 29, 2017
24c65b6
Removing old secrets module.
rasovica Jul 29, 2017
19e50d3
Improving the exceptions thrown.
rasovica Jul 29, 2017
a2ae735
Optimizing docker.
rasovica Jul 29, 2017
c04d4f0
Removing python secrets module in favor of secrets in environment var…
rasovica Jul 30, 2017
e6cb50e
Pulling secrets from environmental variables.
rasovica Jul 30, 2017
9deccc4
Improving comments.
rasovica Jul 30, 2017
2f4f98b
Adding paypal token from environment.
rasovica Jul 30, 2017
bf9bf17
Working mainpage container.
rasovica Aug 2, 2017
baaff8c
Adding database.
rasovica Aug 8, 2017
6594ec0
Adding custom database with ip4r support.
rasovica Aug 11, 2017
77cef7b
Removing nodewatcher v2 step 1.
rasovica Aug 18, 2017
dc63eed
Removing nodewatcher v2 step 2.
rasovica Aug 18, 2017
cc3698c
Fixing wrong dependencies.
rasovica Aug 18, 2017
95fa409
Change postgresql to 8.4.
rasovica Aug 23, 2017
defbe19
Add uwsgi.
rasovica Aug 24, 2017
cbcf209
Remove nodewatcher from path.
rasovica Aug 24, 2017
59a766e
Nginx support.
rasovica Aug 26, 2017
d02adfd
Nginx support with fixed permission issues.
rasovica Aug 26, 2017
4aff742
Nginx support with fixed permission issues.
rasovica Aug 26, 2017
e30abd5
Updating readme file.
rasovica Aug 28, 2017
75d77c4
Updating readme file to rst.
rasovica Aug 28, 2017
96a67ce
Fixing a typo.
rasovica Aug 28, 2017
34b03d5
Fixing a typo.
rasovica Aug 28, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[submodule "docker/database"]
path = docker/database
url = https://github.com/Banno/docker-postgresql-8.4/
branch = postgres-8.4
35 changes: 35 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
FROM tozd/runit:ubuntu-xenial

EXPOSE 80/tcp
ENV DJANGO_SETTINGS_MODULE mainpage.settings
#Set secrets by uncommenting lines below
#ENV DB_PASSWORD ...
#Make this unique, and don't share it with anybody.
#ENV SECRET_KEY ...
#You can find them at: https://www.google.com/recaptcha/admin
#ENV RECAPTCHA_PUBLIC_KEY ...
#ENV RECAPTCHA_PRIVATE_KEY ...
#ENV PAYPAL_IDENTITY_TOKEN_PRODUCTION ...

# Update packages
RUN apt-get update -q -q && \
apt-get install --no-install-recommends -y git curl python python-dev python-pip python-setuptools build-essential libgeoip-dev libpq-dev swig libxml2-dev libxslt1-dev subversion mercurial libaprutil1 apache2-dev python2.7-dev nginx
RUN pip install --upgrade pip setuptools wheel six requests

RUN ln -sf /dev/stdout /var/log/nginx/access.log
RUN ln -sf /dev/stdout /var/log/nginx/error.log

ADD ./requirements.txt /code/requirements.txt
ADD ./requirements-production.txt /code/requirements-production.txt

# Install Python package dependencies
RUN pip install -r /code/requirements-production.txt

# Remove unneeded build-time dependencies
RUN apt-get purge python-dev build-essential -y && \
apt-get autoremove -y && \
rm -f /code/packages.txt /code/requirements.txt
COPY ./etc /etc
# Add the current version of the code (needed for production deployments)
WORKDIR /code
ADD . /code
59 changes: 34 additions & 25 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,37 +1,46 @@
*wlan slovenija* main webpage
=============================
Mainpage
========

Development installation
------------------------
This is a repository for wlan-si web-page docker, it contains:

1. Clone from GitHub_ both webpage and nodewatcher_ repositories::
- Web container for: django, uwgsi, nginx
- Database container with postgresql

git clone https://github.com/wlanslovenija/mainpage.git
git clone https://github.com/wlanslovenija/nodewatcher.git
Installation
------------

You can also use SSH-based URLs or URLs of your forks.
Mainpage requires `docker <https://www.docker.com/>`__ and
`docker-compose <https://docs.docker.com/compose/>`__ to run.

2. Create and activate new `Python virtual environment`_::
Download the repository to disk and run docker-compose build.

virtualenv --no-site-packages --distribute ~/.virtualenv/mainpage
source ~/.virtualenv/mainpage/bin/activate

3. Move to location where you cloned webpage repository and run ``devsetup``
script::
.. code:: sh

python scripts/devsetup.py

This script will install all requirements and import database dump to local database.
git clone https://github.com/wlanslovenija/mainpage
cd mainpage
docker-compose build

4. Run::
Then just run the docker-compose up, which will set up the database
create media and static folders, merge the database, and bring up the
production server for mainpage.

python manage.py runserver
.. code:: sh

and start developing!
docker-compose up

You can safely rerun the ``devsetup`` as needed, but be advised that local
database content **will NOT be preserved**.
About
-----

.. _GitHub: https://github.com/
.. _nodewatcher: http://dev.wlan-si.net/wiki/Nodewatcher
.. _Python virtual environment: http://www.virtualenv.org
This repository was a big part of my 2017 Gsoc project, it was suppose
to revive the unmaintained repository of wlan-si.net web-page, which had
a lot of broken dependencies, outdated dependencies and was not yet
running inside of a docker container.

I managed to successfully create docker containers for both website and
database but my lack of django knowledge left me short of actually
fixing the django project that was suppose to run.

Future
------

All that is left is for someone to fix the broken django project.
22 changes: 22 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
version: '3'
services:
web:
build: .
command: "service nginx start; scripts/docker-wait-pgsql; python scripts/migrate.py; uwsgi --socket /var/tmp/nodewatcher.sock --wsgi-file /code/mainpage/wsgi.py --uid www-data --gid www-data"
entrypoint: scripts/docker-run
environment:
PYTHONUNBUFFERED: 1
volumes:
- .:/code
ports:
- "80:80"
db:
build:
context: docker/database/
dockerfile: Dockerfile
environment:
POSTGRESQL_USER: wlansi_cms
POSTGRESQL_PASS: mainpage
POSTGRESQL_DB: wlansi
expose:
- 5432
25 changes: 25 additions & 0 deletions docker/database/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
FROM ubuntu:14.04

RUN apt-get update && apt-get install -y wget

ADD pgdg.list /etc/apt/sources.list.d/pgdg.list
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -

RUN apt-get -qq update && LC_ALL=en_US.UTF-8 DEBIAN_FRONTEND=noninteractive apt-get install -y -q postgresql-8.4 libpq-dev

# /etc/ssl/private can't be accessed from within container for some reason
# (@andrewgodwin says it's something AUFS related)
RUN mkdir /etc/ssl/private-copy; mv /etc/ssl/private/* /etc/ssl/private-copy/; rm -r /etc/ssl/private; mv /etc/ssl/private-copy /etc/ssl/private; chmod -R 0700 /etc/ssl/private; chown -R postgres /etc/ssl/private

## Add over config files
ADD postgresql.conf /etc/postgresql/8.4/main/postgresql.conf
ADD pg_hba.conf /etc/postgresql/8.4/main/pg_hba.conf
RUN chown postgres:postgres /etc/postgresql/8.4/main/*.conf
ADD init-postgresql /usr/local/bin/init-postgresql
RUN chmod +x /usr/local/bin/init-postgresql

VOLUME ["/var/lib/postgresql"]
EXPOSE 5432

CMD ["/usr/local/bin/init-postgresql"]

25 changes: 25 additions & 0 deletions etc/nginx/sites-enabled/default
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
upstream mainpage {
server unix:/var/tmp/nodewatcher.sock;
}
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
server {
listen 80 default_server;
server_name _;
charset utf-8;
root /tmp;

location /static {
alias /code/mainpage/wlansi/static;
}

location /media {
alias /code/mainpage/media/;
}

location / {
uwsgi_pass mainpage;
include /etc/nginx/uwsgi_params;
}
}

16 changes: 16 additions & 0 deletions etc/nginx/uwsgi_params
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@

uwsgi_param QUERY_STRING $query_string;
uwsgi_param REQUEST_METHOD $request_method;
uwsgi_param CONTENT_TYPE $content_type;
uwsgi_param CONTENT_LENGTH $content_length;

uwsgi_param REQUEST_URI $request_uri;
uwsgi_param PATH_INFO $document_uri;
uwsgi_param DOCUMENT_ROOT $document_root;
uwsgi_param SERVER_PROTOCOL $server_protocol;
uwsgi_param HTTPS $https if_not_empty;

uwsgi_param REMOTE_ADDR $remote_addr;
uwsgi_param REMOTE_PORT $remote_port;
uwsgi_param SERVER_PORT $server_port;
uwsgi_param SERVER_NAME $server_name;
4 changes: 4 additions & 0 deletions etc/service/nginx/run
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash -e

exec /usr/sbin/nginx -g 'daemon off;' 2>&1

2 changes: 2 additions & 0 deletions etc/service/postgresql/run.initialization
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
apt -y install postgresql-9.6-ip4r
psql -c "create extension ip4r" wlansi
7 changes: 7 additions & 0 deletions etc/service/uwsgi/log/run
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/bash -e

mkdir -p /var/log/uwsgi
chown nobody:nogroup /var/log/uwsgi

exec chpst -u nobody:nogroup svlogd -tt /var/log/uwsgi

13 changes: 13 additions & 0 deletions etc/service/uwsgi/run
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash -e

mkdir -p /media
chown -R www-data:www-data /media

mkdir -p /static
cd /

python manage.py collectstatic --noinput
python manage.py migrate --noinput

exec /usr/bin/uwsgi --plugin python --ini /etc/uwsgi/nodewatcher.ini 2>&1

10 changes: 10 additions & 0 deletions etc/uwsgi/mainpage.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[uwsgi]
uid = www-data
gid = www-data
chdir = /code
module = mainpage.wsgi
master = true
processes = 10
socket = /var/tmp/mainpage.sock
harakiri = 120
max-requests = 1000
Empty file added mainpage/frontend/__init__.py
Empty file.
Empty file.
Loading