Skip to content
This repository has been archived by the owner on Aug 22, 2022. It is now read-only.

Update: Python 3.5, Django 1.9, Ubuntu 16.04 #6

Merged
merged 11 commits into from
May 2, 2016
Merged

Update: Python 3.5, Django 1.9, Ubuntu 16.04 #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
8013816
Update build to use Python 3.5
antoviaque Sep 14, 2015
ede58ab
Upgrade Ubuntu to 16.04 (Xenial)
antoviaque Mar 29, 2016
55ff620
Updated dependencies
antoviaque Oct 10, 2015
4d843ab
Django 1.9 Upgrade - Fix incompatible changes
antoviaque Mar 29, 2016
8eaf669
Pylint 1.5 upgrade - Fix new warnings
antoviaque Mar 29, 2016
f2e09b5
Djangular 0.8.0 - Fix backward incompatible name change
antoviaque Mar 29, 2016
cbb0b09
Add `make manage <...>` command for easier access to Django's manage.py
antoviaque Apr 24, 2016
b56e4a7
Move the vagrant xenial base image to the Vagrant Atlas repository
antoviaque Apr 30, 2016
cc3479c
Update copyright notice - Add 2016 & switch email to contact@opencraft
antoviaque Apr 30, 2016
ab3cffa
Make manage.py executable to fix new CircleCI complaint
antoviaque Apr 30, 2016
0d16032
Add vagrant vbguest plugin to ensure matching guest additions
antoviaque May 1, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand All @@ -19,13 +19,13 @@

WORKERS = 4
SHELL = /bin/bash
HONCHO_MANAGE := honcho run python3 manage.py
HONCHO_MANAGE := honcho run python3 manage.py


# Parameters ##################################################################

# For `test_one` use the rest as arguments and turn them into do-nothing targets
ifeq (test_one,$(firstword $(MAKECMDGOALS)))
ifeq ($(firstword $(MAKECMDGOALS)),$(filter $(firstword $(MAKECMDGOALS)),test_one manage))
RUN_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
$(eval $(RUN_ARGS):;@:)
endif
Expand All @@ -37,8 +37,6 @@ all:
rundev

apt_get_update:
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list
sudo apt-get update

clean:
Expand All @@ -60,6 +58,9 @@ install_virtualenv_system:
collectstatic: clean static_external
honcho run ./manage.py collectstatic --noinput

manage:
$(HONCHO_MANAGE) $(RUN_ARGS)

migrate: clean
$(HONCHO_MANAGE) migrate

Expand Down
10 changes: 7 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ First, install [VirtualBox](https://www.virtualbox.org/wiki/Downloads) and

vagrant up

This will provision a virtual machine running Ubuntu 14.04, set up local
This will provision a virtual machine running Ubuntu 16.04, set up local
Postgres, MySQL, MongoDB and Redis, install the dependencies and run the tests.

Once the virtual machine is up and running, you can ssh into it with this
Expand All @@ -45,7 +45,7 @@ web browser.

If you prefer not to use Vagrant, you can install OpenCraft manually. Refer to
the [bootstrap](bin/bootstrap) script used by Vagrant for an example.
Instructions based on Ubuntu 14.04.
Instructions based on Ubuntu 16.04.

Install the system package dependencies & virtualenv:

Expand Down Expand Up @@ -258,7 +258,7 @@ Creating a user
In order to login to the development server locally you will need to create a
superuser by running:

honcho run ./manage.py createsuperuser
make manage createsuperuser

Once created, you will be able to login with the username and password you set
up.
Expand Down Expand Up @@ -432,6 +432,10 @@ manage.py
You can also access the Django `manage.py` command directly, using Honcho to
load the environment:

make manage <command>

Or run any command using the application environment:

honcho run ./manage.py <command>


Expand Down
15 changes: 14 additions & 1 deletion Vagrantfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,25 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :

unless Vagrant.has_plugin?("vagrant-vbguest")
raise "Please install the vagrant-vbguest plugin by running `vagrant plugin install vagrant-vbguest`"
end

Vagrant.configure(2) do |config|
config.vm.box = 'ubuntu/trusty64'
# TODO: Switch back to the official box once it is released
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@antoviaque They seem to have an official box available for this now: https://vagrantcloud.com/ubuntu/boxes/xenial64. Might be worth trying to get it into this PR?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@itsjeyd I had tried it and was actually hoping to use it, but it looks like it still has the same issues it had before the xenial release, and doesn't appear to be correctly prepared to work with vagrant - ie uses the default ubuntu user instead of vagrant, and fails during provisioning (it doesn't seem to have the guest additions installed either):

antoviaque@antoviaque-laptop:~/prog/opencraft$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'ubuntu/xenial64'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'ubuntu/xenial64' is up to date...
==> default: Setting the name of the VM: ubuntu-xenial-16.04-cloudimg
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 2001 => 2001 (adapter 1)
    default: 5000 => 5000 (adapter 1)
    default: 8888 => 8888 (adapter 1)
    default: 22 => 2222 (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: ubuntu
    default: SSH auth method: password
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
    default: Warning: Remote connection disconnect. Retrying...
    default: Warning: Authentication failure. Retrying...
    default: Warning: Remote connection disconnect. Retrying...
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
    default: /home/vagrant/opencraft => /home/antoviaque/prog/opencraft
Failed to mount folders in Linux guest. This is usually because
the "vboxsf" file system is not available. Please verify that
the guest additions are properly installed in the guest and
can work properly. The command attempted was:

mount -t vboxsf -o uid=`id -u ubuntu`,gid=`getent group ubuntu | cut -d: -f3` home_vagrant_opencraft /home/vagrant/opencraft
mount -t vboxsf -o uid=`id -u ubuntu`,gid=`id -g ubuntu` home_vagrant_opencraft /home/vagrant/opencraft

The error output from the last command was:

sudo: unable to resolve host ubuntu-xenial
mesg: ttyname failed: Inappropriate ioctl for device
mount: unknown filesystem type 'vboxsf'

config.vm.box = 'opencraft/xenial64'
config.vm.synced_folder '.', '/vagrant', disabled: true
config.vm.synced_folder '.', '/home/vagrant/opencraft'

config.vbguest.auto_update = true
config.vbguest.auto_reboot = true

config.vm.network 'forwarded_port', guest: 2001, host: 2001
config.vm.network 'forwarded_port', guest: 5000, host: 5000
config.vm.network 'forwarded_port', guest: 8888, host: 8888

config.ssh.forward_x11 = true

config.vm.provision 'shell',
path: 'bin/bootstrap',
privileged: false,
Expand Down
2 changes: 1 addition & 1 deletion api/router.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand Down
2 changes: 1 addition & 1 deletion api/urls.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand Down
17 changes: 11 additions & 6 deletions bin/bootstrap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
#!/bin/bash
# This script provisions a development environment with local postgres and
# redis servers for development and testing. Assumes ubuntu 14.04
# redis servers for development and testing. Assumes Ubuntu 16.04

set -e

# If this is a vagrant VM, cd to /vagrant on login
# If this is a vagrant VM, cd to ~/opencraft on login
if [[ $USER == 'vagrant' ]]; then
cd /vagrant
grep -Fq 'cd /vagrant' ~/.bashrc || echo 'cd /vagrant' >> ~/.bashrc
cd ~/opencraft
grep -Fq 'cd ~/opencraft' ~/.bashrc || echo 'cd ~/opencraft' >> ~/.bashrc
fi

# Install system packages
Expand All @@ -28,14 +28,19 @@ if [[ $USER == 'vagrant' ]]; then
fi

# Install python dependencies
pip3 install -r requirements.txt
pip install -r requirements.txt

# TODO: Should only be needed for the unofficial vagrant image
# Remove when switching to the official, when vagrant release a xenial box
echo "update user set plugin='mysql_native_password' where user='root' and host='localhost'; flush privileges;" \
| sudo -H -u root mysql mysql

# Create postgres user
sudo -u postgres createuser -d $USER ||
echo "Could not create postgres user '$USER' - it probably already exists"

# Allow access to postgres from localhost without password
cat << EOF | sudo tee /etc/postgresql/9.3/main/pg_hba.conf
cat << EOF | sudo tee /etc/postgresql/9.5/main/pg_hba.conf
local all postgres peer
local all all trust
host all all 127.0.0.1/32 trust
Expand Down
2 changes: 1 addition & 1 deletion circle.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
machine:
python:
version: 3.4.3
version: 3.5.1
dependencies:
pre:
- pip install --upgrade pip
Expand Down
2 changes: 1 addition & 1 deletion debian_db_packages.lst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
postgresql
mysql-server
mongodb-org
mongodb
3 changes: 2 additions & 1 deletion instance/admin.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand All @@ -23,6 +23,7 @@
# Imports #####################################################################

from django.contrib import admin

from instance.models.instance import SingleVMOpenEdXInstance
from instance.models.log_entry import GeneralLogEntry, InstanceLogEntry, ServerLogEntry
from instance.models.server import OpenStackServer
Expand Down
9 changes: 4 additions & 5 deletions instance/ansible.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand All @@ -22,20 +22,19 @@

# Imports #####################################################################

from contextlib import contextmanager
import logging
import os
import shutil
import subprocess
import yaml

from contextlib import contextmanager
from tempfile import mkdtemp, NamedTemporaryFile
import yaml

from django.conf import settings


# Logging #####################################################################

import logging
logger = logging.getLogger(__name__)


Expand Down
2 changes: 1 addition & 1 deletion instance/api/instance.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand Down
2 changes: 1 addition & 1 deletion instance/api/server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand Down
8 changes: 4 additions & 4 deletions instance/gandi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand All @@ -22,6 +22,7 @@

# Imports #####################################################################

import logging
import time
import xmlrpc.client

Expand All @@ -31,7 +32,6 @@

# Logging #####################################################################

import logging
logger = logging.getLogger(__name__)


Expand All @@ -45,14 +45,14 @@ def __init__(self, api_url='https://rpc.gandi.net/xmlrpc/'):
self.client = xmlrpc.client.ServerProxy(api_url)

@property
def api_key(self): #pylint: disable=no-self-use
def api_key(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@antoviaque Nice to see that we can remove these now :)

"""
Gandi API key
"""
return settings.GANDI_API_KEY

@property
def zone_id(self): #pylint: disable=no-self-use
def zone_id(self):
"""
Gandi Zone ID of the domain
"""
Expand Down
8 changes: 4 additions & 4 deletions instance/github.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand All @@ -22,19 +22,19 @@

# Imports #####################################################################

import logging
import functools
import operator
import re
import requests
import yaml

from django.conf import settings
from django.template.defaultfilters import truncatewords
import requests
import yaml


# Logging #####################################################################

import logging
logger = logging.getLogger(__name__)


Expand Down
2 changes: 1 addition & 1 deletion instance/logger_adapter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand Down
7 changes: 3 additions & 4 deletions instance/logging.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand All @@ -22,14 +22,13 @@

# Imports #####################################################################

from functools import wraps
import logging
import traceback

from functools import wraps
from swampdragon.pubsub_providers.data_publisher import publish_data

from django.apps import apps
from django.db import models
from swampdragon.pubsub_providers.data_publisher import publish_data

from instance.serializers.logentry import LogEntrySerializer

Expand Down
4 changes: 2 additions & 2 deletions instance/models/instance.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand All @@ -22,6 +22,7 @@

# Imports #####################################################################

from functools import partial
import logging
import string

Expand All @@ -32,7 +33,6 @@
from django.utils import timezone
from django.utils.crypto import get_random_string
from django_extensions.db.models import TimeStampedModel
from functools import partial

from instance.gandi import GandiAPI
from instance.logger_adapter import InstanceLoggerAdapter
Expand Down
2 changes: 1 addition & 1 deletion instance/models/log_entry.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand Down
2 changes: 1 addition & 1 deletion instance/models/mixins/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenCraft -- tools to aid developing and hosting free software projects
# Copyright (C) 2015 OpenCraft <xavier@opencraft.com>
# Copyright (C) 2015-2016 OpenCraft <contact@opencraft.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand Down
Loading