Update: Python 3.5, Django 1.9, Ubuntu 16.04

[antoviaque]

Includes:

• Python 3.5
• Django 1.9
• Ubuntu 16.04
• Pylint 1.5
• All other Python dependencies updated to their latest version.

Exclusions of scope, to be addressed post-merge:

• Upstream warnings raised by the upgrade (OC-1571)
• pylint exclusions cleanup (OC-1572, eg. 1 2 3)

Local test instructions:

1. Make a copy of your current box, in case you need to revert to it: https://www.vagrantup.com/docs/virtualbox/boxes.html
2. After switching to the current branch: vagrant destroy && vagrant up
3. Test the most important features (including provisioning a sandbox from the shell, with the worker processes working separately)

Todo:

• Get tests to pass (green build)
• Get server & shell commands to work
• Add OC-* tickets in backlog for exclusions of scope

[antoviaque]

Currently fails due to pylint not supporting python 3.5 just yet. Seems to be in progress, for the next pylint release: https://bitbucket.org/logilab/pylint/issues/643/regression-attributeerror-call-object-has

[antoviaque]

Rebased on top of #10 which should get merged before.

[omarkhan]

@antoviaque pylint should support python 3.5 now.

[antoviaque]

@omarkhan Yup, the dependencies need an update - Django 1.9 has been released too, for example.

[smarnach]

Django 1.9 isn't an LTS version, so if we do upgrade we will be forced to upgrade for every minor version they release. (I'm just pointing this out and not implying we shouldn't do this.)

[antoviaque]

@smarnach True - although waiting for the next LTS release also means accumulating quite a lot of technical debt in the meantime, as you still need to address the individual version upgrades/incompatibilities at once when the next LTS does come, instead of doing it progressively - you likely end up with more work overall, no?

[smarnach]

@antoviaque I don't think it's more work overall, but it's most definitely less pleasant to have to do it all at once, and you miss out on any Django improvements in the meantime. So yeah, we should probably do all the minor version upgrades.

[bradenmacdonald]

@antoviaque Did you run this in "production mode" or deploy it? Because the Procfile here no longer works:

01:35:31 periodic.1  | Usage: manage.py run_huey [options] 
01:35:31 periodic.1  | 
01:35:31 periodic.1  | Run the queue consumer
01:35:31 periodic.1  | 
01:35:31 periodic.1  | manage.py: error: no such option: --periodic

The fix is simply to remove --periodic from the Procfile, since that option is implied whenever --no-periodic is not specified.

[antoviaque]

@bradenmacdonald Ah damn - I think I only ever ran it with make rundev, sorry for having missed that. The deployment itself is done in OC-1533, so we haven't seen it yet. (FYI @smarnach ). I'll have a look and open a PR for the change.

[antoviaque]

@bradenmacdonald Done: #70

[bradenmacdonald]

I noticed in my VM that since this upgrade (and new vagrant setup), the bash history of the vagrant user was not being saved (which was annoying), because ~/.bash_history was owned by root. Any idea why? I don't see any reference to it in the Vagrantfile or bootstrap script.

[antoviaque]

@bradenmacdonald That is likely coming from the box I used as a base for building opencraft/xenial64 - likely a way for the upstream to ensure he doesn't forget to clear his history before publishing his box : ) If you want I can edit the base box to change the owner back to vagrant. Editing, publishing and testing the base box takes time though, so alternatively we could put this in a ticket for the next update? The plan is eventually to switch to the official ubuntu/xenial64 box anyway, once they have solved the issues it currently has.

[bradenmacdonald]

@antoviaque We could also just put a fix into the Vagrantfile provisioning section, especially if/when we move to the official upstream box.

[antoviaque]

@bradenmacdonald I think the upstream box should not have this - or did you had to do that in the previous box, before the upgrade? My hope is that we'll not have to do anything then - but we'll need to check. If the upstream box also has this, then yep the Vagrantfile would work (or the ansible playbook, actually.)

[bradenmacdonald]

OK, hopefully that is the case :)