[BUG] Cannot install from pip in a venv

[enkiusz]

Describe the bug
Following the steps from the installation manual fails when trying to install graphite-web in a venv using pip.

To Reproduce

enki@boxer:~$ sudo rm -rf /opt/
enki@boxer:~$ sudo mkdir -p /opt/graphite
enki@boxer:~$ sudo -i
root@boxer:~# python3 -m venv /opt/graphite/
root@boxer:~# source /opt/graphite/bin/activate
(graphite) root@boxer:~# export PYTHONPATH="/opt/graphite/lib/:/opt/graphite/webapp/"
(graphite) root@boxer:~# pip install --no-binary=:all: https://github.com/graphite-project/whisper/tarball/master
[...]
Successfully installed six-1.14.0 whisper-1.2.0
(graphite) root@boxer:~# pip install --no-binary=:all: https://github.com/graphite-project/carbon/tarball/master
[...]
Successfully installed Automat-20.2.0 PyHamcrest-2.0.2 Twisted-20.3.0 attrs-19.3.0 cachetools-4.1.0 carbon-1.2.0 constantly-15.1.0 hyperlink-19.0.0 idna-2.9 incremental-17.5.0 txAMQP-0.8.2 urllib3-1.25.9 zope.interface-5.1.0
(graphite) root@boxer:~# pip install --no-binary=:all: https://github.com/graphite-project/graphite-web/tarball/master
Collecting https://github.com/graphite-project/graphite-web/tarball/master
[...]
Successfully installed Django-3.0.5 asgiref-3.2.7 cairocffi-1.1.0 cffi-1.14.0 django-tagging-0.4.3 graphite-web-1.2.0 pycparser-2.20 pyparsing-2.4.7 pytz-2020.1 scandir-1.10.0 setuptools-46.1.3 sqlparse-0.3.1
(graphite) root@boxer:~# cd /opt/graphite/
(graphite) root@boxer:/opt/graphite# PYTHONPATH=$PWD/webapp django-admin.py migrate --settings=graphite.settings
Could not import graphite.local_settings, using defaults!
/opt/graphite/lib/python3.6/site-packages/graphite/settings.py:348: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
  warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
Traceback (most recent call last):
  File "/opt/graphite/bin/django-admin.py", line 5, in <module>
    management.execute_from_command_line()
  File "/opt/graphite/lib/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/opt/graphite/lib/python3.6/site-packages/django/core/management/__init__.py", line 377, in execute
    django.setup()
  File "/opt/graphite/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/opt/graphite/lib/python3.6/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/opt/graphite/lib/python3.6/site-packages/django/apps/config.py", line 90, in create
    module = import_module(entry)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/graphite/lib/python3.6/site-packages/graphite/functions/__init__.py", line 9, in <module>
    from graphite.functions.params import Param, ParamTypes, ParamTypeAggFunc  # noqa
  File "/opt/graphite/lib/python3.6/site-packages/graphite/functions/params.py", line 5, in <module>
    from graphite.logger import log
  File "/opt/graphite/lib/python3.6/site-packages/graphite/logger.py", line 96, in <module>
    log = GraphiteLogger() # import-shared logger instance
  File "/opt/graphite/lib/python3.6/site-packages/graphite/logger.py", line 29, in __init__
    level = logging.DEBUG if settings.DEBUG else logging.INFO,
  File "/opt/graphite/lib/python3.6/site-packages/graphite/logger.py", line 68, in _config_logger
    handler = Rotater(log_file, when=when, backupCount=backupCount)
  File "/usr/lib/python3.6/logging/handlers.py", line 202, in __init__
    BaseRotatingHandler.__init__(self, filename, 'a', encoding, delay)
  File "/usr/lib/python3.6/logging/handlers.py", line 57, in __init__
    logging.FileHandler.__init__(self, filename, mode, encoding, delay)
  File "/usr/lib/python3.6/logging/__init__.py", line 1032, in __init__
    StreamHandler.__init__(self, self._open())
  File "/usr/lib/python3.6/logging/__init__.py", line 1061, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
FileNotFoundError: [Errno 2] No such file or directory: '/opt/graphite/lib/python3.6/storage/log/webapp/info.log'
(graphite) root@boxer:/opt/graphite# 

Expected behavior
The django setup script should work.

Environment (please complete the following information):

• OS flavor:

(graphite) root@boxer:/opt/graphite# lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.4 LTS
Release:	18.04
Codename:	bionic
(graphite) root@boxer:/opt/graphite# 

• Graphite-web version [e.g. 1.1.2, 1.0.1, etc]
• Django/Python version [e.g. 1.11/2.7, 2.1/3.8 etc]

(graphite) root@boxer:/opt/graphite# python --version
Python 3.6.9
(graphite) root@boxer:/opt/graphite# 

• Setup type
  pip in venv according to manual

Additional info
The webapp directory doesn't seem to contain the graphite-web application just the content directory with the static html resources.

(graphite) root@boxer:/opt/graphite# ls -l webapp/
total 4
drwxr-xr-x 6 root root 4096 May  3 01:32 content
(graphite) root@boxer:/opt/graphite#

[deniszh]

I think it's duplicate of #2566

[deniszh]

@mgrela : could you please check if workaround from #2566 works? (pip install virtualenv==16.7.10)

[enkiusz]

It did not help me directly but pointed me towards a solution. I was creating my venv using the "venv" module in python 3:

python3 -m venv /opt/graphite

instead of using virtualenv. Virtualenv on my system was 15.1.0 so I did not have to apply the workaround to have virtualenv < 20.x. After building my venv like so:

virtualenv -p python3 /opt/graphite

the graphite-web django code got installed in /opt/graphite/webapp/graphite and django-admin executed successfully.