CACHE_DIR created too late when using black as a library

[AlJohri]

Describe the bug Black fails to generate grammar files when using it as a library as it has no cache directory.

To Reproduce

Run docker build . --no-cache with the following Dockerfile:

FROM python:alpine
RUN apk add --quiet --no-cache gcc musl-dev
RUN pip install -q black
RUN black --version
RUN python -c "import logging; logging.basicConfig(level='INFO'); import black"
# using black at the CLI will generate the cache directory
RUN black /usr/local/lib/python3.8/site-packages/black.py
# and now the error goes away
RUN python -c "import logging; logging.basicConfig(level='INFO'); import black"

Output:

Sending build context to Docker daemon  2.048kB
Step 1/7 : FROM python:alpine
 ---> 4c30403e92a1
Step 2/7 : RUN apk add --quiet --no-cache gcc musl-dev
 ---> Running in b2ebc87ca240
Removing intermediate container b2ebc87ca240
 ---> 7eaf16ac33ee
Step 3/7 : RUN pip install -q black
 ---> Running in f35cf5216c32
Removing intermediate container f35cf5216c32
 ---> f5f0566d901a
Step 4/7 : RUN black --version
 ---> Running in 979a9e18c2ba
black, version 19.10b0
Removing intermediate container 979a9e18c2ba
 ---> 921b5110292a
Step 5/7 : RUN python -c "import logging; logging.basicConfig(level='INFO'); import black"
 ---> Running in e397fb8de383
INFO:blib2to3.pgen2.driver:Generating grammar tables from /usr/local/lib/python3.8/site-packages/blib2to3/Grammar.txt
INFO:blib2to3.pgen2.driver:Writing grammar tables to /root/.cache/black/19.10b0/Grammar3.8.1.final.0.pickle
INFO:blib2to3.pgen2.driver:Writing failed: [Errno 2] No such file or directory: '/root/.cache/black/19.10b0/tmpz59smnh9'
INFO:blib2to3.pgen2.driver:Generating grammar tables from /usr/local/lib/python3.8/site-packages/blib2to3/PatternGrammar.txt
INFO:blib2to3.pgen2.driver:Writing grammar tables to /root/.cache/black/19.10b0/PatternGrammar3.8.1.final.0.pickle
INFO:blib2to3.pgen2.driver:Writing failed: [Errno 2] No such file or directory: '/root/.cache/black/19.10b0/tmpo51z98bd'
Removing intermediate container e397fb8de383
 ---> f45b0a891017
Step 6/7 : RUN black /usr/local/lib/python3.8/site-packages/black.py
 ---> Running in cecdca8b1662
All done! ✨ 🍰 ✨
1 file left unchanged.
Removing intermediate container cecdca8b1662
 ---> 018298b3b126
Step 7/7 : RUN python -c "import logging; logging.basicConfig(level='INFO'); import black"
 ---> Running in aed6b657c4d8
INFO:blib2to3.pgen2.driver:Generating grammar tables from /usr/local/lib/python3.8/site-packages/blib2to3/Grammar.txt
INFO:blib2to3.pgen2.driver:Writing grammar tables to /root/.cache/black/19.10b0/Grammar3.8.1.final.0.pickle
INFO:blib2to3.pgen2.driver:Generating grammar tables from /usr/local/lib/python3.8/site-packages/blib2to3/PatternGrammar.txt
INFO:blib2to3.pgen2.driver:Writing grammar tables to /root/.cache/black/19.10b0/PatternGrammar3.8.1.final.0.pickle
Removing intermediate container aed6b657c4d8
 ---> c45a0d607331
Successfully built c45a0d607331

Expected behavior

I expect using black as a library to not fail to generate grammar tables. This affects programs that use black only a a library like jupyterlab-code-formatter.

Does this bug also happen on master?

Yes

Additional context Add any other context about the problem here.

This is the root cause of #1143

[AlJohri]

PR: #1224

[ichard26]

Leaving only #1143 opne seems to be a more reasonable option since it's not 100% clear this is the right solution.

[ricoms]

This is happening to me with black version21.6b0 by using package datamodel-code-generator