Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: Skip version cache on read-only filesystems #862

Merged
merged 1 commit into from
Dec 16, 2020

Conversation

effigies
Copy link
Member

@effigies effigies commented Oct 8, 2020

Fixes #853.

Following nipreps/fmriprep#1830.

@pep8speaks
Copy link

pep8speaks commented Oct 8, 2020

Hello @effigies, Thank you for updating!

Cheers! There are no style issues detected in this Pull Request. 🍻 To test for issues locally, pip install flake8 and then run flake8 niworkflows.

Comment last updated at 2020-12-16 19:03:33 UTC

@effigies
Copy link
Member Author

effigies commented Oct 8, 2020

Travis failures are #863. Will not include fix in this PR.

@effigies
Copy link
Member Author

effigies commented Oct 8, 2020

@oesteban There are Bad Gateway errors in the CircleCI tests. URLs aren't exposed, so I don't know what service it's trying to access.

cc @rwblair in case you know and can restart a service.

@rwblair
Copy link
Contributor

rwblair commented Oct 8, 2020

Looks like its running its own version of the webapi locally:
https://github.com/poldracklab/mriqc/blob/5898396026817ce114bbc90bac0dd5d7c48618e0/.circleci/config.yml#L468

Guess I'll rerun job with ssh and inspect nginx logs to see what I can see.

@effigies
Copy link
Member Author

effigies commented Nov 6, 2020

Did we ever figure this out?

@effigies effigies force-pushed the fix/read_only_cache branch 2 times, most recently from edde69a to deb6e49 Compare December 13, 2020 14:53
@effigies
Copy link
Member Author

SSHing in, I see:

$ docker logs dockereve-master_eve_1
[2020-12-13 23:42:24 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-12-13 23:42:24 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2020-12-13 23:42:24 +0000] [1] [INFO] Using worker: sync
[2020-12-13 23:42:24 +0000] [8] [INFO] Booting worker with pid: 8
[2020-12-13 23:42:24 +0000] [9] [INFO] Booting worker with pid: 9
[2020-12-13 23:42:25 +0000] [8] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 119, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 358, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/src/app/app.py", line 21, in <module>
    app.register_blueprint(swagger, url_prefix='/docs/api')
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 98, in wrapper_func
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1156, in register_blueprint
    if blueprint.name in self.blueprints:
AttributeError: module 'eve_swagger.swagger' has no attribute 'name'
[2020-12-13 23:42:25 +0000] [8] [INFO] Worker exiting (pid: 8)
[2020-12-13 23:42:25 +0000] [9] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 119, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 358, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/src/app/app.py", line 21, in <module>
    app.register_blueprint(swagger, url_prefix='/docs/api')
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 98, in wrapper_func
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1156, in register_blueprint
    if blueprint.name in self.blueprints:
AttributeError: module 'eve_swagger.swagger' has no attribute 'name'
[2020-12-13 23:42:25 +0000] [9] [INFO] Worker exiting (pid: 9)
[2020-12-13 23:42:25 +0000] [1] [INFO] Shutting down: Master
[2020-12-13 23:42:25 +0000] [1] [INFO] Reason: Worker failed to boot.

(Message repeats...)

Did some digging, and came up with nipreps/mriqcwebapi#47.

Testing that here.

@effigies
Copy link
Member Author

IQM issue. @oesteban if you could have a look? Or anybody else qualified to look through Circle?

@oesteban
Copy link
Member

Could you rebase?

@effigies effigies force-pushed the fix/read_only_cache branch from a72f03e to 2c60a44 Compare December 16, 2020 19:03
@effigies
Copy link
Member Author

Tests passing and I got an in-principle approval a while back, so I'll go ahead and merge.

I would suggest a release soonish since a lot of these issues are long-standing, though we might also need to release mriqcwebapi. I don't know if there are deployment issues there?

@effigies effigies merged commit 4c005b6 into master Dec 16, 2020
@effigies effigies deleted the fix/read_only_cache branch December 16, 2020 22:14
@yarikoptic
Copy link
Contributor

note to myself -- slated for 0.15.3 or 0.16 whatever comes first, right? ;)

@oesteban
Copy link
Member

I don't know if there are deployment issues there?

You bet, but maybe the webapi wants to give us a nice surprise. @rwblair may actually know better than myself

@effigies
Copy link
Member Author

0.15.3 is out already. I would suggest 0.16, just to mark that it took some work and upgraders may need to consider dependencies.

@yarikoptic
Copy link
Contributor

https://github.com/poldracklab/mriqc/ says latest release 0.15.2

@effigies
Copy link
Member Author

There's a tag that didn't get an official release page. If you click in the releases, you'll see it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Difficulty running singularity image of 0.15.2: Read-only file system: '/home/bidsapp/.cache/mriqc'
5 participants