Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] add caching action for elephant-data #633

Open
wants to merge 87 commits into
base: master
Choose a base branch
from

Conversation

Moritz-Alexander-Kern
Copy link
Member

@Moritz-Alexander-Kern Moritz-Alexander-Kern commented Apr 18, 2024

This PR adds a GitHub Actions runner for caching data from the Gin Elephant data repository and enabling local data usage for validation tests with the elephant.datasets module.
This reduces the time required for data retrieval from gin, overall reducing the runtime of the GitHub Actions runners.

GitHub Actions Runner: Added a new GitHub Actions runner to cache data from the Gin Elephant data repository. Caching action is configured to execute as a cron job on a daily basis. This automated process ensures that data caches remains up-to-date.

Local Data Usage for Validation Tests: The elephant.datasets module has been modified to use local data for validation tests. Additionally validation tests across various modules have been adapted to accommodate the option of using either local data or retrieving data from the Gin Elephant repository.
The path to local data can be provided by setting the environment variable ELEPHANT_DATA_LOCATION.

Changes to CI
Current CI runners have been adapted to retrieve the cached elephant-data.

@coveralls
Copy link
Collaborator

coveralls commented Apr 18, 2024

Coverage Status

coverage: 88.401% (+0.05%) from 88.35%
when pulling f6e6353 on INM-6:enh/add_caching_action
into 0984e19 on NeuralEnsemble:master.

@Moritz-Alexander-Kern Moritz-Alexander-Kern changed the title [ENH] add caching acion for elephant-data [ENH] add caching action for elephant-data Apr 19, 2024
@NeuralEnsemble NeuralEnsemble deleted a comment from pep8speaks Jun 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Include a datalad centric way of downloading validation data files
2 participants