Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the size of the bids examples repository and rewrite the history #373

Open
Remi-Gau opened this issue Jun 2, 2023 · 5 comments
Open

Comments

@Remi-Gau
Copy link
Contributor

Remi-Gau commented Jun 2, 2023

the bids examples repo is now close to a 100 Mo: this is really not ideal for something meant to be lightweight

Most of the weight comes from the stimuli examples.

$ du -h -d 0  */stimuli | sort -rh
23M    eeg_ds000117/stimuli
8,1M    eeg_ds003645s_hed/stimuli
8,1M    eeg_ds003645s_hed_longform/stimuli
8,1M    eeg_ds003645s_hed_inheritance/stimuli
2,0M    fnirs_automaticity/stimuli

Given that sitmuli folder content is almost completely unspecified in the BIDS specification it seems excessive that they would take so much space.

Discussed during the last maintainers meeting, our first approach would be to zero the content of any stimuli file in there and then rewrite the history.

Pinging @VisLab to get the opinion of the HED team.
Should we be concerned that not having examples of the actual stimuli images may reduce the usefulness of the examples when it to understand the hed tags?

@Remi-Gau
Copy link
Contributor Author

Remi-Gau commented Jun 2, 2023

What ever decision we agree on, the history of the repo may need to be rewritten:

@VisLab
Copy link
Member

VisLab commented Jun 2, 2023 via email

@Remi-Gau
Copy link
Contributor Author

Remi-Gau commented Jun 2, 2023

We can remove the stimuli directories. The dataset on OpenNeuro has them. Do you want me to do it?

If you can do it that'd be much appreciated.

No need to remove them, you can simply truncate the datafiles in them:

https://github.com/bids-standard/bids-examples/blob/master/CONTRIBUTING.md#how-to-truncate-data-files-to-0kb

@Remi-Gau Remi-Gau changed the title Reduce the size of the bids examples repository Reduce the size of the bids examples repository and rewrite the history Jul 14, 2023
@Remi-Gau
Copy link
Contributor Author

Amended the title of this issue to reflect the steps left to do

@yarikoptic
Copy link
Contributor

FWIW, recently-ish added git-replace command could be of help after history rewrite to establish "continuity" of history. See e.g. https://andrewlock.net/reducing-the-size-of-a-git-repository-with-git-replace/ .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

3 participants