Codespaces (conda): Fix No space left on device

[mkoeppe]

We put the conda package files, only needed during install, in a tmpfs.
This allows the creation of the Docker image to succeed, even on the smallest Codespace machines.

Fixes #36952

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

[tscrim]

LGTM.

[mkoeppe]

Thanks!

[tobiasdiez]

As I have explained in #36952, this is not really a nice fix and needs further investigation of what's the actual origin for the space problems (sorry, should have posted this here in the PR instead).

Also, I cannot reproduce the space problems at all. A new codespace created off the default branch has >12gb free.

[mkoeppe]

As I have explained in #36952, this is not really a nice fix

Also there you did not explain what would make it not "nice".

and needs further investigation of what's the actual origin for the space problems

It does not need further investigation. I investigated it, which enabled me to fix it.

Also, I cannot reproduce the space problems at all.

You'll have to explain what precisely you tried, or this report has no value.

[mkoeppe]

I'll note that declaring that "the root cause is not understood" is not appropriate review activity. (Previous instance of this method: #36498 (comment))

It is normal and expected - as we invite everyone to participate in review - that not all reviewers understand all technical points of a PR. This does not mean that more info is needed or more work is needed.

I would suggest for future participation in the review process something along the following lines: "Could you add a comment to the file that explains what the added line does, and why it is necessary - after all, a new codespace created off the default branch has >12gb free."

[tobiasdiez]

and needs further investigation of what's the actual origin for the space problems

It does not need further investigation. I investigated it, which enabled me to fix it.

Awesome, could you then please share your analysis and answer the questions raised in #36952 (comment)?

Also, I cannot reproduce the space problems at all.

You'll have to explain what precisely you tried, or this report has no value.

I don't know what more to say then "create a new codespace on the develop branch". Maybe the direct link helps...https://github.com/codespaces/new?hide_repo_select=true&ref=develop&repo=597660615&skip_quickstart=true

[mkoeppe]

Also, I cannot reproduce the space problems at all.

You'll have to explain what precisely you tried, or this report has no value.

I don't know what more to say then "create a new codespace on the develop branch".

As you well know, one can select the machine type to use. Which one did you use.

[mkoeppe]

and needs further investigation of what's the actual origin for the space problems

It does not need further investigation. I investigated it, which enabled me to fix it.

Awesome, could you then please share your analysis and answer the questions raised in #36952 (comment)?

Yes, let's take a look what the questions are.

The conda packages should only take about 2 * 7gb of space. That should still give plenty of space to build sage. Also it looks like that it already fails for you at the download, which would mean that it has less then 7gb to start with. Strange.

No question so far. You're expressing your expectations and puzzlement.

Did you analyze what takes so much space, except for the conda packages?

No, and this question does not make sense because the container on which "docker build" is run in order to build our image is not under our control.

It is simply an observed fact what space is available in the temporary container that "docker build" uses. I already shared this observed fact at
#36952 (comment)

Filesystem      Size  Used Avail Use% Mounted on
overlay          32G   23G  7.5G  76% /
tmpfs            64M     0   64M   0% /dev
shm              64M     0   64M   0% /dev/shm
/dev/sda1        44G   52K   42G   1% /tmp
/dev/root        29G   22G  7.2G  76% /vscode
/dev/loop3       32G   23G  7.5G  76% /workspaces
tmpfs           3.9G     0  3.9G   0% /proc/acpi
tmpfs           3.9G     0  3.9G   0% /proc/scsi                         
tmpfs           3.9G     0  3.9G   0% /sys/firmware

So, not enough space in /opt (where the conda packages go to) as per your estimates.

Can we now please know what would make my solution "not nice", as you say?

[tobiasdiez]

Did you analyze what takes so much space, except for the conda packages?

No, and this question does not make sense because the container on which "docker build" is run in order to build our image is not under our control.

It is simply an observed fact what space is available in the temporary container that "docker build" uses. I already shared this observed fact at #36952 (comment)

I don't understand this. The onCreateCommand is run inside the main container, after the image is build.

[mkoeppe]

In pictures - it's happening in this step.

[tobiasdiez]

Yes, and whatever is run at that point is inside the main container image that already has been build. So it does make sense to ask the question what takes up these 32gb of space at that moment.

[mkoeppe]

whatever is run at that point is inside the main container image that already has been build

Images don't "run". Containers do. And how much space they have when they run depends on more than what is "inside it".

[mkoeppe]

If you are still continuing your review, set it to "needs review" please.

[mkoeppe]

@tobiasdiez I had to call you out many times last year for abusive use of your Triage team privileges.
That you are now apparently using an automatic script that manipulates labels is a new escalation.

[tobiasdiez]

@tobiasdiez I had to call you out many times last year for abusive use of your Triage team privileges.

Please stop these repeated, and wrong, accusations of abusive use of privileges. You don't have the rights to publicly judge other members of the community. If you think I've crossed the boundaries, please directed this to the appropriate channels, such as the sage-abuse team, for thorough investigation and resolution.

But do please explain publicly why you think that removing the positive review label on a PR labeled as "disputed" (and thus "controversial, needs community involvement") is an abuse of privileges, or publicly apologize for making such wrong accusations.

[mkoeppe]

Tobias, as you know, I added the "disputed" label here. I did so to mark this PR as (yet another) one that is affected by your persistent misconduct.

[github-actions]

Documentation preview for this PR (built with commit d67b011; changes) is ready! 🎉

[vbraun]

Editor decision

Will merge since there is a PR already. Don't really care about workarounds for github codespace limitations.

[tobiasdiez]

@vbraun please reconsider this. With a new codespace created from the develop branch (without any prebuilds, smallest system config) the build goes through without any problems and afterwards one has quite a bit of space available:

@tobiasdiez ➜ /workspaces/sage (develop) $ df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          32G  3.9G   26G  14% /

While Matthias write in the changed file that he sees with this PR here

# Filesystem      Size  Used Avail Use% Mounted on
# overlay          32G   23G  7.5G  76% /

I have no idea how he got these numbers. Maybe there were some temporary problems with how the codespaces were created and indeed some file from the build directory leaked into the codespace. I have no idea, but whatever it was it seemed to be fixed either on github's side or perhaps by #35986.

So, in my opinion, there is no need for any workaround and, in particular, for removing the conda cache that is quite handy to have when you work with conda to create a new env. (And the whole point of this codespace is to make it convient to work with conda.)

[vbraun]

OK if its not needed any more then even better.

In general one should "treat containers as cattle, not pets" though. Don't update the container, just get rid of it and build a new one.

[mkoeppe]

I can confirm that the "No space left" condition does not repro as of today.