push of locally built snap fails on arm64 device

[kenvandine]

Bug Description

When attempting to push a locally built snap to the store with my qualcomm based laptop I get the following stderr:

FATAL ERROR: Data queue size is too large

To Reproduce

snapcraft push SOMELOCALLYBUILT.snap

Environment

snapcraft: 12827

uname: Linux xps13-qc 6.11.0-35-qcom-x1e #35-Ubuntu SMP PREEMPT_DYNAMIC Wed Oct 23 15:18:06 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

snapcraft.yaml

NA

Relevant log output

2024-10-22 16:18:30.743 Starting snapcraft, version 8.4.3
2024-10-22 16:18:30.743 Log verbosity level set to BRIEF
2024-10-22 16:18:30.743 Preparing application...
2024-10-22 16:18:30.744 Configuring application...
2024-10-22 16:18:30.744 Setting up ConfigService
2024-10-22 16:18:30.758 Build plan: platform=None, build_for=None
2024-10-22 16:18:30.758 Running snapcraft push on host
2024-10-22 16:18:30.819 snapcraft internal error: SnapDataExtractionError('thunderbird_128.3.3esr-1_arm64.snap')
2024-10-22 16:18:30.829 Traceback (most recent call last):
2024-10-22 16:18:30.829   File "/snap/snapcraft/12827/lib/python3.10/site-packages/snapcraft_legacy/_store.py", line 58, in get_data_from_snap_file
2024-10-22 16:18:30.829     output = subprocess.check_output(
2024-10-22 16:18:30.829   File "/snap/snapcraft/12827/usr/lib/python3.10/subprocess.py", line 421, in check_output
2024-10-22 16:18:30.829     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
2024-10-22 16:18:30.829   File "/snap/snapcraft/12827/usr/lib/python3.10/subprocess.py", line 526, in run
2024-10-22 16:18:30.829     raise CalledProcessError(retcode, process.args,
2024-10-22 16:18:30.829 subprocess.CalledProcessError: Command '['/snap/snapcraft/12827/usr/bin/unsquashfs', '-d', '/tmp/tmpoh95i6tt/squashfs-root', PosixPath('thunderbird_128.3.3esr-1_arm64.snap'), 'meta/snap.yaml', 'snap/manifest.yaml']' returned non-zero exit status 1.
2024-10-22 16:18:30.829 
2024-10-22 16:18:30.829 During handling of the above exception, another exception occurred:
2024-10-22 16:18:30.829 Traceback (most recent call last):
2024-10-22 16:18:30.829   File "/snap/snapcraft/12827/lib/python3.10/site-packages/craft_application/application.py", line 568, in run
2024-10-22 16:18:30.829     return_code = self._run_inner()
2024-10-22 16:18:30.829   File "/snap/snapcraft/12827/lib/python3.10/site-packages/snapcraft/application.py", line 215, in _run_inner
2024-10-22 16:18:30.829     return_code = super()._run_inner()
2024-10-22 16:18:30.829   File "/snap/snapcraft/12827/lib/python3.10/site-packages/craft_application/application.py", line 549, in _run_inner
2024-10-22 16:18:30.829     return_code = dispatcher.run() or os.EX_OK
2024-10-22 16:18:30.829   File "/snap/snapcraft/12827/lib/python3.10/site-packages/craft_cli/dispatcher.py", line 487, in run
2024-10-22 16:18:30.829     return self._loaded_command.run(self._parsed_command_args)
2024-10-22 16:18:30.829   File "/snap/snapcraft/12827/lib/python3.10/site-packages/snapcraft/commands/upload.py", line 116, in run
2024-10-22 16:18:30.829     snap_yaml, manifest_yaml = get_data_from_snap_file(snap_file)
2024-10-22 16:18:30.829   File "/snap/snapcraft/12827/lib/python3.10/site-packages/snapcraft_legacy/_store.py", line 70, in get_data_from_snap_file
2024-10-22 16:18:30.829     raise SnapDataExtractionError(os.path.basename(snap_path))
2024-10-22 16:18:30.829 snapcraft_legacy.internal.errors.SnapDataExtractionError: Cannot read data from snap 'thunderbird_128.3.3esr-1_arm64.snap'. The file may be corrupted.

### Additional context

_No response_

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-3614.

This message was autogenerated

[mr-cal]

I was unable to reproduce this on an M1.

@kenvandine was unable to reproduce this when running the same subprocess call manually /snap/snapcraft/current/usr/bin/unsquashfs -d /tmp/tmpoh95i6tt/squashfs-root SOMELOCALLYBUILT.snap meta/snap.yaml snap/manifest.yaml on the same machine. It could be useful to try snap run --shell snapcraft and then try unsquashing. Perhaps it's something in snapd's environment.

The snap/manifest.yaml is a red herring - unsquashfs should not fail if the file does not exist.

The FATAL ERROR: Data queue size is too large error comes from unsquashfs: https://github.com/plougher/squashfs-tools/blob/0d1e6b0944f3938962238d886d9d1a8bfdf4dfcc/squashfs-tools/unsquashfs.c#L4607-L4608

Snapcraft is also using the version of squashfs-tools that shipped with Jammy. There have been many bugfixes since then, so it would be instructive to build and test with a copy of snapcraft that uses a newer version.

[dariuszd21]

What's the size of your tmpfs ?
As I can see you're on XPS with X1 chip, so you're probably using 24.10.
May it be related with the oracular change that /tmp is tmpfs mount ?
Out of curiosity, could you check the size of your /tmp dir?

df -h /tmp

https://discourse.ubuntu.com/t/oracular-oriole-release-notes/44878

[kenvandine]

What's the size of your tmpfs ? As I can see you're on XPS with X1 chip, so you're probably using 24.10. May it be related with the oracular change that /tmp is tmpfs mount ? Out of curiosity, could you check the size of your /tmp dir?

df -h /tmp

https://discourse.ubuntu.com/t/oracular-oriole-release-notes/44878

You are correct, I am on 24.10 and it is using tmpfs for /tmp

df reports /tmp is 15G with 15G available