File-copy qrexec service is overly restrictive

[Nurmagoz]

Qubes OS release

4.2

Brief summary

moving/coping files was working ok in 4.1 regardless their name, now its rejected due to the name of the file used.

Steps to reproduce

Try to move/copy a file in none latin letters

Expected behavior

To move/copy file regardless what characters used to name it.

Actual behavior

Tasks

More detailed design in #8332 (comment)

• Extend qfile-unpacker to have extra options for disabling various restrictions (independently): file names validation, symlink target restriction (refusing absolute symlinks or pointing outside of target directory)
• Add a new qubes.UnsafeFilecopy qrexec service with file names validation disabled (but symlinks validation enabled)
• Modify qvm-copy to select the service based on chosen files (see File-copy qrexec service is overly restrictive #8332 (comment))
• Make qubes-builderv2 disable both restrictions when copying files into dispvm

[marmarek]

Indeed R4.2 introduces stricter filename validation, it needs to be (among other things) proper UTF-8 string. Maybe you use some different encoding for your file names?

[Nurmagoz]

Maybe you use some different encoding for your file names?

I was using Debian 11 as a standalone backup, which I had shifted from Qubes 4.1 using Qubes backup. After restoring my backup, I now want to create a new standalone based on Debian 12. I successfully created the new Debian 12 standalone and attempted to transfer my files to it. However, I encountered an issue where only files with non-English characters became stuck during the transfer process.

Workaround: you can change the file names to English before transferring them. Once the files have reached the destination, you can revert the changes by renaming them back to their original names.

[brendanhoar]

Workaround: you can change the file names to English before transferring them. Once the files have reached the destination, you can revert the changes by renaming them back to their original names.

I recommend archiving them all into a single archive file so that you can keep the original names in the destination.

B

[DemiMarie]

Workaround: you can change the file names to English before transferring them. Once the files have reached the destination, you can revert the changes by renaming them back to their original names.

I recommend archiving them all into a single archive file so that you can keep the original names in the destination.

B

Note that this loses the protection provided by qfile-unpacker’s filtering.

[DemiMarie]

The error message (“invalid or incomplete multibyte or wide character”) is horrible, not least because it provides no information as to which character was disallowed. At a minimum, there should be separate errors for “name is not valid UTF-8” and “name contains a forbidden UTF-8 character”, and the latter must state explicitly which character was forbidden.

[jamke]

I recommend archiving them all into a single archive file so that you can keep the original names in the destination.

Yes, and it destroys the whole concept of security improvements that qvm-copy now should provide. The whole strict policy change makes no sense in this case, only affects user experience.

I personally think, that not being able to copy symlinks and files with non-utf-8 chars in name anymore is a major design mistake. The user data should be preserved at all costs. Ext2/3/4 allows non-utf8 chars in path and that is how it is. Only '\0' is not allowed, it is the only exception according to FS specs.

@marmarek @DemiMarie
What about adding some flags to qvm-copy to make it work as it was prior to R4.2, as expected by all current users?
Is it possible to add it before R4.2 release, that will break users experience and force users to pack everything in tar (like @brendanhoar suggested and I will also have to do) just due to the fear that qvm-copy will silently fail with the copy process, skip something or break intentionally?

[jamke]

Moving my messages here:

Let's consider a directory with 100 000 files. It has 1 absolute symlink somewhere in the depth, 1 relative symlink targeting above this directory, and several files that are fine except their filenames have byte sequences that cannot be interpreted as utf-8 string.

User wants to copy it to the other qube (maybe much less secure) the same way as EVERY other copy tool works: like cp and including qvm-copy that existed for many years up to R4.1.

What should the user do to achieve it with the new R4.2 approach?

[jamke]

There are 2 cases: from terminal and from GUI (Nautilus or Dolphin).
What the user should do in both cases to copy files and avoid user's data loss?

[jamke]

Would it make sense to have a "qvm-copy --insecure" tool/option (including in the file manager)?

I think it definitely would!
I understand what you are saying, I just think consequences can be worse, than what's gained, at least in my use cases. Because qvm-copy is already "secure", and the filtering is like adding additional layer. Like: why not check copied files with antivirus, or block copying shell/ELF files or something, it all looks as the same approach that copy tool should do to be called secure.

For me secure is more like:

• for any directory/file it does not cause any code execution itself because of the file naming or data (like buffer overflow or bad parsing code),
• for any directory/file it should not cause user data losses (or increase changes of it); other meaning of secure as in safe.

The R4.1 meets these criteria completely.

---

My main point:
After these changes the user will never be sure if Qubes OS copy tool will work as they expect:

• User moves a directory with a big size, goes to drink tea or even sleep, comes back and sees that there is an error about symlink or filename that happened in the beginning of the process.
• After that user can remove this symlink or tar it and start again, once again not being sure that it will not stop in a minute, or an hour.
• If user is able to auto-skip such files, than they will almost certainly loose some files eventually. At least I would.

[adrelanos]

Best to model the Linux coreutils `cp` command?

[DemiMarie]

@marmarta what do you think? I don’t know enough about UX to be able to make an informed statement here.

@jamke So the problem with symlinks is:

echo > ~/QubesIncoming/x/a
# oops, overwrote ~/.bashrc (or some other important file)
cat ~/QubesIncoming/x/b
# oops, just read e.g. ~/.ssh/id_ed25519 (or some other secret key)

[andrewdavidwong]

As far as I can tell, symlinks are off-topic in this issue.

[DemiMarie]

@jamke This would need to be a separate service (qubes.FileCopyUnsafe?) so that the prompt presented to the user indicates that this is a more dangerous action. @marmarta thoughts on the name?

[jamke]

This would need to be a separate service (qubes.FileCopyUnsafe?) so that the prompt presented to the user indicates that this is a more dangerous action.

@DemiMarie Sounds interesting, I like the idea.

[DemiMarie]

This would need to be a separate service (qubes.FileCopyUnsafe?) so that the prompt presented to the user indicates that this is a more dangerous action.

@DemiMarie Sounds interesting, I like the idea.

I think this is something that could be considered. This would still preserve less metadata than tar or cp -a, because the latter two preserve e.g. file ownership, SELinux contexts, and possibly hard links as well.

[jamke]

I think this is something that could be considered. This would still preserve less metadata than tar or cp -a, because the latter two preserve e.g. file ownership, SELinux contexts, and possibly hard links as well.

I agree.

• time data (like mtime) filenames, filecontent should be preserved as is,
• the ownership can be dropped, as user IDs can be different in general case,
• the rights (chmod) can be preserved if it is OK,
• symlinks should not be affected,
• and hardlinks should be dropped for sure, as it is not a copy process inside one drive by design,
• other metadata (like SELinux contexts) can be dropped to.

The whole process can be considered similar to copying between 2 air-gapped PCs with a flash-drive formatted to ext4. At least it sounds logical and fits architecture of qubes as separated PCs.

[UndeadDevel]

Absolutely agree that there should be a "copy_unsafe" option; ran into this issue now several times and it is extremely annoying to deal with...even when a file name has only Latin characters it sometimes crops up, not even to mention non-Latin ones.

I'll also add that UX is still pretty bad with the error message as the target qube is asked with a pop up dialogue, but the error / failure message is just an echo in the terminal; the last time I did qvm-copy and it failed I actually lost the data, because I assumed that the error would be another pop up dialogue, which didn't happen, and the filename was only Latin characters (and some apparently non-standard apostrophes) and then I shut down the disposable thinking everything had been transferred when it wasn't (it was easy to recreate that data - this time -, but still...super bad UX).

[rustybird]

I assumed that the error would be another pop up dialogue, which didn't happen

That should be fixed by QubesOS/updates-status#4240 + QubesOS/updates-status#4233

Edit: Those are for GUI file copy

[andrewdavidwong]

If you look at related discussions, another way of looking at it is that change in R4.2 to make it refuse some file names is a regression or at least incompatible change that shouldn't happen in minor release...

Fair enough. In that case, it would just be a bug fix. Since it could be classified either way, I think it's your call, @marmarek. Either way, I suppose there's no reason not to include it in the release notes.

Since this change is being included in Qubes OS 4.2.2, which is a patch release, I infer that the decision has been made to classify this issue as a bug rather than an enhancement. Updated labels and issue title accordingly.

[andrewdavidwong]

No, the idea is to make the "unsafe" variant the default. I use "unsafe" in quotes because the practical impact is rather small (exploiting this requires somebody finding another bug in a font rendering engine - those happen, but are very rare). On the other hand, as proven in several reports, hitting issues with copying files with for example Arabic names, users fallback to much less safe options (like packing into zip).

It sounds like you don't believe that it's truly unsafe. If you did, I don't believe you would be willing to make it the default or classify this change as a bug fix. In light of this, I believe that naming the new service allow-unsafe-characters is a mistake. Naming it "unsafe":

• Does not seem to accurately reflect the security impact of using the service.
• Does not seem to accurately reflect the Qubes core developers' opinion of the service.
• Will likely lead many users to believe that the service is truly unsafe.
• Will likely lead many users to seek the "safe" alternative, even if it would not benefit them and may even cause problems for them.
• May cause some users to question why the "unsafe" version is being made the default (rather than opt-in).
• May cause some users to have doubts about whether Qubes OS ships with secure defaults in general.
• May further encourage some users to tinker with their systems in ways they do not fully understand in an effort to improve upon the default level of security, which may break their systems and undermine their security in various ways.
• May cause some users to lose confidence in the way the Qubes OS Project uses words like "safe" and "unsafe." (They won't know what we mean, and they won't be able to trust that we're using such words in the same way they understand them.)
• Effectively "hard codes" a value judgment into the service that should instead be left for the user to decide. Different users have different priorities and risk tolerances. Several users in this thread have expressed a strong desire to use the new service. They probably don't really believe that what they want to do is unsafe for them. Qubes OS generally tries to avoids hard coding such value judgments. (E.g., the meanings of color labels are intentionally left to the user to define.)

I discussed this with @DemiMarie, who suggested that allow-all-bytes may be a more accurate name.

[DemiMarie]

I discussed this with @DemiMarie, who suggested that allow-all-bytes may be a more accurate name.

allow-all-names would be even better.