debian software package does not install from url when version contains a colon character

[reubenmiller]

Describe the bug

The tedge-apt-plugin fails to install a software package from a URL (e.g. a package hosted in Cumulocity IoT) if the software version contains a colon in its name due to apt-get install <file> ignoring the <file> argument if a colon is present.

apt interprets the colon as the Epoch separator, so it seems that apt-get install silently ignores the given file and does not exit.

The root cause is the fact that tedge-apt-plugin downloads the debian file from the given software url, and save the file using the path built from the name and version, e.g. /tmp/{name}_{version}.deb, and the filename is not sanitized to be compatible with apt/apt-get.

There is a similar issue found in another project:

• Failed to install packages with colon on version via apt-get camicri/cube-get#10

Note

• You can't just rename the version field of the software package definition in Cumulocity IoT as tedge-apt-plugin does a meta information validation and while throw an error (MetaDataMismatch) if the given version from the cloud does not match the package's version

To Reproduce

1. Create a software package in Cumulocity for vim

   • Name: vim
   • Version: 2:8.2.2434-3+deb11u1
   • Url: Upload a package that matches your device's CPU architecture from the debian website

2. Install vim via the Cumulocity IoT Device Management Software page for the device under test

3. Inspect the software management logs on the device to see if the apt-get install command actually tried installing the package or not (see below for an example where it doesn't process the given package)

   ----- $ Downloading: http://127.0.0.1:8001/c8y/inventory/binaries/33714613 to /tmp/vim_2:8.2.2434-3+deb11u1
   ----- $ /usr/bin/sudo "/etc/tedge/sm-plugins/apt" "update-list"
   exit status: 0
   
   stdout <<EOF
   apt-get install args: ["install", "--quiet", "--yes", "/tmp/vim_2:8.2.2434-3+deb11u1.deb"]
   Reading package lists...
   Building dependency tree...
   Reading state information...
   0 upgraded, 0 newly installed, 0 to remove and 39 not upgraded.
   EOF
   
   stderr <<EOF
   EOF

Expected behavior

If the tedge-apt-plugin downloads the debian package from a given URL, then the temporary file created from the download should sanitize the version by substituting any unsupported characters.

Finding an example spec seems to be difficult, but this post might be helpful:

• https://askubuntu.com/questions/26008/filenames-in-the-apt-cache-directory

It might be enough to just replace : with the URL encoded %3a, but the results will have to be verified.

Screenshots

The following shows the output when trying to install a debian package from file with a colon in it (notice that there are no errors displayed):

# apt install '/tmp/vim_2:8.2.2434-3+deb11u1.deb'
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 39 not upgraded.

Renaming the file to replace the : with %3a, is enough for apt-get to process the package (though the install is still failing, however this is just because the dependencies aren't installed, and installing from file does not pull in dependencies automatically.)

# mv /tmp/vim_2:8.2.2434-3+deb11u1.deb /tmp/vim_2%3a8.2.2434-3+deb11u1.deb
# apt-get install /tmp/vim_2%3a8.2.2434-3+deb11u1.deb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'vim' instead of '/tmp/vim_2%3a8.2.2434-3+deb11u1.deb'
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
vim : Depends: vim-common (= 2:8.2.2434-3+deb11u1) but 2:9.0.1378-2 is to be installed
    Depends: vim-runtime (= 2:8.2.2434-3+deb11u1) but 2:9.0.1378-2 is to be installed
E: Unable to correct problems, you have held broken packages.

Environment (please complete the following information):

Property	Value
OS [incl. version]	Debian GNU/Linux 12 (bookworm)
Hardware [incl. revision]	Raspberry Pi 5 Model B Rev 1.0
System-Architecture	Linux rpi5-d83addab8e9f 6.1.0-rpi4-rpi-2712 #1 SMP PREEMPT Debian 1:6.1.54-1+rpt2 (2023-10-05) aarch64 GNU/Linux
thin-edge.io version	tedge 1.0.0-rc.2~297+geb93afc

Additional context

[reubenmiller]

@Ruadhri17 We could live with a naive implementation for now rather than doing a full url encoding:

file

crates/core/plugin_sm/src/plugin.rs

fn sm_path(name: &str, version: &Option<String>, target_dir_path: impl AsRef<Path>) -> PathBuf {
    let mut filename = name.to_string();
    if let Some(version) = version {
        filename.push('_');
        filename.push_str(version.as_str());
    }

    target_dir_path.as_ref().join(sanitize_filename(&filename))
}

fn sanitize_filename(filename: &String) -> String {
    // Replace special characters
    // see https://github.com/thin-edge/thin-edge.io/issues/2666
    filename.replace(":", "__")
}

[reubenmiller]

Implemented in #2683

See the system test included in the PR. A dummy package was included in the tests to test the exact scenario highlighted in this test:

• tests/RobotFramework/tests/plugin_apt/install_package_with_epoch.robot

[gligorisaev]

QA has thoroughly checked the bug and here are the results:

• Test for ticket exists in the test suite.
• tests/RobotFramework/tests/plugin_apt/install_package_with_epoch.robot
• QA has tested the bug and it's not reproducable anymore