When using upload protocol s3 on Red Hat incorrect upload_url_string is returned.

[TrevorBenson]

I was testing out sos collect and upload protocol s3 for release 4.8.1.

On the very first test the collection was uploaded to the expected S3 bucket when using --config-file

/etc/sos/custom.conf contents:

[global]
batch = yes
all-logs = no
log-size = 1

[collect]
encrypt = yes
encrypt-key = sosreport.example.com
upload-protocol = s3
upload-s3-endpoint = https://sosreport.example.com
upload-s3-bucket = projectb-sosreports
upload-s3-access-key = <OBFUSCATED_ACCESS_KEY>
upload-s3-secret-key = <OBFUSCATED_SECRET_KEY>

I didn't include a case ID, the archive was uploaded to the expected prefix under a UUID, year and month (defined at the time of upload): a75d5bb0-fd17-4422-8b1b-497310c74efd/2024/12/sos-collector-2024-12-20-yosca.tar.xz.gpg

However, I noticed that the following messages were being provided at the end, even if the archive was being uploaded to an alternative protocol.

No case id provided, uploading to SFTP
No case id provided, uploading to SFTP
Attempting upload to Red Hat Secure FTP

I don't believe that anything is being uploaded to SFTP as I have each collection within the defined bucket, but the RHELPolicy isn't considering this in its version of get_upload_url:

sos/sos/policies/distros/redhat.py

Lines 250 to 261 in a642018

	def get_upload_url(self):
	if self.upload_url:
	return self.upload_url
	if self.commons['cmdlineopts'].upload_url:
	return self.commons['cmdlineopts'].upload_url
	if self.commons['cmdlineopts'].upload_protocol == 'sftp':
	return RH_SFTP_HOST
	if not self.commons['cmdlineopts'].case_id:
	self.ui_log.info("No case id provided, uploading to SFTP")
	return RH_SFTP_HOST
	rh_case_api = "/support/v1/cases/%s/attachments"
	return RH_API_HOST + rh_case_api % self.case_id

[TrevorBenson]

The following appears to resolve the reporting of using SFTP for the upload when it is really being uploaded via s3 protocol into a bucket:

    def get_upload_url(self):
        if self.upload_url:
            return self.upload_url
        if self.commons['cmdlineopts'].upload_url:
            return self.commons['cmdlineopts'].upload_url
        if self.commons['cmdlineopts'].upload_protocol == 'sftp':
            return RH_SFTP_HOST
        if self.commons['cmdlineopts'].upload_protocol == 's3':
            bucket = self.get_upload_s3_bucket()
            if self.commons['cmdlineopts'].case_id:
                rh_case_api = "/support/v1/cases/%s/attachments"
                return f"s3://{bucket}" + rh_case_api % self.case_id
            else:
                prefix = self.get_upload_s3_object_prefix()
                return f"s3://{bucket}/{prefix}"
        if not self.commons['cmdlineopts'].case_id:
            self.ui_log.info("No case id provided, uploading to SFTP")
            return RH_SFTP_HOST
        rh_case_api = "/support/v1/cases/%s/attachments"
        return RH_API_HOST + rh_case_api % self.case_id

Example when uploading with an object prefix (a uuid plus year and month) to a bucket named project-sosreports:

Attempting upload to s3://project-sosreports/ba59c727-bfd2-4808-8f5f-f62c63903c13/2024/12
Uploaded archive successfully

When both a case id (eg. 12345) and an object prefix (same as above) are provided the case id wins out:

The following archive has been created. Please provide it to your support team.
	/var/tmp/sos-collector-12345-2024-12-23-hpvlk.tar.xz.gpg

Attempting upload to s3://project-sosreports/support/v1/cases/12345/attachments
Uploaded archive successfully

$ aws s3 ls --recursive s3://project-sosreports/support
2024-12-23 12:11:56   27130931 support/v1/cases/12345/attachments/sos-collector-12345-2024-12-23-hpvlk.tar.xz.gpg

While this resolves the erroneous message, I wanted to add it to this issue before opening any PR for review in case there was a different approach preferred, or potentially any work already underway in parallel it might conflict with.

[jcastill]

While this resolves the erroneous message, I wanted to add it to this issue before opening any PR for review in case there was a different approach preferred, or potentially any work already underway in parallel it might conflict with.

Nice fix! I'm working on the upload subsystem, but I can wait for your PR and then adapt what I've done so far to your changes.

[TrevorBenson]

While this resolves the erroneous message, I wanted to add it to this issue before opening any PR for review in case there was a different approach preferred, or potentially any work already underway in parallel it might conflict with.

Nice fix! I'm working on the upload subsystem, but I can wait for your PR and then adapt what I've done so far to your changes.

I reviewed more closely and decided since the string is a URL to check the contents for non s3. Since https://api.access.redhat.com/support/v1/cases/None/attachments was shown, I'll adjust accordingly to use an S3 Path Style URL which ensures the endpoint is shown along with the bucket and any prefix:

Attempting upload to https://s3.example.com/project-sosreports/support/v1/cases/12345/attachments

or

Attempting upload to https://s3.example.com/project-sosreports/ba59c727-bfd2-4808-8f5f-f62c63903c13/2024/12