Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a check for packages distributed via sourceforge to check if a mirror link is used in yaml file #673

Open
lychichem opened this issue Dec 29, 2020 · 6 comments
Labels
Issue-Feature This is a feature request for the Windows Package Manager client.

Comments

@lychichem
Copy link

lychichem commented Dec 29, 2020

Description of the new feature/enhancement

Add a check for packages distributed via sourceforge to check if a mirror link is used in yaml file

There are users reporting they cannot install a certain package due to a mirror link is used in yaml file, and he/she cannot connect to the mirror. (microsoft/winget-pkgs#5770 )
If we add a check for this situation and change the link to sourceforge main site link like what in microsoft/winget-pkgs#5772 , at least users can go to sourceforge main site to change which mirror server to use and therefor can finish the install.

Proposed technical implementation details (optional)

If a mirror link is used in the yaml file, a robot can edit it to use the sourceforge main site URL. Like in microsoft/winget-pkgs#5770 , the link can be changed from https://ayera.dl.sourceforge.net/project/xming/Xming/6.9.0.31/Xming-6-9-0-31-setup.exe to https://sourceforge.net/projects/xming/files/Xming/6.9.0.31/Xming-6-9-0-31-setup.exe/download.

@denelon
Copy link
Contributor

denelon commented Dec 29, 2020

This seems like more of a client feature that may also require some changes to the manifest.

@denelon denelon transferred this issue from microsoft/winget-pkgs Dec 29, 2020
@ghost ghost added the Needs-Triage Issue need to be triaged label Dec 29, 2020
@doctordns
Copy link

Great idea.
question: what is the support status of Sourceforge? Does MSFT support this repo officially?

@lychichem
Copy link
Author

lychichem commented Dec 30, 2020

This seems like more of a client feature that may also require some changes to the manifest.

I mean, if a PR uses a mirror link in yaml file, like https://ayera.dl.sourceforge.net/project/xming/Xming/6.9.0.31/Xming-6-9-0-31-setup.exe, a robot will automatically change the link to https://sourceforge.net/projects/xming/files/Xming/6.9.0.31/Xming-6-9-0-31-setup.exe/download, or note the contributor "Seems you used a mirror link. You should use direct link when submitting." I believe this is just related to the winget/pkgs repo. As for which mirror sourceforge really use, it relates to users' cookie on sourceforge but not this repo.

@megamorf
Copy link

megamorf commented Dec 31, 2020

I think it would make sense to have the bot automatically update installer URLs that tend to break easily as witnessed with the sourceforge mirrors. This would not only cut down on manual work by the community but also ensure a certain level of standardization.

Note: The original issue is microsoft/winget-pkgs#5787 with microsoft/winget-pkgs#5770 being a duplicate.

Edit: @lychichem is the URL scheme consistent for mirror and main sourceforge download links respectively? In that case a simple conversion could be done by the winget bot.

@lychichem
Copy link
Author

lychichem commented Dec 31, 2020

I think it would make sense to have the bot automatically update installer URLs that tend to break easily as witnessed with the sourceforge mirrors. This would not only cut down on manual work by the community but also ensure a certain level of standardization.

Note: The original issue is microsoft/winget-pkgs#5787 with microsoft/winget-pkgs#5770 being a duplicate.

Edit: @lychichem is the URL scheme consistent for mirror and main sourceforge download links respectively? In that case a simple conversion could be done by the winget bot.

I think a pattern may be written. You can see, the mirror is like this: https://<abbr. for the mirror>.dl.sourceforge.net/project/<project name,no capital letters>/<project name, capital letter can be used>/<version>/<installer name>, and the direct link is like this: https://sourceforge.net/projects/<project name, no capital letters>/files/<project name, capital letter can be used>/<version>/<installer name>/download. So a bot only need to delete <abbr. for the mirror>.dl, add s after project, and add files between two <project name>

@megamorf
Copy link

megamorf commented Dec 31, 2020

Okay, I've checked the sourceforge documentation for command line based downloads and it seems that there are two URL schemes that work reliably and people who submitted the manifests managed to pick the one and only URL that is not expected to work at all times.

# Status URL
1 Correct https://downloads.sourceforge.net/project/xming/Xming/6.9.0.31/Xming-6-9-0-31-setup.exe
2 Correct https://sourceforge.net/projects/xming/files/Xming/6.9.0.31/Xming-6-9-0-31-setup.exe/download?use_mirror=ayera (if the file is not on the mirror then another mirror that has the file will be chosen by sourceforge)
3 Wrong https://ayera.dl.sourceforge.net/project/xming/Xming/6.9.0.31/Xming-6-9-0-31-setup.exe

The first correct link type 1 will actually redirect to the link type 2 so it would make sense to always use 2. Using the use_mirror query parameter is optional as it seems to get appended automatically.

In that case a simple regular expression based replacement could be done based on following URL pattern:

https://${mirrorName}.dl.sourceforge.net/project/${projectName}/${appName}/${appVersion}/${package}

As regex that would be:

# Pattern:
https:\/\/(?<mirrorName>[\w-]+)\.dl\.sourceforge\.net\/project\/(?<projectName>[^/]+)\/(?<appName>[^/]+)\/(?<appVersion>[^/]+)\/(?<package>.+)

# Replacement:
https://sourceforge.net/projects/$2/files/$3/$4/$5/download

Here's a practical example: https://regexr.com/5jesj

@denelon denelon added Issue-Feature This is a feature request for the Windows Package Manager client. and removed Needs-Triage Issue need to be triaged labels Jan 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue-Feature This is a feature request for the Windows Package Manager client.
Projects
None yet
Development

No branches or pull requests

4 participants