Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADO-2059 added Artifactory as a tool source #78

Closed
wants to merge 3 commits into from
Closed

Conversation

afujiwara-roblox
Copy link
Collaborator

@afujiwara-roblox afujiwara-roblox commented Sep 6, 2023

added Artifactory as a tool source

@github-actions
Copy link

github-actions bot commented Sep 6, 2023

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

src/config.rs Outdated Show resolved Hide resolved
src/config.rs Outdated
Comment on lines 71 to 76
let res = match self {
ToolSpec::Github { .. } => format!("github.com/{}@{}", self.source(), self.version()),
ToolSpec::Gitlab { .. } => format!("gitlab.com/{}@{}", self.source(), self.version()),
ToolSpec::Artifactory { .. } => format!("{}@{}", self.source(), self.version()),
};
write!(f, "{}", res)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, any rusty way to simplify it like before?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could inline the match like above, but this reads just fine to me

src/config.rs Outdated
Comment on lines 211 to 230
#[test]
fn artifactory_from_artifactory_field() {
let artifactory: ToolSpec = toml::from_str(
&[
r#"artifactory = "https://artifactory-edge1.rbx.com""#,
r#"path = "generic-rbx-local-tools/rotriever/""#,
r#"version = "0.5.4""#,
]
.join("\n"),
)
.unwrap();
assert_eq!(
artifactory,
new_artifactory(
"https://artifactory-edge1.rbx.com",
"generic-rbx-local-tools/rotriever/",
version("0.5.4"),
)
);
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any other tests we'd like to see?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to support leading and trailing backslashes in the artifactory and path sections such as

[tools.rotrieve]
artifactory = "https://artifactory-edge1.rbx.com/"
path = "/generic-rbx-local-tools/rotriever/"
version = "0.5.4"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure about this, it'd be nice to be flexible. I suspect when you join them together it doesn't matter too much if there's an extra slash, but it might also not matter at all if we use a Url class that has its own notions of "path" and "domain" and probably handles the rest for us?

@afujiwara-roblox afujiwara-roblox marked this pull request as ready for review September 6, 2023 16:54
@afujiwara-roblox afujiwara-roblox changed the title added Artifactory as a tool source ADO-2059 added Artifactory as a tool source Sep 6, 2023
src/config.rs Outdated
Comment on lines 40 to 47
source.to_owned()
}
ToolSpec::Artifactory {
artifactory: source,
path,
..
} => {
format!("{}/{}", source, path)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any rusty way to avoid making a new String? Maybe add a new field and move line 47 to when we initialize the ToolSpec::Artifactory?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I see your point; our definition of source gets more complicated with the artifactory definition, I almost wonder if we could separate source and path in the abstraction in general?

That way you'd have sources like: https://github.com, https://gitlab.com, https://somehost.artifactory.com, https://otherhost.artifactory.com, etc.

And then paths like: Roblox/foreman, path/to/artifact, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One way we could be thinking about this would be to have a separate definition that aliases artifactory servers to sources, and a toml that looks roughly like this:

[sources]
# implicit default sources are:
# github = "https://github.com"
# gitlab = "https://gitlab.com"
rbx_artifactory = "https://rbx.artifactory.com"

[tools]
# this tool is still on github
tarmac = { github = "Roblox/tarmac", version = "0.7.0" }
# this tool has been migrated
rotrieve = { rbx_artifactory = "generic-rbx-sage-infra/tools/rotriever", version = "0.5.12" }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative would be to have every tool spec be composed of source (more like host) + path + version, in which case we'd maybe want the existing stuff like rotrieve = { github = "roblox/rotriever", version = "0.5.12" } is just a shorthand for:

[tools.rotrieve]
source = "https://github.com"
path = "roblox/rotriever"
version = "0.5.12"

Copy link
Contributor

@ZoteTheMighty ZoteTheMighty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good start, I had some high level thoughts on how we can adapt tool specs to generalize around the artifactory case

src/config.rs Outdated
Comment on lines 71 to 76
let res = match self {
ToolSpec::Github { .. } => format!("github.com/{}@{}", self.source(), self.version()),
ToolSpec::Gitlab { .. } => format!("gitlab.com/{}@{}", self.source(), self.version()),
ToolSpec::Artifactory { .. } => format!("{}@{}", self.source(), self.version()),
};
write!(f, "{}", res)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could inline the match like above, but this reads just fine to me

src/config.rs Outdated
Comment on lines 40 to 47
source.to_owned()
}
ToolSpec::Artifactory {
artifactory: source,
path,
..
} => {
format!("{}/{}", source, path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I see your point; our definition of source gets more complicated with the artifactory definition, I almost wonder if we could separate source and path in the abstraction in general?

That way you'd have sources like: https://github.com, https://gitlab.com, https://somehost.artifactory.com, https://otherhost.artifactory.com, etc.

And then paths like: Roblox/foreman, path/to/artifact, etc.

src/config.rs Outdated
Comment on lines 40 to 47
source.to_owned()
}
ToolSpec::Artifactory {
artifactory: source,
path,
..
} => {
format!("{}/{}", source, path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One way we could be thinking about this would be to have a separate definition that aliases artifactory servers to sources, and a toml that looks roughly like this:

[sources]
# implicit default sources are:
# github = "https://github.com"
# gitlab = "https://gitlab.com"
rbx_artifactory = "https://rbx.artifactory.com"

[tools]
# this tool is still on github
tarmac = { github = "Roblox/tarmac", version = "0.7.0" }
# this tool has been migrated
rotrieve = { rbx_artifactory = "generic-rbx-sage-infra/tools/rotriever", version = "0.5.12" }

src/config.rs Outdated
Comment on lines 40 to 47
source.to_owned()
}
ToolSpec::Artifactory {
artifactory: source,
path,
..
} => {
format!("{}/{}", source, path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative would be to have every tool spec be composed of source (more like host) + path + version, in which case we'd maybe want the existing stuff like rotrieve = { github = "roblox/rotriever", version = "0.5.12" } is just a shorthand for:

[tools.rotrieve]
source = "https://github.com"
path = "roblox/rotriever"
version = "0.5.12"

Comment on lines +7 to +12
unable to parse Foreman configuration file (at {{CWD}}foreman.toml): data is not properly formatted for tool:

[tools.tool]
invalid = "roblox/tooling"
version = "0.0.0"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to change the snapshot. We lose line information since we're longer relying on Serde to deserialize the toml. I figured replacing the incorrect line with the incorrect tool is good enough

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's probably reasonable, especially given all the cases in which Serde's line annotation falls apart

@afujiwara-roblox
Copy link
Collaborator Author

afujiwara-roblox commented Sep 11, 2023

Closing to break up into smaller PRs

  1. Manual deserialization of existing sources (github & gitlab) LUAFDN-1756 Manual deserialization of toml into toolspec struct #79
  2. Support for hosts and support for artifactory

@github-actions github-actions bot locked and limited conversation to collaborators Sep 11, 2023
@afujiwara-roblox afujiwara-roblox deleted the ADO-2059 branch September 12, 2023 20:28
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants