Support for Bintray #120

ghost · 2018-05-15T16:54:44Z

Hi,

This solution is pretty cool. Setting up Amazon S3 looks scary though. Something like bintray might be cool?

https://bintray.com/

tmspzz · 2018-05-15T18:48:28Z

I'll look into it!

In the mean time you can also use https://www.minio.io/

Take a look at #118

tmspzz · 2018-05-16T21:11:07Z

@rob-nash I took a look at Bintray and I have few questions I could not find an answer to.

Bintray's Authentication requires Username/APIKey. Is there a standard location to store these? like ~/.bintray/credentials
Bintray's repos are namespaced with :subject.

GET https://dl.bintray.com/:subject/:repo/:file_path
GET https://:subject.bintray.com/:repo/:file_path // Enterprise only

I guess this value should be configurable rather than just take the username of the current user?

I don't have access to the enterprise version so I can't implement anything that requires that version.

ghost · 2018-05-17T07:38:09Z

Hi there.

I am new to Bintray myself. I setup an open source account yesterday and uploaded some binaries via their API. Seems pretty good.

Open source accounts are free. I think I need to accept a EULA before I attempt my first download of one of these binaries. I'll get back to you on that.

Facebook v0.3.0 is 18Mb so my Carthage bootstrap is pretty slow! This is a snapshot of an API call I built with postman.

The body contains a zip of all the files built by Carthage.

baseURL = https://api.bintray.com

Authentication is basic. So base64 encode the following.

username:APIKey

example

rob-nash:57fexamplekey

A checksum of the file you're uploaded is probably worth doing. Can achieve like so

openssl dgst -sha256 /full/path/to/file

Maybe just support open source accounts, for now?

ghost · 2018-05-17T08:18:37Z

If I attempt to download one of these binaries myself with

https://dl.bintray.com/rob-nash/BinaryDump/iOS/dequable.zip

I don't need to accept a EULA.

The repo is public so I think someone attempting to download this without a Binary account, might get an error code and must accept a EULA, beforehand.

It might be easier to mention in README

create a free account
accept EULA
use Rome

tmspzz · 2018-05-17T08:32:52Z

Hi,

The cache structure that Rome uses is not quite what you have there, so I'll explain to understand if this is fine also for Bintray.

.dSYM, .framework, .version, .bsymbolmap are handled separately. They are zipped separately and stored at separate locations. Locations where artifacts are stored follow some conventions derived from the Cartfile.resolved.

See https://github.com/blender/Rome#cache-structure

For example, dequable would be stored at:

https://dl.bintray.com/rob-nash/Dequable/iOS/Dequable.framework.1.3.0.zip
https://dl.bintray.com/rob-nash/Dequable/iOS/Dequable.framework.dSYM.1.3.0.zip
https://dl.bintray.com/rob-nash/Dequable/Dequable.version.1.3.0.zip

If I break this down in Bintray terms this would be

https://dl.bintray.com/:subject/:repo    /:file_path
https://dl.bintray.com/rob-nash/Dequable/iOS/Dequable.framework.1.3.0.zip

I'm trying to understand if this is legal with the API or not.

ghost · 2018-05-17T08:39:52Z

The 'file_path' parameter can specify any path. So for instance, I currently have one file located under /iOS

But I guess there is no harm is building any file structure you want, inside the bintray repo.

Could we not just create one file named Dequable.zip which contains all 3?

ghost · 2018-05-17T08:43:28Z

The following would work.

https://api.bintray.com/content/rob-nash/BinaryDump/iOS/Dequable.framework.1.3.0.zip
https://api.bintray.com/content/rob-nash/BinaryDump/iOS/Dequable.framework.dSym.1.3.0.zip
https://api.bintray.com/content/rob-nash/BinaryDump/iOS/Dequable.version.1.3.0.zip

tmspzz · 2018-05-17T08:47:50Z

Cool! I'll play some more around and see that I can do.

The reasons for not having one zip are:

Have the version file accessible ahead of downloading the binary (the version file is like a manifest)
Allow upload/download of resources separately
Don't have to replace the whole zip if one file is missing

tmspzz · 2018-05-17T20:31:51Z

@rob-nash When you uploaded your binaries, what did you use as version in the api?

In my idea this would be the version of the library you're trying to upload as stated in the Cartfile.resolved .

What is the point of version ? I don't see how it's used in the download API.

ghost · 2018-05-17T21:44:31Z

For Uploading

Set a header in your request

X-Bintray-Version

For Downloading

In the above example Dequable is a package.

Get package will give you the available versions.

Get Version Files returns the files found for a given version.

You will be able to extract a path from these responses.

tmspzz · 2018-05-17T22:18:56Z

So what version is at https://dl.bintray.com/rob-nash/BinaryDump/iOS/dequable.zip ? Where is the package information here?

I find this service very confusing in general, or at least confusing to use given how Rome works.

Rome has to go from a Cartfile.resolved like this

github "Alamofire/Alamofire" "4.7.2"
git "ssh://git@stash.someserver.com:7999/iossdk/ios-rest.git" "v7.1.0"
binary "https://raw.githubusercontent.com/Building42/Specs/master/Carthage/Answers.json" "1.3.6"

to upload/download locations in Bintray

The upload API is:

PUT /content/:subject/:repo/:package/:version/:file_path

:subject will be something configurable but :repo and :package have to be derivable from the Cartfile.resolved

How would Rome derive :repo and :package from the above Cartfile.resolved for all 3 cases?

Note that ios-rest is not the final name of the framework binary, this is why there is a https://github.com/blender/Rome#repositorymap section

ghost · 2018-05-18T07:18:43Z

You have a lot of work and Bintray looks like quite a bit of effort. Let me see if I can build a small wrapper for Bintray, deployable as a cross-platform executable.

It might take me a while, so you might want to close this ticket for now.

If I am successful, I will re-open this ticket, so you can take a look at my work and assess if it is suitable for Rome.

What you think?

ghost · 2018-05-18T08:20:14Z

Just had a look at the API.

So what version is at https://dl.bintray.com/rob-nash/BinaryDump/iOS/dequable.zip ? Where is the package information here?

My binary profile currently has Dequable 1.3.0

When I attempt to upload 1.2.0 with the same :file_path and the following headers, I receive a 409 conflict error code and a message.

{
"message": "Unable to upload files: An artifact with the path 'iOS/dequable.zip' already exists under another version"
}

Navigating the website, I can see that the file directory responds to version filters. So it's important to realise two things here.

The metadata must include the version number
The file path must be unique.

So going forward, a naming convention is required. Such as

/content/rob-nash/BinaryDump/iOS/dequable_1.2.0.zip
/content/rob-nash/BinaryDump/iOS/dequable_1.3.0.zip

:subject will be something configurable but :repo and :package have to be derivable from the Cartfile.resolved. How would Rome derive :repo and :package from the above Cartfile.resolved for all 3 cases?

:subject = rob-dash
Agreed, this must be configurable.

:repo = BinaryDump
One caveat to using this service, is that you need to create and name a repo. There can be many repos. The onus should be on the user, in my opinion, to create one and provide the name to Rome. So, this also has to be configurable.

:package = Dequable
This can be extracted from Cartfile.resolved. In your above example, it would be Alamofire.

Note that ios-rest is not the final name of the framework binary, this is why there is a https://github.com/blender/Rome#repositorymap section

So we can supply the following paths

iOS/Dequable.framework.1.3.0.zip
iOS/Dequable.framework.dSym.1.3.0.zip
Dequable.version.1.3.0.zip

Get Version Files will return information about all of these files.

ghost · 2018-05-18T10:31:41Z

Looks like there is already a wrapper 👍

Phew!

https://www.npmjs.com/package/bintray

https://github.com/h2non/node-bintray

tmspzz · 2018-05-18T13:31:49Z

If I understand correctly your idea is to have 1 :repo where each dependency in the Cartfile.resolved is a :package.

So, in practice, given the following Cartfile.resolved

github "Alamofire/Alamofire" "4.7.2"
git "ssh://git@stash.someserver.com:7999/iossdk/ios-rest.git" "v7.1.0"
binary "https://raw.githubusercontent.com/Building42/Specs/master/Carthage/Answers.json" "1.3.6"

where ios-rest.git resolves to 2 targets Rest.framework and Helper.framework

It would translate to (omitting bcsymbolmaps):

PUT /content/:subject   /:repo    /:package  /:version/:file_path
PUT /content/rob-nash/BinaryDump/Alamofire/   4.7.2   /Alamofire/iOS/Alamofire.framework.1.3.0.zip
PUT /content/rob-nash/BinaryDump/Alamofire/   4.7.2   /Alamofire/iOS/Alamofire.framework.1.3.0.dSYM.zip
PUT /content/rob-nash/BinaryDump/Alamofire/   4.7.2   /Alamofire/Alamofire.version.1.3.0.zip

PUT /content/:subject   /:repo  /:package  /:version /:file_path
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/iOS/Rest.framework.7.1.0.zip
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/iOS/Rest.framework.7.1.0.dSYM.zip
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/ios-rest.version.1.3.0.zip
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/iOS/Helper.framework.7.1.0.zip
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/iOS/Helper.framework.7.1.0.dSYM.zip

PUT /content/:subject   /:repo  /:package  /:version   /:file_path
PUT /content/rob-nash/BinaryDump/Answers/   1.3.6      /Answers/iOS/Alamofire.framework.1.3.0.zip
PUT /content/rob-nash/BinaryDump/Answers/   1.3.6      /Answers/iOS/Alamofire.framework.1.3.0.dSYM.zip
PUT /content/rob-nash/BinaryDump/Answers/   1.3.6      /Answers/Answers.version.1.3.0.zip

So, puts are solved.

How about Gets?

Should the flow be:

get package
get version
get final URL ?

ghost · 2018-05-19T20:17:33Z

Hi. Sorry for delay.

Yes, I believe it should be as you say. Perform get 1, perform get 2, then build URL.

ghost · 2018-05-19T20:20:22Z

It looks like the API is still released as version 1, even after several years of it being in the public domain.

You may want to use that specific version, to be safe.

https://bintray.com/api/v1

tmspzz · 2019-05-18T19:17:59Z

@rob-nash Thanks to @BalestraPatrick you can now use whatever backend you want

tmspzz · 2019-05-18T19:36:19Z

You can try the pre-release at: https://github.com/blender/Rome/releases/tag/v0.22.0.59

tmspzz added the enhancement label May 16, 2018

ghost mentioned this issue May 19, 2018

Bintray Version h2non/node-bintray#6

Closed

tmspzz added the help wanted label Aug 6, 2018

tmspzz mentioned this issue Aug 9, 2018

Support Nexus repository #139

Closed

tmspzz mentioned this issue Aug 27, 2018

Add custom script engines for donwloading/uplaod/listing #148

Closed

tmspzz closed this as completed May 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for Bintray #120

Support for Bintray #120

ghost commented May 15, 2018

tmspzz commented May 15, 2018

tmspzz commented May 16, 2018

ghost commented May 17, 2018 •

edited by ghost

Loading

ghost commented May 17, 2018

tmspzz commented May 17, 2018

ghost commented May 17, 2018

ghost commented May 17, 2018

tmspzz commented May 17, 2018

tmspzz commented May 17, 2018 •

edited

Loading

ghost commented May 17, 2018 •

edited by ghost

Loading

tmspzz commented May 17, 2018

ghost commented May 18, 2018

ghost commented May 18, 2018 •

edited by ghost

Loading

ghost commented May 18, 2018 •

edited by ghost

Loading

tmspzz commented May 18, 2018 •

edited

Loading

ghost commented May 19, 2018

ghost commented May 19, 2018

tmspzz commented May 18, 2019

tmspzz commented May 18, 2019

Support for Bintray #120

Support for Bintray #120

Comments

ghost commented May 15, 2018

tmspzz commented May 15, 2018

tmspzz commented May 16, 2018

ghost commented May 17, 2018 • edited by ghost Loading

ghost commented May 17, 2018

tmspzz commented May 17, 2018

ghost commented May 17, 2018

ghost commented May 17, 2018

tmspzz commented May 17, 2018

tmspzz commented May 17, 2018 • edited Loading

ghost commented May 17, 2018 • edited by ghost Loading

tmspzz commented May 17, 2018

ghost commented May 18, 2018

ghost commented May 18, 2018 • edited by ghost Loading

ghost commented May 18, 2018 • edited by ghost Loading

tmspzz commented May 18, 2018 • edited Loading

ghost commented May 19, 2018

ghost commented May 19, 2018

tmspzz commented May 18, 2019

tmspzz commented May 18, 2019

ghost commented May 17, 2018 •

edited by ghost

Loading

tmspzz commented May 17, 2018 •

edited

Loading

ghost commented May 17, 2018 •

edited by ghost

Loading

ghost commented May 18, 2018 •

edited by ghost

Loading

ghost commented May 18, 2018 •

edited by ghost

Loading

tmspzz commented May 18, 2018 •

edited

Loading