Document deployment of artifacts

[kwin]

This closes #1587

Following this checklist to help us incorporate your
contribution quickly and easily:

• Your pull request should address just one issue, without pulling in other changes.
• Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
• Each commit in the pull request should have a meaningful subject line and body.
  Note that commits might be squashed by a maintainer on merge.
• Write unit tests that match behavioral changes, where the tests fail if the changes to the runtime are not applied.
  This may not always be possible but is a best-practice.
• Run mvn verify to make sure basic checks pass.
  A more thorough check will be performed on your pull request automatically.
• You have run the integration tests successfully (mvn -Prun-its verify).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

• I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004
• In any other case, please file an Apache Individual Contributor License Agreement.

[cstamas]

The "Maven way" of wanted topic "deploy API using PUT by default for HTTP" is true IF following conditions are true:

• last connector in pipeline (mvn3 does not have it, but similar thing happens effectively, while mvn4 has dedicated API for this for programmatic construction) is basic connector
• transport being used is some of HTTP implementations (apache, jdk or jetty) -- usually transport selection happens based 0n RemoteRepository#getProtocol() (but as always, there are exceptions)

In that case, one can say that "deploy" call maps to HTTP PUT method for each artifact and metadata.

[kwin]

What about checksums? Are they treated as separate artifacts like signatures?

[cstamas]

No, checksums are feature of basic connector (check out M2 layout, that one contains the "configured" checksums), and they are opaquely (to client) being uploaded (on deploy) or being asked for (on consume) by basic connector. No client code should ever mangle with them (while deploying or resolving).

Though, as they are treated/cached in same was as artifacts and metadata, they are stored in local repository (on consume), and also, they can be explicitly resolved too by same calls as artifacts or metadata is resolved.

In short: checksums are NOT separate artifacts (even if they do behave like "separate artifacts" in some respect), but there are important differences. In general, you should NOT assume they are separate artifacts, you should leave their handling to basic connector: the role of basic connector among others, is that when it returns from a call with downloaded artifact, you KNOW the checksums are checked and are okay (according to session config like checksum policy and configured checksum algs), otherwise basic connector fails the call.

[cstamas]

And one more important point: if you DO provide checksums, basic connector will let them pass (and deploy in opaque manner) unless configured to use same checksum algorithm. Think ASF source bundle deploys, we use a plugin to create SHA512 for them, (while Resolver is using 'standard' checksum config of MD5 and SHA1). IF Resolver would be configured to use SHA512 as well (mvn3: -Daether.checksums.algorithms=... and mvn4: aether.layout.maven2.checksumAlgorithms), it would recalculate and replace checksum with its own on deploy.

[cstamas]

Also, important is the "kind" of checksum:

maven-resolver/maven-resolver-spi/src/main/java/org/eclipse/aether/spi/connector/checksum/ChecksumPolicy.java

Lines 65 to 93 in f737339

	/**
	* Enum denoting origin of checksum.
	*
	* @since 1.8.0
	*/
	enum ChecksumKind {
	/**
	* Remote external kind of checksum are retrieved from remote doing extra transport round-trip (usually by
	* getting "file.jar.sha1" for corresponding "file.jar" file). This kind of checksum is part of layout, and
	* was from beginning the "official" (and one and only) checksum used by resolver. If no external checksum
	* present, {@link #onNoMoreChecksums()} method is invoked that (by default) fails retrieval.
	*/
	REMOTE_EXTERNAL,
	/**
	* Included checksums may be received from remote repository during the retrieval of the main file, for example
	* from response headers in case of HTTP transport. They may be set with
	* {@link org.eclipse.aether.spi.connector.transport.GetTask#setChecksum(String, String)}. If no included
	* checksum present, {@link #REMOTE_EXTERNAL} is tried for.
	*/
	REMOTE_INCLUDED,
	/**
	* Provided checksums may be provided by {@link org.eclipse.aether.spi.checksums.ProvidedChecksumsSource}
	* components, ahead of artifact retrieval. If no provided checksum present, {@link #REMOTE_INCLUDED} is
	* tried for.
	*/
	PROVIDED
	}

[kwin]

The "Maven way" of wanted topic "deploy API using PUT by default for HTTP" is true IF following conditions are true:

• last connector in pipeline (mvn3 does not have it, but similar thing happens effectively, while mvn4 has dedicated API for this for programmatic construction) is basic connector
• transport being used is some of HTTP implementations (apache, jdk or jetty) -- usually transport selection happens based 0n RemoteRepository#getProtocol() (but as always, there are exceptions)

In that case, one can say that "deploy" call maps to HTTP PUT method for each artifact and metadata.

What exactly do you want me to change/add to the doc?

[cstamas]

Maybe just emphasize that "basic connector + HTTP transport" is the "most common way users are using Maven" (given Central is HTTP)? But, that is exactly NOT the typical how they deploy 😄

Unsure, what we want really to document here... as today, NOTHING aside of MRMs (that provide "internal" or "private" repositories, and usually proxies as well) support the "basic Maven deploy" (HTTP PUT artifact to remote repo) use case.

[kwin]

Unsure, what we want really to document here.

The way deploy is implemented by standard resolver services, obviously all those standard services can be overidden/extended in which case the picture may change.

as today, NOTHING aside of MRMs (that provide "internal" or "private" repositories, and usually proxies as well) support the "basic Maven deploy" (HTTP PUT artifact to remote repo) use case.

That is impossible to understand without this piece of information.

[cstamas]

Just for ref, the Slack discussion as well:

So, I don't know can we go with it so simple, as for example, new mvn4 feature is this (visible with -X):

[DEBUG] Final pipeline: offline( rrf( basic( apache-snapshots-b6ad1b6b4141cf7b4e128785f79cbc45945d1ec4 (https://repository.apache.org/content/groups/snapshots, default, snapshots) ) ) )

Maven 4 now "pipeline" connectors, and this is final assembled pipeline in resolver 2:

offline( rrf( basic( target-usually remote repo ) ) )

Roles:

• offline, if repo offline, "shortcuts" and refuse request
• rrf, based (and if present) RRF rules, bounce/refuse requests
• basic, is our "basic" transport, handles transport (Transport) and checksums (corruption protection)

Transport is implementing actual transport like HTTP, FTP, S3 etc. Also, we plan to add new connectors in pipeline, like sigcheck().

What this doco describe is true ONLY if basic connector is used (it implements the well known "roundtrip" maven repo, which focuses on immediate availability), but one could implement other connectors, like for example P2 is (gets stuff from P2 repo), or OBR and those formats are wildly different (btw, the connector/transport separation happened quite late, around maven 3.3.? As before connector+transport was one). For example, Njord is another connector as well (not transport, it in fact reuses file transport from resolver). Mimir is another connector, that implements read-through cache around basic.

Resolver is layered on purpose, and at the same time due this is not quite trivial to explain how it works.

Also, important distinction about "abstraction levels": connectors deals with Artifacts, while transport only about URIs. Transport is blind for "which artifact is this URI for"-like questions.

[cstamas]

@kwin is this good to go into 2.0.12? If so, pls merge

[kwin]

Good as a starting point, I would say. Can be extended later on.

[github-actions]

@kwin Please assign appropriate label to PR according to the type of change.

[github-actions]

@kwin The PR can't be associated to a milestone, because there are multiple open milestones. Please add the text "branch: master" to the description to the milestone where this PR belongs to.