RFC: generalize resource interface

024-artifact-resources/examples/git-v2/Dockerfile

@@ -0,0 +1,15 @@
+FROM alpine
+
+RUN apk add git ruby ruby-json ruby-bundler build-base ruby-dev cmake openssl-dev
+
+ENV GEM_HOME="/usr/local/bundle"
+ENV PATH $GEM_HOME/bin:$GEM_HOME/gems/bin:$PATH
+ADD Gemfile .
+RUN bundle install
+
+ADD git-resource.rb /usr/bin/git-resource
+RUN chmod +x /usr/bin/git-resource
+
+WORKDIR /tmp/request
+
+CMD ["git-resource", "info"]

024-artifact-resources/examples/git-v2/Gemfile

@@ -0,0 +1,4 @@
+source :rubygems
+
+gem 'rugged'
+gem 'pry'

024-artifact-resources/examples/git-v2/Gemfile.lock

@@ -0,0 +1,19 @@
+GEM
+ remote: http://rubygems.org/
+ specs:
+ coderay (1.1.2)
+ method_source (0.9.2)
+ pry (0.12.2)
+ coderay (~> 1.1.0)
+ method_source (~> 0.9.0)
+ rugged (0.28.1)
+
+PLATFORMS
+ ruby
+
+DEPENDENCIES
+ pry
+ rugged
+
+BUNDLED WITH
+ 1.16.2

024-artifact-resources/examples/git-v2/git-resource.rb

@@ -0,0 +1,115 @@
+#!/usr/bin/env ruby
+
+require "json"
+require "rugged"
+require "pry"
+require "benchmark"
+
+$request = JSON.parse(STDIN.read, symbolize_names: true)
+
+def commit_fragment(commit)
+ JSON.dump({
+ config: {ref: commit.oid},
+ metadata: [
+ {name: "author", value: enc(commit, commit.author[:name])},
+ {name: "committer", value: enc(commit, commit.committer[:name])},
+ {name: "message", value: enc(commit, commit.message)}
+ ]
+ })
+end
+
+def icon(uri)
+ case uri
+ when /github/
+ "github-circle"
+ when /gitlab/
+ "gitlab"
+ when /bitbucket/
+ "bitbucket"
+ else
+ "git"
+ end
+end
+
+def bench(label, &blk)
+ time = Benchmark.realtime(&blk)
+ $stderr.puts "#{label}: #{time}s"
+end
+
+def enc(commit, str)
+ str = str.force_encoding("ISO-8859-1") unless commit.header_field("Encoding")
+ str.encode("UTF-8")
+end
+
+case ARGV[0]
+when "info"
+ puts JSON.dump({
+ "interface_version": "2.0",
+ "icon": icon($request[:config][:uri]),
+ "actions": {
+ "check": "git-resource check",
+ "get": "git-resource get",
+ "put": "git-resource put"
+ # delete is unsupported
+ }
+ })
+
+when "check"
+ repo =
+ if File.exists?("HEAD")
+ Rugged::Repository.new(".").tap do |r|
+ r.fetch("origin")
+ end
+ else
+ Rugged::Repository.clone_at(
+ $request[:config][:uri],
+ ".",
+ checkout_branch: $request[:config][:branch],
+ bare: true,
+ progress: lambda { |t| $stderr.print t })
+ end
+
+ walker = Rugged::Walker.new(repo)
+ walker.sorting(Rugged::SORT_TOPO|Rugged::SORT_REVERSE)
+ walker.simplify_first_parent
+ walker.push(repo.head.target)
+
+ response = File.new $request[:response_path], 'w'
+ total_commits = 0
+
+ from = $request[:config][:ref]
+ if from && repo.include?(from)
+ commit = repo.lookup(from)
+ walker.hide(commit)
+
+ response.puts commit_fragment(commit)
+ total_commits += 1
+ end
+
+ bench("walk") do
+ walker.walk do |c|
+ response.puts commit_fragment(c)
+ total_commits += 1
+ end
+ end
+
+ $stderr.puts "commits: #{total_commits}"
+
+ response.close
+
+when "get"
+ repo =
+ Rugged::Repository.clone_at(
+ $request[:config][:uri],
+ ".",
+ checkout_branch: $request[:config][:branch])
+
+ repo.checkout($request[:config][:ref])
+
+ response = File.new $request[:response_path], 'w'
+ response.puts commit_fragment(repo.head.target)
+ response.close
+
+when "put"
+ puts "putting"
+end

024-artifact-resources/proposal.md

@@ -0,0 +1,57 @@
+# Artifact Resources
+
+This proposal uses the [generalized resource interface](../024-generalized-resources/proposal.md) to show how the interface would be implemented and interpreted by Concourse to support "artifact resources", today's primary use case for v1 resources.
+
+## Motivation
+
+* Support for creating multiple versions from `put`: [concourse/concourse#2660](https://github.com/concourse/concourse/issues/2660)
+
+* Support for deleting versions: [concourse/concourse#362](https://github.com/concourse/concourse/issues/362), [concourse/concourse#524](https://github.com/concourse/concourse/issues/524)
+
+* Make the `get` after `put` opt-in: [concourse/concourse#3299](https://github.com/concourse/concourse/issues/3299), [concourse/registry-image-resource#16](https://github.com/concourse/registry-image-resource/issues/16)
+
+## Examples
+
+* A basic implementation of the `git` resource can be found in [`git-v2`](examples/git-v2).
+
+## Proposal
+
+A v2 resource type can be used as a versioned artifact resource by treating the **config fragments** as **versions** and emitting them in chronological order from `check`. This way the resource type is used to model change in an external resource over time.
+
+### `check`: discover versions in order
+
+The `check` action will first be run with a "naked" config, containing only what the user specified. In this situation `check` must emit an `ActionResponse` for all versions discovered in the config, in chronological order.
+
+Subsequent calls to `check` will be given a config that has been spliced with the last emitted version config fragment. The `check` script must an `ActionResponse` for the given version if it still exists, followed by a response for any versions that came after it.
+
+If the specified version is no longer present, the `check` action must go back to returning all versions, as if the version was not specified in the first place. Concourse will detect this scenario by noticing that the first `ActionResponse` emitted does not match the requested version. All versions that existed before that were emitted will be automatically marked "deleted".
+
+The `check` action can use the **bits** directory to cache state between runs of the `check` on that worker. On the first run, the directory will be empty.
+
+### `get`: fetch a version of an artifact
+
+The `get` action will always be invoked with a spliced config specifying which version to fetch. It is given an empty **bits** directory in which to fetch the data.
+
+An `ActionResponse` must be emitted for all versions that have been fetched into the bits directory. Each version will be recorded as an input to the build.
+
+### `put`: idempotently create artifact versions
+
+The `put` action will be invoked with user-provided configuration and arbitrary bits.
+
+An `ActionResponse` must be emitted for all versions that have been created/updated. Each version will be recorded as an output of the build.
+
+### `delete`: idempotently destroy artifact versions
+
+The `delete` action will be invoked with user-provided configuration and arbitrary bits.
+
+An `ActionResponse` must be emitted for all versions that have been destroyed. These versions will be marked "deleted" and no longer be available for use in other builds.
+
+## Open Questions
+
+* [resource-determined triggerability of versions?](https://github.com/concourse/rfcs/issues/11)
+* webhooks?
+ * should these instead be something supported by *Concourse*?
+
+## Answered Questions
+
+* Version filtering is probably best left to `config`.