Skip to content

Commit

Permalink
fix #4; add support for custom object metadata
Browse files Browse the repository at this point in the history
file-maps now can contain another key: `:metadata`.
This key can contain S3 metadata specified via kebab-cased
keywords, eg. `:content-encoding`.

More information about supported metadata can be found here:
http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html
  • Loading branch information
martinklepsch committed Dec 2, 2015
1 parent 8bac7d9 commit 9bf9209
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 4 deletions.
2 changes: 2 additions & 0 deletions build.boot
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
(set-env!
:source-paths #{"src"}
:dependencies '[[adzerk/bootlaces "0.1.11" :scope "test"]
[org.clojure/clojure "1.7.0"]
[com.novemberain/pantomime "2.7.0"]
[amazonica/amazonica "0.3.33"]
[digest "1.4.4"]])

Expand Down
20 changes: 16 additions & 4 deletions src/confetti/s3_deploy.clj
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
[clojure.data :as data]
[clojure.string :as string]
[digest :as dig]
[pantomime.mime :as panto]
[amazonica.aws.cloudfront :as cf]
[amazonica.aws.s3 :as s3]))

Expand Down Expand Up @@ -83,11 +84,21 @@
(into (mapv fm->op file-maps)
(mapv rm->op (:removed deduped)))))

(defn upload [creds bucket key file metadata]
(with-open [in (io/input-stream file)]
(let [base-meta {:content-type (panto/mime-type-of file)
:content-length (.length file)}]
(s3/put-object creds bucket key in (merge base-meta metadata)))))

(defn sync!
"Sync files described by `file-maps` to S3 `bucket-name`.
The file-maps collection may be odered in which case S3
The file-maps collection may be ordered in which case S3
operations will be executed in the same order.
`file-maps`need to have the following keys `:s3-key` & `:file`.
Optionally a `:metadata` key can be supplied to add custom
metadata to the uploaded S3 object.
Recognized options:
- `report-fn` takes 2 arguments (type, data) will be called
for each added and changed file being uploaded to S3.
Expand All @@ -99,19 +110,20 @@
([cred bucket-name file-maps {:keys [report-fn prune? dry-run?]}]
(validate-creds! cred)
(let [report* (or report-fn (fn [_]))
upload* (partial upload cred bucket-name)
objs (get-bucket-objects cred bucket-name)
ops (calculate-ops objs file-maps)]
(reduce (fn [stats {:keys [s3-key file] :as op}]
(reduce (fn [stats {:keys [s3-key file metadata] :as op}]
(cond (= ::upload (:op op))
(do (report* op)
(when-not dry-run?
(s3/put-object cred bucket-name s3-key file))
(upload* s3-key file metadata))
(update stats :uploaded conj s3-key))

(= ::update (:op op))
(do (report* op)
(when-not dry-run?
(s3/put-object cred bucket-name s3-key file))
(upload* s3-key file metadata))
(update stats :updated conj s3-key))

(= ::delete (:op op))
Expand Down

0 comments on commit 9bf9209

Please sign in to comment.