Skip to content

Commit

Permalink
improve README; cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
@jackdbd
jackdbd committed Oct 2, 2024
1 parent 53ad304 commit 34403ff
Show file tree
Hide file tree
Showing 7 changed files with 111 additions and 182 deletions.
9 changes: 6 additions & 3 deletions .github/workflows/ci-cd.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,7 @@ jobs:
mv windows-latest-x86_64/${{ env.POD_NAME }}-${{ env.POD_VERSION }}-windows-latest-x86_64.zip .
# Troubleshooting: let's see if we have every assets we want to include in the GitHub release
- run: ls -R
# - run: ls -R

# https://github.com/marketplace/actions/gh-release
# See also this workflow that uses the bruceadams/get-release action:
Expand Down Expand Up @@ -455,11 +455,14 @@ jobs:
echo "UBUNTU_LATEST_x86_64_ZIP_URL is $UBUNTU_LATEST_x86_64_ZIP_URL"
echo "WINDOWS_LATEST_x86_64_ZIP_URL is $WINDOWS_LATEST_x86_64_ZIP_URL"
bb bb/manifest.bb --license MIT --pod-id com.github.jackdbd/pod.jackdbd.jsoup --version ${{ env.POD_VERSION }} --uberjar $UBERJAR_URL --linux-x86_64 $UBUNTU_LATEST_x86_64_ZIP_URL --macos-aarch64 $MACOS_LATEST_AARCH64_ZIP_URL --windows-x86_64 $WINDOWS_LATEST_x86_64_ZIP_URL
bb bb/manifest.bb --version ${{ env.POD_VERSION }} --uberjar $UBERJAR_URL --linux-x86_64 $UBUNTU_LATEST_x86_64_ZIP_URL --macos-aarch64 $MACOS_LATEST_AARCH64_ZIP_URL --windows-x86_64 $WINDOWS_LATEST_x86_64_ZIP_URL
echo "Here is the manifest.edn generated"
cat manifest.edn
- name: ⬆️ Upload manifest.edn (TODO Download it and make a PR to Babashka pod registry?)
# With this step the manifest.edn can be downloaded from the GitHub
# workflow. Then you will need to "register the pod", i.e. open a PR on
# the Babashka pod registry and commit your manifest.edn there.
- name: ⬆️ Upload manifest.edn
uses: actions/upload-artifact@v4
with:
name: manifest-edn
Expand Down
78 changes: 52 additions & 26 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,38 +7,50 @@ Babashka pod for parsing HTML with [jsoup](https://jsoup.org/).

## How to use it?

See [examples/jsoup.bb](./examples/jsoup.bb).
If you want to load a version of this pod that was *registered* (see below) on the [Pod registry](https://github.com/babashka/pod-registry), you can do it using the **qualified keyword** of this pod.

```clj
(require '[babashka.pods :as pods])
(pods/load-pod 'com.github.jackdbd/pod.jackdbd.jsoup "0.1.2")
```

Whether the pod is *registered* or not, you can always load it with this method:

1. Download the archive (e.g. `.zip`) containing the pod binary for your OS/architecture from a [GitHub release](https://github.com/jackdbd/pod-jackdbd-jsoup/releases).
1. Extract the binary from the archive (e.g. `unzip` it).
1. Move the binary to your [`BABASHKA_PODS_DIR`](https://github.com/babashka/pods?tab=readme-ov-file#where-does-the-pod-come-from).
1. Load the pod using its **file name**.

```clj
(require '[babashka.pods :as pods])
(pods/load-pod "pod-jackdbd-jsoup")
```


See also [examples/jsoup.bb](./examples/jsoup.bb).

## Development

The developer environment for this project is declared using [devenv](https://github.com/cachix/devenv).
The file [`devenv.nix`](./devenv.nix) declares a developer environment for this project. This file is used by [devenv](https://github.com/cachix/devenv) to create such environment. If you don't use devenv you can ignore this file, or use it to understand which dependencies are required by this project.

This project is managed with [neil](https://github.com/babashka/neil) and [Babashka tasks](https://book.babashka.org/#tasks). You can use `bb tasks` to view all available tasks.
This project uses a [`bb.edn`](./bb.edn) file to define a few [Babashka tasks](https://book.babashka.org/#tasks). You can type `bb tasks` to view them.

### Linux binary

If you are on Linux, you can compile a statically-linked binary using `bb build:binary`.
Double check that the binary is statically linked.
If you are on Linux, you can compile a statically-linked binary using this Babashka task.

```sh
ldd target/pod-jackdbd-jsoup
objdump --dynamic-syms target/pod-jackdbd
bb build:binary
```

### Upgrade version

Use `neil` to update the version in `deps.edn`. Here are a few examples:
You can double check that the binary is statically linked using one of the following commands.

```sh
neil version set 0.1.0
neil version patch
neil version minor
ldd target/pod-jackdbd-jsoup
objdump --dynamic-syms target/pod-jackdbd
```

A few things to keep in mind about `neil version`:

- it creates a Git commit and tag (this can be bypassed with `--no-tag`)
- it requires the working directory to be clean (this can be bypassed with `--force`)
### Upgrade version

At the moment `neil version` creates a git commit but fails to create a git tag. You can use the following commands instead.

Expand All @@ -47,17 +59,31 @@ bb bump:patch
bb tag
```

These commands bump the patch version and create an annotated tag.
These commands bump the patch version and create an annotated git tag.

### Create `manifest.edn` and register the pod on Pod registry

The [CI/CD pipeline](./.github/workflows/ci-cd.yaml) takes care of creating a GitHub release with compilation artifacts for Linux, macOS and Windows.

### Create manifest.edn
After all compilation artifacts for one pod version have been uploaded to the GitHub release associated with that version, that pod version can be *registered* in the [Pod registry](https://github.com/babashka/pod-registry).

> [!IMPORTANT]
> Every pod version has its own `manifest.edn`.
Currently, registering a pod involves two manual steps.

First, create a `manifest.edn` file for one version of the pod.

```sh
VERSION=0.1.4 && \
bb bb/manifest.bb \
--license MIT \
--pod-id "com.github.jackdbd/pod.jackdbd.jsoup" \
--version 0.1.2 \
--uberjar "https://github.com/jackdbd/pod-jackdbd-jsoup/releases/download/v0.1.2/pod-jackdbd-jsoup-0.1.2-ubuntu-latest-x86_64.zip" \
--linux-x86_64 "https://github.com/jackdbd/pod-jackdbd-jsoup/releases/download/v0.1.2/pod-jackdbd-jsoup-0.1.2-ubuntu-latest-x86_64.zip" \
--macos-aarch64 "https://github.com/jackdbd/pod-jackdbd-jsoup/releases/download/v0.1.2/pod-jackdbd-jsoup-0.1.2-macos-latest-aarch64.zip" \
--windows-x86_64 "https://github.com/jackdbd/pod-jackdbd-jsoup/releases/download/v0.1.2/pod-jackdbd-jsoup-0.1.2-windows-latest-x86_64.zip"
--version $VERSION \
--uberjar "https://github.com/jackdbd/pod-jackdbd-jsoup/releases/download/v$VERSION/pod-jackdbd-jsoup-$VERSION-ubuntu-latest-x86_64.zip" \
--linux-x86_64 "https://github.com/jackdbd/pod-jackdbd-jsoup/releases/download/v$VERSION/pod-jackdbd-jsoup-$VERSION-ubuntu-latest-x86_64.zip" \
--macos-aarch64 "https://github.com/jackdbd/pod-jackdbd-jsoup/releases/download/v$VERSION/pod-jackdbd-jsoup-$VERSION-macos-latest-aarch64.zip" \
--windows-x86_64 "https://github.com/jackdbd/pod-jackdbd-jsoup/releases/download/v$VERSION/pod-jackdbd-jsoup-$VERSION-windows-latest-x86_64.zip"
```

Then, make a PR on Pod registry following [these instructions](https://github.com/babashka/pod-registry?tab=readme-ov-file#registering-a-pod).

Once the PR on [Pod registry](https://github.com/babashka/pod-registry) gets merged, that pod version will be considered *registered* and users will be able to load it using the **qualified keyword** for the pod and the desired **version**.
2 changes: 1 addition & 1 deletion bb.edn
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
jar-path (format "target/%s-%s.jar" pod-name pod-version)]
(shell "jar --file" jar-path "--list"))}

list:uberjar
list:uber
{:doc "List the contents of the uberjar"
:depends [build:uber]
:task (let [project (-> (edn/read-string (slurp "deps.edn")) :aliases :neil :project)
Expand Down
63 changes: 44 additions & 19 deletions bb/manifest.bb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,14 @@
(ns manifest
(:require
[babashka.cli :as cli]
[utils :refer [format-edn]]
[clojure.string :as str]))
[clojure.edn :as edn]
[clojure.pprint :refer [pprint]]
[clojure.string :as str]
[utils :refer [format-edn]]))

(def project (-> (edn/read-string (slurp "deps.edn")) :aliases :neil :project))
(def default-version (:version project))
(def default-name (:name project))

;; Show help with `bb bb/manifest.bb --help` (or with -h, :help, :h)
(defn show-help
Expand All @@ -12,16 +18,26 @@

(def cli-spec
{:spec
{:license {:desc "Pod license (e.g. MIT)"
:require true}
:pod-id {:desc "Pod ID (e.g. com.github.jackdbd/pod.jackdbd.jsoup)"
:require true}
:linux-x86_64 {:desc "URL where the x86_64 Linux binary is hosted (GitHub release)"}
:macos-aarch64 {:desc "URL where the AArch64 macOS binary is hosted (GitHub release)"}
:windows-x86_64 {:desc "URL where the x86_64 Windows binary is hosted (GitHub release)"}
:uberjar {:desc "URL where the uberjar is hosted (GitHub release)"}
:version {:desc "Pod version (e.g. 1.2.3)"
:require true}}
{:license {:desc "Project license (e.g. MIT)"
:default "MIT"}
:description {:desc "Project description"
:alias :d
:default "Babashka pod for parsing HTML with jsoup."}
:example {:desc "Example that shows how to use the pod"
:alias :e
:default "examples/jsoup.bb"}
:name {:desc "Namespaced identifier of this project <group-id>/<pod-id> (e.g. com.github.jackdbd/pod.jackdbd.jsoup)"
:alias :n
:default default-name}
:language {:desc "The programming language the pod is written in"
:default "clojure"}
:linux-x86_64 {:desc "URL where the x86_64 Linux binary is hosted (e.g. URL of a GitHub release asset)"}
:macos-aarch64 {:desc "URL where the AArch64 macOS binary is hosted (e.g. URL of a GitHub release asset)"}
:windows-amd64 {:desc "URL where the AMD64 Windows binary is hosted (e.g. URL of a GitHub release asset)"}
:windows-x86_64 {:desc "URL where the x86_64 Windows binary is hosted (e.g. URL of a GitHub release asset)"}
:uberjar {:desc "URL where the uberjar is hosted (e.g. URL of a GitHub release asset)"}
:version {:desc "Project version (e.g. 1.2.3)"
:default default-version}}
:error-fn
(fn [{:keys [spec type cause msg option opts] :as data}]
(when (and (= :org.babashka/cli type) (not (or (:help opts) (:h opts))))
Expand All @@ -30,15 +46,19 @@
(println (format "Missing required argument: %s\n" option))
(System/exit 1))
:validate (do (println (format "%s does not exist!\n" msg))
(System/exit 1)))))})
(System/exit 1))
(do (println (str "Unsupported error cause: " cause))
(println "\nCLI spec:")
(pprint spec)
(System/exit 1)))))})

(defn -main
[args]
(let [opts (cli/parse-opts args cli-spec)]
(when (or (:help opts) (:h opts))
(println (show-help cli-spec))
(System/exit 0))
(let [pod-id (:pod-id opts)
(let [pod-id (:name opts)
pod-name (str/replace pod-id #"\." "-")
artifacts (cond-> []
(:linux-x86_64 opts) (conj {:os/arch "x86_64"
Expand All @@ -49,21 +69,26 @@
:os/name "Mac.*"
:artifact/url (:macos-aarch64 opts)
:artifact/executable pod-name})
(:windows-amd64 opts) (conj {:os/arch "amd64"
:os/name "Windows.*"
:artifact/url (:windows-amd64 opts)
:artifact/executable (format "%s.exe" pod-name)})
(:windows-x86_64 opts) (conj {:os/arch "x86_64"
:os/name "Windows.*"
:artifact/url (:windows-x86_64 opts)
:artifact/executable (format "%s.exe" pod-name)})
(:uberjar opts) (conj {:artifact/url (:uberjar opts)}))
manifest-data {:pod/name pod-id
:pod/description "Babashka pod for parsing HTML with jsoup."
:pod/example "examples/jsoup.bb"
:pod/language "clojure"
manifest-data {:pod/name (symbol pod-id)
:pod/description (:description opts)
:pod/example (:example opts)
:pod/language (:language opts)
:pod/license (:license opts)
:pod/version (:version opts)
:pod/artifacts artifacts}
filepath "manifest.edn"
edn-content (format-edn manifest-data)]
(spit filepath edn-content)
(println (str "Wrote " filepath)))))
(println (str "\nWrote " filepath))
(println (str "\nUseful websites for formatting/converting EDN:\n" (str/join "\n" ["https://repo.tiye.me/mvc-works/edn-formatter/" "http://cljson.com/"]))))))

(-main *command-line-args*)
130 changes: 2 additions & 128 deletions bb/tasks.bb
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
(ns tasks
(:require
[babashka.classpath :refer [get-classpath split-classpath]]
[babashka.http-client :as http]
[babashka.pods :as pods]
[clojure.edn :as edn]
[utils :refer [format-edn]]))
[babashka.classpath :refer [get-classpath split-classpath]]))

(defn print-classpath []
(println "=== CLASSPATH BEGIN ===")
Expand All @@ -13,128 +9,6 @@
(println path))
(println "=== CLASSPATH END ==="))


(defn manifest.edn
"Generates a `manifest.edn` for [Babashka pod registry](https://github.com/babashka/pod-registry/)."
[{:keys [aarch64-linux
aarch64-macos
amd64-linux
amd64-macos
amd64-windows
license
pod-id
version
x86_64-linux
x86_64-macos]}]
(let [pod-name (name pod-id)
artifacts (cond-> []
aarch64-linux (conj {:os/arch "aarch64"
:os/name "Linux.*"
:artifact/url aarch64-linux
:artifact/executable pod-name})
aarch64-macos (conj {:os/arch "aarch64"
:os/name "Mac.*"
:artifact/url aarch64-macos
:artifact/executable pod-name})
amd64-linux (conj {:os/arch "amd64"
:os/name "Linux.*"
:artifact/url amd64-linux
:artifact/executable pod-name})
amd64-macos (conj {:os/arch "amd64"
:os/name "Mac.*"
:artifact/url amd64-macos
:artifact/executable pod-name})
amd64-windows (conj {:os/arch "amd64"
:os/name "Windows.*"
:artifact/url amd64-windows
:artifact/executable (format "%s.exe" pod-name)})
x86_64-linux (conj {:os/arch "x86_64"
:os/name "Linux.*"
:artifact/url x86_64-linux
:artifact/executable pod-name})
x86_64-macos (conj {:os/arch "x86_64"
:os/name "Mac.*"
:artifact/url x86_64-macos
:artifact/executable pod-name}))]
{:pod/name pod-id
:pod/description "Babashka pod for parsing HTML with jsoup."
:pod/example "examples/jsoup.bb"
:pod/language "clojure"
:pod/license license
:pod/version version
:pod/artifacts artifacts}))

(defn write-manifest.edn [opts]
(prn "*command-line-args*" *command-line-args*)
;; (when *command-line-args*
;; (prn "got CLI args" *command-line-args*))
(let [filepath "manifest.edn"
manifest-data (manifest.edn opts)
edn-content (format-edn manifest-data)]
(spit filepath edn-content)
(println (str "Wrote " filepath))
;; (println (edn/read-string (slurp filepath)))
))

(comment
(print-classpath)

(def project (-> (edn/read-string (slurp "deps.edn")) :aliases :neil :project))
(def pod-id (:name project))
(def pod-name (name pod-id))
(def pod-version (:version project))

(def uber-file (format "target/%s-%s-standalone.jar" pod-name pod-version))

;; It seems these functions cannot be defined outside of a rich comment block,
;; otherwise the Babashka task runner cannot run ANY task. I guess it's due to
;; the fact that jsoup/select cannot be resolved until the pod is loaded.
(defn demo-pod-native []
(prn (str "Load Babashka pod " pod-name " version " pod-version " (binary)"))
(pods/load-pod "./target/pod-jackdbd-jsoup")
(prn "pod loaded (binary)")

(require '[pod.jackdbd.jsoup :as jsoup])
(prn "pod required (binary)")

(-> (http/get "https://clojure.org")
:body
(jsoup/select "div p")
first
:text))

(defn demo-pod-uberjar []
(prn (str "Load Babashka pod " pod-name " version " pod-version " (uberjar)"))
(pods/load-pod ["java" "-jar" uber-file])
(prn "pod loaded (uberjar)")

(require '[pod.jackdbd.jsoup :as jsoup])
(prn "pod required (uberjar)")

(-> (http/get "https://clojure.org")
:body
(jsoup/select "div p")
first
:text))

(demo-pod-native)
(demo-pod-uberjar)

(def aarch64-macos "https://github.com/fluree/pod-fluree-crypto/releases/download/v0.1.2/pod-fluree-crypto-linux-arm64.zip")
(def amd64-windows "some-github-release-url-windows_amd64.zip")
(def x86_64-linux "some-github-release-url-linux_x86_64.zip")

(manifest.edn {:aarch64-macos aarch64-macos
:amd64-windows amd64-windows
:x86_64-linux x86_64-linux
:license "MIT"
:pod-id pod-id
:version pod-version})

(write-manifest.edn {:aarch64-macos aarch64-macos
:amd64-windows amd64-windows
:x86_64-linux x86_64-linux
:license "MIT"
:pod-id pod-id
:version pod-version})
)
)
Loading

0 comments on commit 34403ff

Please sign in to comment.