Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move Rust and Go SDKs to their own repos #2289

Merged
merged 5 commits into from
Feb 21, 2024
Merged

move Rust and Go SDKs to their own repos #2289

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ecc3352
move Rust and Go SDKs to their own repos
dicej Feb 20, 2024
98aaeca
restore accidentally-removed lines from release-process.md
dicej Feb 21, 2024
1282dd8
update SDK examples links to point to `stable` branches
dicej Feb 21, 2024
65ed372
re-add (updated version of) `push-templates-tag` to release.yml
dicej Feb 21, 2024
04b8754
re-add `build-and-sign` as dependency of `push-templates-tag`
dicej Feb 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions .github/gh-checktags.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
#!/usr/bin/env bash

## for the release tags v*, check if
## - golang sdk tag exists
## - spin templates tag exists

set -e
Expand All @@ -23,12 +22,6 @@ for tag in $tags; do
continue
fi

# check golang sdk tag
if [[ -z "$(git tag -l sdk/go/$tag)" ]]; then
echo "tag sdk/go/$tag does not exist"
exit_code=1
fi

# remove trailing .\d+ from the version
major_minor=`echo $tag | sed 's/\.[0-9]*$//g'`

Expand Down
30 changes: 0 additions & 30 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,36 +167,6 @@ jobs:
# Only run integration tests on macOS as they will be run on ubuntu separately
if: ${{ matrix.runner == 'macos-latest' }}

test-go:
name: Test Spin SDK - Go
runs-on: ubuntu-latest
needs: build-rust-ubuntu
steps:
- uses: actions/checkout@v3

- name: Retrieve saved Spin Binary
uses: actions/download-artifact@v3
with:
name: spin-ubuntu-latest
path: target/debug/

- name: Fix Spin Binary permissions
run: |
ls -lah target/
ls -lah target/debug
chmod +x target/debug/spin

- name: setup dependencies
uses: ./.github/actions/spin-ci-dependencies
with:
rust: true
golang: true
tinygo: true
wasmtime: true

- name: "Test Go SDK"
run: make test-sdk-go

all-integration-tests:
# run on a larger runner for more SSD/resource access
runs-on: ubuntu-22.04-4core-spin
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/check-tags.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Check tags for templates and sdk
name: Check tags for templates
on:
schedule:
- cron: '0 0 * * *'
Expand Down
91 changes: 2 additions & 89 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,84 +254,17 @@ jobs:
--title ${{ github.ref_name }} \
--generate-notes ${{ env.PRERELEASE }}

create-go-sdk-tag:
name: create tag sdk/go/v*
runs-on: ubuntu-latest
needs: build-and-sign
if: startsWith(github.ref, 'refs/tags/v')
steps:
- uses: actions/checkout@v3

- name: Set the tag to sdk/go/v*
shell: bash
run: echo "GO_SDK_TAG=sdk/go/${GITHUB_REF/refs\/tags\//}" >> $GITHUB_ENV

- name: Tag sdk/go/v* and push it
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
git tag ${{ env.GO_SDK_TAG }}
git push origin ${{ env.GO_SDK_TAG }}

create-template-sdk-update-pr:
name: Create PR with template SDK updates
runs-on: ubuntu-latest
needs: create-go-sdk-tag
if: startsWith(github.ref, 'refs/tags/v')
steps:
- uses: actions/checkout@v3

- name: Set the spin tag
shell: bash
run: |
echo "SPIN_TAG=${GITHUB_REF/refs\/tags\//}" >> $GITHUB_ENV

- name: Set the PR base branch
shell: bash
run: |
IFS=. read -r major minor patch <<< "${{ env.SPIN_TAG }}"
echo "RELEASE_BRANCH=$major.$minor" >> $GITHUB_ENV

- name: Change sdk version
shell: bash
run: |
cd templates
SDK_VERSION=${{ env.SPIN_TAG }} make

- name: Import GPG key
uses: crazy-max/ghaction-import-gpg@v5
with:
gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}
passphrase: ${{ secrets.PASSPHRASE }}
git_user_signingkey: true
git_commit_gpgsign: true

- name: Create Pull Request
uses: peter-evans/create-pull-request@v4
with:
commit-message: "feat(templates): update sdk to ${{ env.SPIN_TAG }}"
title: "feat(templates): update sdk to ${{ env.SPIN_TAG }}"
body: Update the SDK version used by the templates
branch: update-sdk-${{ env.SPIN_TAG }}
base: ${{ env.RELEASE_BRANCH }}
delete-branch: true
committer: fermybot <103076628+fermybot@users.noreply.github.com>
author: fermybot <103076628+fermybot@users.noreply.github.com>
signoff: true

# This will run when the PR above is approved and merged into main via a merge commit
push-templates-tag:
Copy link
Contributor

@vdice vdice Feb 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double-checking with @itowlson on the subject of templates:

For go, Assuming the v3 version of the sdk in its new home is a ways out and there may be a handful of subsequent Spin releases (eg 2.3, 2.4, etc) before it is ready, do we need to do anything ensure templates function as expected? For as long as the go templates are in this repo and no new templates tags are pushed, will future spin clients just get the last, most recent templates (2.2) for this interim period?

The rust templates meanwhile should be ready to move to their new sdk home, right? Would we update the spin templates cli commands to add the new sdk repo as a default to fetch from?

(Or maybe the go templates are also ready to migrate to their new sdk home as well -- especially if spin clients just fetch the 2.2-tagged templates near-term?)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If template tags are not updated, then post-2.2 Spin releases will fall forward to the templates on main.

Moving the templates has implications for the installer, docs, and spin new. I am wary of moving those without careful coordination.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's assume we won't move the templates for the foreseeable future. In that case, I guess our release process could look something like this:

  • SDKs are released on their own schedule; for example, let's say we release spin-rust-sdk 3.0.0 tomorrow
  • Templates remain tied to Spin releases, so they are not updated until the next Spin release; e.g. Spin 2.3 would be our first opportunity to update the Rust templates to use spin-rust-sdk 3.0.0 (although I guess we could update them in main sooner than that)
    • i.e. when we make a Spin release, we tag the templates as usual, and those templates will point to whatever the latest stable versions of the SDKs happen to be

Does that sound right? (No strong opinions here -- just trying to form a plan of some kind)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to tag templates at minimum when we do a (non-patch) Spin release.

We can optionally update the templates and move the tags when we do a SDK release, even if no Spin release is planned. But that's not part of the Spin release process and can be ad-hocked in if we ever feel the urge to actually do it.

Copy link
Contributor

@vdice vdice Feb 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i.e. when we make a Spin release, we tag the templates as usual, and those templates will point to whatever the latest stable versions of the SDKs happen to be

That sounds reasonable to me. So, we'd re-introduce the corresponding automation around pushing the templates tag (eg spin/templates/v2.3) on a Spin release for the near-term? edit: as @itowlson says above, didn't yet see that comment when I posted this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Er, maybe just keep push-templates-tag and remove create-template-sdk-update-pr, since we don't want to update SDK versions automatically.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g.:

  push-templates-tag:
    runs-on: ubuntu-latest
    needs: build-and-sign
    if: startsWith(github.ref, 'refs/tags/v')
    steps:
      - uses: actions/checkout@v3

      - name: Set the tag to spin/templates/v*
        shell: bash
        run: |
          spin_tag=$(echo "${{ github.ref }}" | grep -Eo "refs/tags/v[0-9.]+")
          IFS=. read -r major minor patch <<< "${spin_tag}"
          echo "TEMPLATE_TAG=spin/templates/$major.$minor" >> $GITHUB_ENV

      - name: Tag spin/templates/v* and push it
        shell: bash
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          git tag ${{ env.TEMPLATE_TAG }} -f
          git push origin ${{ env.TEMPLATE_TAG }} -f

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe I've addressed this in 65ed372, but not sure how to test it. Maybe push a release tag to my fork?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it worked: https://github.com/dicej/spin/actions/runs/7995470091/job/21835889372

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that should be it... thanks for testing!

runs-on: ubuntu-latest
needs: build-and-sign
if: github.event.commits[0].author.name == 'fermybot' && contains(github.event.commits[0].message, 'update sdk')
if: startsWith(github.ref, 'refs/tags/v')
steps:
- uses: actions/checkout@v3

- name: Set the tag to spin/templates/v*
shell: bash
run: |
spin_tag=$(echo "${{ github.event.commits[0].message }}" | grep -Eo v[0-9.]+)
spin_tag=$(echo "${{ github.ref }}" | grep -Eo "refs/tags/v[0-9.]+")
IFS=. read -r major minor patch <<< "${spin_tag}"
echo "TEMPLATE_TAG=spin/templates/$major.$minor" >> $GITHUB_ENV

Expand Down Expand Up @@ -431,23 +364,3 @@ jobs:
with:
name: spin
path: _dist/spin-${{ env.RELEASE_VERSION }}-static-${{ env.RUNNER_OS }}-${{ matrix.config.arch }}.tar.gz

crates:
name: Publish to crates.io
runs-on: ubuntu-latest
needs: build-and-sign
if: |
startsWith(github.ref, 'refs/tags/v') &&
github.repository_owner == 'fermyon'
steps:
- uses: actions/checkout@v2

- name: Publish spin-macro to crates.io
working-directory: ./sdk/rust/macro
run: |
cargo publish --token ${{ secrets.CARGO_REGISTRY_TOKEN }}

- name: Publish spin-sdk to crates.io
working-directory: ./sdk/rust
run: |
cargo publish --token ${{ secrets.CARGO_REGISTRY_TOKEN }}
132 changes: 0 additions & 132 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,6 @@ llm-cublas = ["llm", "spin-trigger-http/llm-cublas"]
[workspace]
members = [
"crates/*",
"sdk/rust",
"sdk/rust/macro",
"tests/runtime-tests",
"tests/testing-framework",
]
Expand Down
8 changes: 2 additions & 6 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ export PATH := target/debug:target/release:$(HOME)/.cargo/bin:$(PATH)

## overrides for Windows
ifeq ($(OS),Windows_NT)
LOG_LEVEL_VAR =
LOG_LEVEL_VAR =
endif

.PHONY: build
Expand Down Expand Up @@ -39,7 +39,7 @@ lint-all: lint lint-rust-examples

## Bring all of the checked in `Cargo.lock` files up-to-date
.PHONY: update-cargo-locks
update-cargo-locks:
update-cargo-locks:
echo "Updating Cargo.toml"
cargo update -w --offline; \
for manifest_path in $$(find examples -name Cargo.toml); do \
Expand Down Expand Up @@ -71,10 +71,6 @@ test-integration: test-runtime
test-integration-full: test-runtime-full
cargo test --release integration_tests --no-default-features --features extern-dependencies-tests --no-fail-fast -- --nocapture

.PHONY: test-sdk-go
test-sdk-go:
$(MAKE) -C sdk/go test

# simple convenience for developing with TLS
.PHONY: tls
tls: ${CERT_NAME}.crt.pem
Expand Down
Loading
Loading