Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delta Lake Diff MVP: Update releaser and rename plugin artifact #5427

Merged
merged 40 commits into from
Mar 22, 2023

Conversation

Jonathan-Rosenberg
Copy link
Contributor

@Jonathan-Rosenberg Jonathan-Rosenberg commented Mar 9, 2023

Linked Issue

Closes #4976


Change Description

  1. Rename the goreleaser workflow to releaser.
  2. Add Delta Lake plugin building jobs to the workflow.
  3. Change the .goreleaser.yaml file to include the new archives.
  4. Rename the Delta Lake plugin artifact to delta_diff.

How was this tested?

Tested the workflow without actually releasing any artifact, then uploading the artifacts to a custom S3 bucket, and then using the plugin that was created.


Don't merge this PR!

@Jonathan-Rosenberg Jonathan-Rosenberg changed the title Feature/delta/releaser Delta Lake Diff MVP: Update releaser and rename plugin artifact Mar 14, 2023
@Jonathan-Rosenberg Jonathan-Rosenberg marked this pull request as ready for review March 14, 2023 10:19
@Jonathan-Rosenberg Jonathan-Rosenberg added include-changelog PR description should be included in next release changelog pr/do-not-merge labels Mar 14, 2023
Copy link
Contributor

@arielshaqed arielshaqed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment on lines +87 to +92
mv delta-aarch64-apple-darwin delta-darwin-arm64
mv delta-x86_64-apple-darwin delta-darwin-x86_64
mv delta-aarch64-pc-windows-msvc delta-windows-arm64
mv delta-x86_64-pc-windows-msvc delta-windows-x86_64
mv delta-aarch64-unknown-linux-musl delta-linux-arm64
mv delta-x86_64-unknown-linux-musl delta-linux-x86_64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use variables from the matrix here, too? I worry that having configuration in two places makes it harder to understand.

* change default behavior of plugin loading to scan in the default plugins directory for known plugins

* get delta lake diff plugin from default path if available

* add homedir.Expand call to custom plugin path

* remove empty line

Co-authored-by: Barak Amar <barak.amar@treeverse.io>

* change version from a pointer to an int

* Delta Lake Diff: Dockerfile (#5496)

* Display images in markdown with support for lakefs:// URIs (#5449)

* Display images in markdown with support for lakefs:// URIs

* change dockerfile to include delta diff

* change Dockerfile to include the Delta Lake diff plugin

* fix docker-compose files

* add a stage with both lakefs and plugins and separate it from a lakefs-only stage

* PR fixes

* Delta Lake Diff: Enable feature (#5508)

* fix hover issue when hovering over a tree-entry-row

* enable Delta Lake diff feature

* remove experimental delta diff button

* PR fixes

* change docker image publish target to lakefs-plugins

* fix bug hunt bugs (#5534)

* remove empty line

* Delta Lake: Update Dockerfile base images to Alpine (#5546)

* change the Dockerfile back to Alpine and build the Delta plugin dynamically

* remove alpine-sdk from dockerfile

* add alpine-sdk

* add comment about the 'RUSTFLAGS=-Ctarget-feature=-crt-static' flag

* change back from bullseye to alpine

---------

Co-authored-by: eladlachmi <110764839+eladlachmi@users.noreply.github.com>

---------

Co-authored-by: Barak Amar <barak.amar@treeverse.io>
Co-authored-by: eladlachmi <110764839+eladlachmi@users.noreply.github.com>
@Jonathan-Rosenberg Jonathan-Rosenberg merged commit 6a1a67b into master Mar 22, 2023
@Jonathan-Rosenberg Jonathan-Rosenberg deleted the feature/delta/releaser branch March 22, 2023 09:57
nopcoder added a commit that referenced this pull request Apr 17, 2023
* add makefile target to build delta diff plugin

* test goreleaser  with rust

* remove cache step

* add activate on push to this branch

* change working-directory

* remove working-directory

* remove target from args

* add protoc for mac and name steps

* add version to protoc action

* switch from Arduino

* use correct version

* change back to arduino

* change version to 3.x

* add protoc to unix and windows

* change version to 3.x

* change to release, upload artifact

* downloading artifacts

* add needs to releaser

* add build

* move --release to arguments

* fix artifact location

* fix naming

* add cache

* add .exe suffix to windows build target

* add id to the download artifact step

* add registry and index to cache

* add renaming of artifacts

* move download-artifacts to the goreleaser job

* change cache support to use custom action

* run goreleaser without truly releasing

* fix goreleaser

* change delta artifact location

* fix releaser yaml

* change releaser to be production ready

* change from 3 jobs definitions to a single job definition with a more complex matrix

* fix makefile

* Delta Lake Diff: Change default plugin loading (#5495)

* change default behavior of plugin loading to scan in the default plugins directory for known plugins

* get delta lake diff plugin from default path if available

* add homedir.Expand call to custom plugin path

* remove empty line

Co-authored-by: Barak Amar <barak.amar@treeverse.io>

* change version from a pointer to an int

* Delta Lake Diff: Dockerfile (#5496)

* Display images in markdown with support for lakefs:// URIs (#5449)

* Display images in markdown with support for lakefs:// URIs

* change dockerfile to include delta diff

* change Dockerfile to include the Delta Lake diff plugin

* fix docker-compose files

* add a stage with both lakefs and plugins and separate it from a lakefs-only stage

* PR fixes

* Delta Lake Diff: Enable feature (#5508)

* fix hover issue when hovering over a tree-entry-row

* enable Delta Lake diff feature

* remove experimental delta diff button

* PR fixes

* change docker image publish target to lakefs-plugins

* fix bug hunt bugs (#5534)

* remove empty line

* Delta Lake: Update Dockerfile base images to Alpine (#5546)

* change the Dockerfile back to Alpine and build the Delta plugin dynamically

* remove alpine-sdk from dockerfile

* add alpine-sdk

* add comment about the 'RUSTFLAGS=-Ctarget-feature=-crt-static' flag

* change back from bullseye to alpine

---------

Co-authored-by: eladlachmi <110764839+eladlachmi@users.noreply.github.com>

---------

Co-authored-by: Barak Amar <barak.amar@treeverse.io>
Co-authored-by: eladlachmi <110764839+eladlachmi@users.noreply.github.com>

---------

Co-authored-by: Barak Amar <barak.amar@treeverse.io>
Co-authored-by: eladlachmi <110764839+eladlachmi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
include-changelog PR description should be included in next release changelog pr/do-not-merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Delta Diff MVP - Task: Update the goreleaser GitHub workflow
3 participants