Impl benchmark jobs #1977

vankichi · 2023-03-14T01:20:37Z

Description:

I have implemented another benchmark jobs (pkg/tools/benchmark/jobs/service):

insert
update
upsert
remove
exsists
getobject

And, I have also updated the continuous benchmark CRD:

add rpc for set the rpc when requesting Vald cluster
add disabled_balanced_update for update / upsert APIs

To use rate limit feature, I have added the internal/timeutil/rate.

Related Issue:

Versions:

Go Version: 1.20.2
Docker Version: 20.10.8
Kubernetes Version: 1.22.0
NGT Version: 2.0.9

Checklist:

I have read the CONTRIBUTING document and completed our CLA agreement.
I have checked open Pull Requests for the similar feature or fixes?

Special notes for your reviewer:

cloudflare-workers-and-pages · 2023-03-14T01:20:59Z

Deploying with Cloudflare Pages

Latest commit:	`7af26af`
Status:	✅ Deploy successful!
Preview URL:	https://75d7f0c3.vald.pages.dev
Branch Preview URL:	https://feature-pkg-internal-impl-be.vald.pages.dev

View logs

vdaas-ci · 2023-03-14T01:22:29Z

[CHATOPS:HELP] ChatOps commands.

🙆‍♀️ /approve - approve
💌 /changelog - replace the PR body by changelog details
🍱 /format - format codes and add licenses
✅ /gen-test - generate test codes
🏷️ /label - add labels
⏪ /rebase - rebase main
🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

vdaas-ci · 2023-03-14T01:24:07Z

[WARNING:INTCFG] Changes in interal/config may require you to change Helm charts. Please check.

codecov · 2023-03-14T02:03:04Z

Codecov Report

Patch coverage has no change and project coverage change: -0.13 ⚠️

Comparison is base (5165409) 29.48% compared to head (7af26af) 29.35%.

Additional details and impacted files

@@                       Coverage Diff                        @@
##           feature/continuous-benchmark    #1977      +/-   ##
================================================================
- Coverage                         29.48%   29.35%   -0.13%     
================================================================
  Files                               369      369              
  Lines                             34607    34678      +71     
================================================================
- Hits                              10205    10181      -24     
- Misses                            23978    24070      +92     
- Partials                            424      427       +3

Impacted Files	Coverage Δ
internal/config/benchmark.go	`0.00% <0.00%> (ø)`
internal/errors/benchmark.go	`0.00% <0.00%> (ø)`
pkg/tools/benchmark/job/config/config.go	`8.77% <0.00%> (-0.32%)`	⬇️

... and 4 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

github-actions · 2023-03-14T02:05:40Z

pkg/tools/benchmark/job/service/insert.go

+			b.ReportMetric(float64(latency.Microseconds()), "latency")
+			log.Infof("[benchmark job] iter=%d, bN=%d, Name=%s, Uuid=%s, Ips=%v, latency=%d", i, b.N, res.Name, res.Uuid, res.Ips, latency.Microseconds())
+		})
+		// TODO: send metrics to the Prometeus


🚫 [golangci] _{reported by reviewdog 🐶}
pkg/tools/benchmark/job/service/insert.go:86: Line contains TODO/BUG/FIXME: "TODO: send metrics to the Prometeus" (godox)

pkg/tools/benchmark/job/service/remove.go

pkg/tools/benchmark/job/service/update.go

pkg/tools/benchmark/job/service/upsert.go

github-actions · 2023-03-14T02:05:41Z

pkg/tools/benchmark/job/service/insert.go

+		err := errors.NewErrInvalidOption("insertConfig", j.insertConfig)
+		select {
+		case <-ctx.Done():
+			if err != context.Canceled {


🚫 [golangci] _{reported by reviewdog 🐶}
comparing with != will fail on wrapped errors. Use errors.Is to check for a specific error (errorlint)

pkg/tools/benchmark/job/service/upsert.go

internal/test/data/hdf5/hdf5.go

pkg/tools/benchmark/job/service/update.go

pkg/tools/benchmark/job/service/upsert.go

pkg/tools/benchmark/job/service/remove.go

pkg/tools/benchmark/job/service/object.go

* Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl status handle of continuous benchmark crds (#1955) Signed-off-by: vankichi <kyukawa315@gmail.com> * Impl benchmark jobs (#1977) * Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl benchmark jobs Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * create helm template for benchmark operator (#2027) * ✨ create helm template for benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ refactor helm template Signed-off-by: vankichi <kyukawa315@gmail.com> * style: Format code with gofumpt and prettier --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * Refactor helm template and operator logic (#2043) * Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl status handle of continuous benchmark crds (#1955) Signed-off-by: vankichi <kyukawa315@gmail.com> * Impl benchmark jobs (#1977) * Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl benchmark jobs Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ♻️ Refactor helm template and operator logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Add download original dataset URL option Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Set docker image location at the benchmark operator configmap and use it when information is set Signed-off-by: vankichi <kyukawa315@gmail.com> * add search algorithm benchmark and update search aggregation algo Signed-off-by: kpango <kpango@vdaas.org> * ✨ Add search result aggregation option Signed-off-by: vankichi <kyukawa315@gmail.com> * style: Format code with prettier and gofumpt * Improve job performance (#2061) * 🐛 Fix job function to apply rate limiter * ♻️ Add pyroscope setting Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Fix build error Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Fix docker file and add concurrencyLimit for job goroutine Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Fix job_template.go by feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Fix job logic by feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Fix Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: kpango <kpango@vdaas.org> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * Impl benchmark jobs (#1977) * Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl benchmark jobs Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * Refactor helm template and operator logic (#2043) * Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl status handle of continuous benchmark crds (#1955) Signed-off-by: vankichi <kyukawa315@gmail.com> * Impl benchmark jobs (#1977) * Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl benchmark jobs Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ♻️ Refactor helm template and operator logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Add download original dataset URL option Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Set docker image location at the benchmark operator configmap and use it when information is set Signed-off-by: vankichi <kyukawa315@gmail.com> * add search algorithm benchmark and update search aggregation algo Signed-off-by: kpango <kpango@vdaas.org> * ✨ Add search result aggregation option Signed-off-by: vankichi <kyukawa315@gmail.com> * style: Format code with prettier and gofumpt * Improve job performance (#2061) * 🐛 Fix job function to apply rate limiter * ♻️ Add pyroscope setting Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Fix build error Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Fix docker file and add concurrencyLimit for job goroutine Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Fix job_template.go by feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Fix job logic by feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Fix Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: kpango <kpango@vdaas.org> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ♻️ Add noise to vector when update/upsert and update crd Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ add deepmerge func for override default config by user-defined config Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor deepmerge Signed-off-by: vankichi <kyukawa315@gmail.com> * style: Format code with prettier and gofumpt * style: Format code with prettier and gofumpt * ♻️ change add noise func Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ use golang/sync/error for prevent OOM KILL Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix config adn add test Signed-off-by: vankichi <kyukawa315@gmail.com> * style: Format code with prettier and gofumpt --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: kpango <kpango@vdaas.org> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl benchmark jobs Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl status handle of continuous benchmark crds (#1955) Signed-off-by: vankichi <kyukawa315@gmail.com> * Impl benchmark jobs (#1977) * Implement base of continuous benchmark tool (#1776) * Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ✨ impl benchmark jobs Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * ♻️ Refactor helm template and operator logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Add download original dataset URL option Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Set docker image location at the benchmark operator configmap and use it when information is set Signed-off-by: vankichi <kyukawa315@gmail.com> * add search algorithm benchmark and update search aggregation algo Signed-off-by: kpango <kpango@vdaas.org> * ✨ Add search result aggregation option Signed-off-by: vankichi <kyukawa315@gmail.com> * style: Format code with prettier and gofumpt * Improve job performance (#2061) * 🐛 Fix job function to apply rate limiter * ♻️ Add pyroscope setting Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Fix build error Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Fix docker file and add concurrencyLimit for job goroutine Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Fix job_template.go by feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Fix job logic by feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ Fix Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: kpango <kpango@vdaas.org> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>