Code

Add Flink Operator Bakery#21

Code

Merged

yuvipanda merged 44 commits intomainpangeo-forge/pangeo-forge-runner:mainfrom flinkpangeo-forge/pangeo-forge-runner:flinkCopy head branch name to clipboardSep 22, 2022

+558-3

Commits

Commits on Sep 22, 2022

• Add very basic flink runner

  Show description for af21b72
  yuvipanda

  committed
  af21b72

  Copy full SHA for af21b72

• Create FlinkCluster & port-forward it appropriately

  yuvipanda

  committed
  010f1f3

  Copy full SHA for 010f1f3

• Make sure kubectl is installed

  yuvipanda

  committed
  a5e5731

  Copy full SHA for a5e5731

• Rename the Bakery to better reflect what it is

  Show description for 225cde3
  yuvipanda

  committed
  225cde3

  Copy full SHA for 225cde3

• Parameterize flink configuration

  yuvipanda

  committed
  6cd5c5a

  Copy full SHA for 6cd5c5a

• Setup k3s + flink operator in CI

  yuvipanda

  committed
  f062a19

  Copy full SHA for f062a19

• Move flink tests to their own workflow

  yuvipanda

  committed
  e1327e3

  Copy full SHA for e1327e3

• Setup cert-manager in GHA

  yuvipanda

  committed
  73ab352

  Copy full SHA for 73ab352

• Try waiting for cert-manager to be setup

  yuvipanda

  committed
  1ade0a8

  Copy full SHA for 1ade0a8

• Wait for cert-manager to be ready before installing flink

  yuvipanda

  committed
  5dba943

  Copy full SHA for 5dba943

• Wait for cert-manager correctly

  yuvipanda

  committed
  621f0c7

  Copy full SHA for 621f0c7

• Setup Python on flink test

  yuvipanda

  committed
  ae8ce86

  Copy full SHA for ae8ce86

• Setup minio

  yuvipanda

  committed
  4d6a1be

  Copy full SHA for 4d6a1be

• Try debug why minio isn't working

  yuvipanda

  committed
  1ba8b70

  Copy full SHA for 1ba8b70

• See why minio is failing

  yuvipanda

  committed
  07b5eaa

  Copy full SHA for 07b5eaa

• Set some minio config

  yuvipanda

  committed
  1d59508

  Copy full SHA for 1d59508

• Speed up readiness test for sdk harnass

  yuvipanda

  committed
  4ed0347

  Copy full SHA for 4ed0347

• Set rootPassword to be at least 8 chars long

  yuvipanda

  committed
  9828cc9

  Copy full SHA for 9828cc9

• Wait for minio install to finish

  yuvipanda

  committed
  c8a94aa

  Copy full SHA for c8a94aa

• Try using minio from local machine

  Show description for b775532
  yuvipanda

  committed
  b775532

  Copy full SHA for b775532

• Don't set up minio in GHA inside k8s

  yuvipanda

  committed
  8053015

  Copy full SHA for 8053015

• Install minio + run flink test

  yuvipanda

  committed
  a17de4a

  Copy full SHA for a17de4a

• Actually add flink test

  yuvipanda

  committed
  fc9b924

  Copy full SHA for fc9b924

• Fix test fixtures

  Show description for 8b87481
  yuvipanda

  committed
  8b87481

  Copy full SHA for 8b87481

• Try fix kubectl wait

  Show description for 1a39464
  yuvipanda

  committed
  1a39464

  Copy full SHA for 1a39464

• Temporarily add a time.sleep to get kubectl wait to work

  yuvipanda

  committed
  6c125fb

  Copy full SHA for 6c125fb

• Show what various pods are doing after test fails

  yuvipanda

  committed
  952f182

  Copy full SHA for 952f182

• Mark flink bakery as blocking

  Show description for 4148251
  yuvipanda

  committed
  4148251

  Copy full SHA for 4148251

• Install socat explicitly in GHA

  Show description for 8cf2951
  yuvipanda

  committed
  8cf2951

  Copy full SHA for 8cf2951

• Add sudo to apt commands

  yuvipanda

  committed
  315627c

  Copy full SHA for 315627c

• Specify full name of kubectl

  yuvipanda

  committed
  95e8ab6

  Copy full SHA for 95e8ab6

• Just wait for 5 mins for flink test to complete

  yuvipanda

  committed
  bcc5ed6

  Copy full SHA for bcc5ed6

• Output more diagnostics

  yuvipanda

  committed
  bdff932

  Copy full SHA for bdff932

• Bump down default requiremebts

  Show description for 398b1ff
  yuvipanda

  committed
  398b1ff

  Copy full SHA for 398b1ff

• Reduce wait in flink test

  yuvipanda

  committed
  b3b94a0

  Copy full SHA for b3b94a0

• Bump up flink test time

  Show description for f987668
  yuvipanda

  committed
  f987668

  Copy full SHA for f987668

• Don't run flink tests on unit test job

  yuvipanda

  committed
  6414dc1

  Copy full SHA for 6414dc1

• Actually test the flink test

  yuvipanda

  committed
  3e53d27

  Copy full SHA for 3e53d27

• Upload cov to codecov for flink test too

  yuvipanda

  committed
  136abdf

  Copy full SHA for 136abdf

• Make resources of flink deployment configurable

  yuvipanda

  committed
  317e35e

  Copy full SHA for 317e35e

• Set default memory for both job & taskmanager

  Show description for b8e3a64
  yuvipanda

  committed
  b8e3a64

  Copy full SHA for b8e3a64

• Add a tutorial on running on AWS

  yuvipanda

  committed
  9ca9004

  Copy full SHA for 9ca9004

• Explicitly specify CPU, or flink asks for 2

  yuvipanda

  committed
  6d2530e

  Copy full SHA for 6d2530e

• Add a little more documentation for flink operator

  yuvipanda

  committed
  8b90539

  Copy full SHA for 8b90539