Code

Introduce repartitioning#309

Code

Merged

PhilippeMoussalli merged 37 commits intomainml6team/fondant:mainfrom introduce-repartioningml6team/fondant:introduce-repartioningCopy head branch name to clipboardJul 25, 2023

+455-36

Commits

Commits on Jul 11, 2023

• optimize image download component

  PhilippeMoussalli

  committed
  2683b93

  Copy full SHA for 2683b93

• Merge branch 'main' into improve-imagedownload

  PhilippeMoussalli

  committed
  78a60c2

  Copy full SHA for 78a60c2

Commits on Jul 17, 2023

• add dask diagnostics to fondant

  PhilippeMoussalli

  committed
  1b7521c

  Copy full SHA for 1b7521c

Commits on Jul 18, 2023

• add repartitioning strategy to fondant

  PhilippeMoussalli

  committed
  1c53aa1

  Copy full SHA for 1c53aa1

• remove changes from component

  PhilippeMoussalli

  committed
  897f5f8

  Copy full SHA for 897f5f8

• Merge branch 'main' into improve-imagedownload

  PhilippeMoussalli

  committed
  0c48233

  Copy full SHA for 0c48233

Commits on Jul 19, 2023

• change save path diagnostics

  PhilippeMoussalli

  committed
  2edcd91

  Copy full SHA for 2edcd91

• add client to dataIO

  PhilippeMoussalli

  committed
  d63eed7

  Copy full SHA for d63eed7

• remove progressbar

  PhilippeMoussalli

  committed
  51cadfe

  Copy full SHA for 51cadfe

• move client

  PhilippeMoussalli

  committed
  678b3e1

  Copy full SHA for 678b3e1

• move dataframe visualization

  PhilippeMoussalli

  committed
  44d5493

  Copy full SHA for 44d5493

• add bokeh to dependencies

  PhilippeMoussalli

  committed
  592f3ae

  Copy full SHA for 592f3ae

• add bokeh to dependencies

  PhilippeMoussalli

  committed
  5c56ddf

  Copy full SHA for 5c56ddf

• add bokeh to dependencies

  PhilippeMoussalli

  committed
  8e575c7

  Copy full SHA for 8e575c7

• move client to execute

  PhilippeMoussalli

  committed
  0d1e76b

  Copy full SHA for 0d1e76b

• silence logs

  PhilippeMoussalli

  committed
  43132f5

  Copy full SHA for 43132f5

• remove log supression

  PhilippeMoussalli

  committed
  7f38c31

  Copy full SHA for 7f38c31

• add log silencing to cluster

  PhilippeMoussalli

  committed
  d764ad4

  Copy full SHA for d764ad4

• add worker log filter

  PhilippeMoussalli

  committed
  d50cc6b

  Copy full SHA for d50cc6b

• supress logs

  PhilippeMoussalli

  committed
  9e1eb31

  Copy full SHA for 9e1eb31

• supress logs

  PhilippeMoussalli

  committed
  f344ea8

  Copy full SHA for f344ea8

• add output_partition_size as an argument and revert diagnosis

  PhilippeMoussalli

  committed
  08b74cc

  Copy full SHA for 08b74cc

• remove output_partition from user arguments

  PhilippeMoussalli

  committed
  df19ad3

  Copy full SHA for df19ad3

• enable more control over partitioning

  PhilippeMoussalli

  committed
  09ef925

  Copy full SHA for 09ef925

• fix test

  PhilippeMoussalli

  committed
  6b1e63b

  Copy full SHA for 6b1e63b

• Merge branch 'main' into introduce-repartioning

  PhilippeMoussalli

  committed
  57a8af4

  Copy full SHA for 57a8af4

• correct load partitioning

  PhilippeMoussalli

  committed
  8715369

  Copy full SHA for 8715369

Commits on Jul 24, 2023

• address PR feedback

  PhilippeMoussalli

  committed
  c15e24c

  Copy full SHA for c15e24c

• Adjust argument passing

  PhilippeMoussalli

  committed
  9ff1e68

  Copy full SHA for 9ff1e68

• debug

  PhilippeMoussalli

  committed
  4c8d482

  Copy full SHA for 4c8d482

• Further debugging

  PhilippeMoussalli

  committed
  5bf3a50

  Copy full SHA for 5bf3a50

• Further debugging

  PhilippeMoussalli

  committed
  18e1f79

  Copy full SHA for 18e1f79

• Revert debug and improve logging

  PhilippeMoussalli

  committed
  b2bdecc

  Copy full SHA for b2bdecc

• Merge branch 'main' into introduce-repartioning

  PhilippeMoussalli

  committed
  951a786

  Copy full SHA for 951a786

• fix indentation

  PhilippeMoussalli

  committed
  3394fcf

  Copy full SHA for 3394fcf

• fix tests

  PhilippeMoussalli

  committed
  02561ab

  Copy full SHA for 02561ab

Commits on Jul 25, 2023

• add docs

  PhilippeMoussalli

  committed
  99746dc

  Copy full SHA for 99746dc