Code

Dataset v2.0#461

Code

Merged

aliberts merged 130 commits intomainhuggingface/lerobot:mainfrom user/aliberts/2024_09_25_reshape_datasethuggingface/lerobot:user/aliberts/2024_09_25_reshape_datasetCopy head branch name to clipboardNov 29, 2024

+6,109-2,229

Commits

Commits on Oct 3, 2024

• WIP

  aliberts

  committed
  ad115b6

  Copy full SHA for ad115b6

Commits on Oct 4, 2024

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  17a1214

  Copy full SHA for 17a1214

• Add upload folders

  aliberts

  committed
  1016a98

  Copy full SHA for 1016a98

• Add info.json link

  aliberts

  committed
  07e113c

  Copy full SHA for 07e113c

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  028c17f

  Copy full SHA for 028c17f

Commits on Oct 6, 2024

• Add pixel channels

  aliberts

  committed
  21ba4b5

  Copy full SHA for 21ba4b5

Commits on Oct 8, 2024

• Update info.json format

  aliberts

  committed
  2d75b93

  Copy full SHA for 2d75b93

Commits on Oct 9, 2024

• Rework LeRobotDataset.__init__

  aliberts

  committed
  096824b

  Copy full SHA for 096824b

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  3113038

  Copy full SHA for 3113038

Commits on Oct 10, 2024

• Update LeRobotDataset.__get_item__

  aliberts

  committed
  b417ceb

  Copy full SHA for b417ceb

Commits on Oct 11, 2024

• Add doc, scrap video_frame_keys attribute

  aliberts

  committed
  6d2bc11

  Copy full SHA for 6d2bc11

• Add huggingface-hub patch for offline snapshot_download with local_dir

  aliberts

  committed
  7f68088

  Copy full SHA for 7f68088

• Add padding keys and download_data option

  aliberts

  committed
  3ea5312

  Copy full SHA for 3ea5312

• Add suggestions from code review

  aliberts

  committed
  8bd406e

  Copy full SHA for 8bd406e

Commits on Oct 13, 2024

• Add multitask support, refactor conversion script

  aliberts

  committed
  cf63334

  Copy full SHA for cf63334

Commits on Oct 14, 2024

• Extend v1 compatibility

  aliberts

  committed
  cbc51e1

  Copy full SHA for cbc51e1

• Fix safe_version

  aliberts

  committed
  f96773d

  Copy full SHA for f96773d

Commits on Oct 15, 2024

• Cleanup, fix load_tasks

  aliberts

  committed
  835ab5a

  Copy full SHA for 835ab5a

• Update load_tasks doc

  aliberts

  committed
  da78bbf

  Copy full SHA for da78bbf

• WIP add batch convert

  aliberts

  committed
  9433ac5

  Copy full SHA for 9433ac5

• Add fixes for batch convert

  aliberts

  committed
  1102640

  Copy full SHA for 1102640

Commits on Oct 16, 2024

• Add episode chunks logic, move_videos & lfs tracking fix

  aliberts

  committed
  c146ba9

  Copy full SHA for c146ba9

Commits on Oct 17, 2024

• Write episodes as jsonlines

  aliberts

  committed
  50a75ad

  Copy full SHA for 50a75ad

• Add fixes for lfs tracking

  aliberts

  committed
  ad3f112

  Copy full SHA for ad3f112

• Add batch conversion log

  aliberts

  committed
  3ee3739

  Copy full SHA for 3ee3739

• Cleanup

  aliberts

  committed
  7242c57

  Copy full SHA for 7242c57

• Add unitreeh and umi

  aliberts

  committed
  d0d8193

  Copy full SHA for d0d8193

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  354f37a

  Copy full SHA for 354f37a

• Update doc

  aliberts

  committed
  be64d54

  Copy full SHA for be64d54

• Cleanup

  aliberts

  committed
  beacb7e

  Copy full SHA for beacb7e

• Add copyrights

  aliberts

  committed
  3a9f964

  Copy full SHA for 3a9f964

Commits on Oct 18, 2024

• Remove caret requirement

  aliberts

  committed
  91e8ce7

  Copy full SHA for 91e8ce7

• Fix episodes.jsonl

  aliberts

  committed
  e7355ba

  Copy full SHA for e7355ba

• Add download_metadata, move default paths

  aliberts

  committed
  1a51505

  Copy full SHA for 1a51505

• Add load_metadata

  aliberts

  committed
  bce3dc3

  Copy full SHA for bce3dc3

• Move default paths, use jsonlines for tasks

  aliberts

  committed
  ac3798b

  Copy full SHA for ac3798b

Commits on Oct 20, 2024

• Add file paths

  aliberts

  committed
  9316cf4

  Copy full SHA for 9316cf4

• Change card creation

  aliberts

  committed
  e46bdb9

  Copy full SHA for e46bdb9

• Add ImageWriter

  aliberts

  committed
  3b925c3

  Copy full SHA for 3b925c3

• Add add_frame, empty dataset creation

  aliberts

  committed
  c1232a0

  Copy full SHA for c1232a0

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  9ebf8b8

  Copy full SHA for 9ebf8b8

Commits on Oct 21, 2024

• Add add_episode & task logic

  aliberts

  committed
  299451a

  Copy full SHA for 299451a

• add delete_episode, WIP on consolidate

  aliberts

  committed
  c4c0a43

  Copy full SHA for c4c0a43

• Improve consistency between __init__() and create(), WIP on consolidate

  aliberts

  committed
  e991a31

  Copy full SHA for e991a31

Commits on Oct 22, 2024

• Add local_files_only, encode_videos, fix bugs to pass tests (WIP)

  aliberts

  committed
  a805458

  Copy full SHA for a805458

• Add channels to intelrealsense

  aliberts

  committed
  ee52b8b

  Copy full SHA for ee52b8b

• Fix tests

  aliberts

  committed
  b46db7e

  Copy full SHA for b46db7e

• Remove populate dataset

  aliberts

  committed
  6c2cb6e

  Copy full SHA for 6c2cb6e

• Fix paths & add add_frame doc

  aliberts

  committed
  237a484

  Copy full SHA for 237a484

• Remove total_episodes from default parquet path

  aliberts

  committed
  c72dc23

  Copy full SHA for c72dc23

• Update & fix conversion script

  aliberts

  committed
  c3c0141

  Copy full SHA for c3c0141

• Fix episode chunk

  aliberts

  committed
  9dca233

  Copy full SHA for 9dca233

• Update dataset doc

  aliberts

  committed
  fb73cdb

  Copy full SHA for fb73cdb

Commits on Oct 23, 2024

• add write_stats, changes names, add some typing

  aliberts

  committed
  a2a8538

  Copy full SHA for a2a8538

• Fix visualization

  aliberts

  committed
  7ae8d05

  Copy full SHA for 7ae8d05

• Fix check_delta_timestamps

  aliberts

  committed
  b8bdbc1

  Copy full SHA for b8bdbc1

• Fix _query_videos return shapes

  aliberts

  committed
  07570f8

  Copy full SHA for 07570f8

• Fix consolidate

  aliberts

  committed
  1aba80d

  Copy full SHA for 1aba80d

• Nits

  aliberts

  committed
  0098bd2

  Copy full SHA for 0098bd2

• Move ImageWriter creation inside the dataset

  aliberts

  committed
  0d77be9

  Copy full SHA for 0d77be9

• Allow dataset creation without robot

  aliberts

  committed
  60865e8

  Copy full SHA for 60865e8

• Add error msg

  aliberts

  committed
  450eae3

  Copy full SHA for 450eae3

Commits on Oct 24, 2024

• Add test_same_attributes_defined

  aliberts

  committed
  615894d

  Copy full SHA for 615894d

• Add todo

  aliberts

  committed
  8bcf81f

  Copy full SHA for 8bcf81f

• Add json/jsonl io functions

  aliberts

  committed
  18ffa42

  Copy full SHA for 18ffa42

Commits on Oct 25, 2024

• Add video_info, fix image_writer

  aliberts

  committed
  e210d79

  Copy full SHA for e210d79

Commits on Oct 26, 2024

• Speedup test

  aliberts

  committed
  df3d2ec

  Copy full SHA for df3d2ec

Commits on Oct 28, 2024

• Fix image writer

  aliberts

  committed
  51e87f6

  Copy full SHA for 51e87f6

• Add sanity check

  aliberts

  committed
  4c22de2

  Copy full SHA for 4c22de2

Commits on Oct 29, 2024

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  8d57093

  Copy full SHA for 8d57093

• Remove image_writer arg

  aliberts

  committed
  fee5fa5

  Copy full SHA for fee5fa5

• Remove dataset from image_transform tests

  aliberts

  committed
  ee51f54

  Copy full SHA for ee51f54

Commits on Oct 31, 2024

• Add dataset fixtures

  aliberts

  committed
  ff84024

  Copy full SHA for ff84024

• Add test_delta_timestamps.py

  aliberts

  committed
  e69f0c5

  Copy full SHA for e69f0c5

• Update doc

  aliberts

  committed
  e1845d4

  Copy full SHA for e1845d4

• Update doc

  aliberts

  committed
  c70b8d0

  Copy full SHA for c70b8d0

• Split fixtures into factories and files

  aliberts

  committed
  1267c3e

  Copy full SHA for 1267c3e

• Add fixtures in test_datasets

  aliberts

  committed
  ab23a4f

  Copy full SHA for ab23a4f

• Remove/comment obsolete tests

  aliberts

  committed
  443a9ee

  Copy full SHA for 443a9ee

• Remove obsolete code

  aliberts

  committed
  5ea7c78

  Copy full SHA for 5ea7c78

Commits on Nov 1, 2024

• Mock snapshot_download

  aliberts

  committed
  cd1509d

  Copy full SHA for cd1509d

• Add tasks and episodes factories

  aliberts

  committed
  2650872

  Copy full SHA for 2650872

• Rename num_samples -> num_frames for consistency

  aliberts

  committed
  79d114c

  Copy full SHA for 79d114c

• Simplify, add test content, add todo

  aliberts

  committed
  293bdc7

  Copy full SHA for 293bdc7

Commits on Nov 2, 2024

• Add img and img_tensor factories

  aliberts

  committed
  375abd3

  Copy full SHA for 375abd3

• Add test_image_writer, accept PIL images, improve ImageWriter perf in main process

  aliberts

  committed
  6b2ec1e

  Copy full SHA for 6b2ec1e

• Add more options to img factories

  aliberts

  committed
  7a342db

  Copy full SHA for 7a342db

• Add todo in skipped test

  aliberts

  committed
  df2cb51

  Copy full SHA for df2cb51

Commits on Nov 3, 2024

• Fix test_online_buffer.py

  aliberts

  committed
  ac79e8c

  Copy full SHA for ac79e8c

• Add LeRobotDatasetMetadata

  aliberts

  committed
  e4ba084

  Copy full SHA for e4ba084

• Fix hanging

  aliberts

  committed
  16103cb

  Copy full SHA for 16103cb

• Fix vizualize

  aliberts

  committed
  95a4b59

  Copy full SHA for 95a4b59

• Fix werkzeug alert

  aliberts

  committed
  c2d6fb6

  Copy full SHA for c2d6fb6

• Remove end-to-end tests

  aliberts

  committed
  f6c90ca

  Copy full SHA for f6c90ca

• Deactivate policies backward compatibility test

  aliberts

  committed
  56e4603

  Copy full SHA for 56e4603

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  fde29e0

  Copy full SHA for fde29e0

• Fix advanced example 2

  aliberts

  committed
  a6762ec

  Copy full SHA for a6762ec

• Remove reset_episode_index

  aliberts

  committed
  74270c8

  Copy full SHA for 74270c8

• Move calculate_episode_data_index

  aliberts

  committed
  7b159a6

  Copy full SHA for 7b159a6

• Fix test_examples

  aliberts

  committed
  b69a132

  Copy full SHA for b69a132

• Fix test_examples

  aliberts

  committed
  757ea17

  Copy full SHA for 757ea17

Commits on Nov 5, 2024

• Refactor dataset features

  aliberts

  committed
  aed9f40

  Copy full SHA for aed9f40

• Fix tests

  aliberts

  committed
  f3630ad

  Copy full SHA for f3630ad

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  4d15861

  Copy full SHA for 4d15861

Commits on Nov 18, 2024

• Add extra info to dataset card, various fixes from Remi's review

  aliberts

  committed
  a91b7c6

  Copy full SHA for a91b7c6

• Fix test_visualize_dataset_html

  aliberts

  committed
  8546358

  Copy full SHA for 8546358

• Skip test_visualize_local_dataset

  aliberts

  committed
  eda02fa

  Copy full SHA for eda02fa

• Skip test_examples

  aliberts

  committed
  c72ad49

  Copy full SHA for c72ad49

• Add comment on license

  aliberts

  committed
  acae4b4

  Copy full SHA for acae4b4

Commits on Nov 19, 2024

• Improve dataset v2 (#498)

  Cadene

  authored
  1f13bda

  Copy full SHA for 1f13bda

• Use HWC for images

  aliberts

  committed
  6203641

  Copy full SHA for 6203641

• Update example 1

  aliberts

  committed
  9ee8711

  Copy full SHA for 9ee8711

• Fix tests

  aliberts

  committed
  f43e5d0

  Copy full SHA for f43e5d0

Commits on Nov 20, 2024

• Enhance dataset cards

  aliberts

  committed
  c6ad495

  Copy full SHA for c6ad495

• Fix conversion script

  aliberts

  committed
  37da50b

  Copy full SHA for 37da50b

• Add open X datasets

  aliberts

  committed
  93d9bf8

  Copy full SHA for 93d9bf8

• Update example 1

  aliberts

  committed
  36b9b60

  Copy full SHA for 36b9b60

• Remove todos

  aliberts

  committed
  f56d769

  Copy full SHA for f56d769

Commits on Nov 25, 2024

• Apply suggestions from code review

  aliberts

  committed
  23f6c87

  Copy full SHA for 23f6c87

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  3b5af7e

  Copy full SHA for 3b5af7e

• Refactor pusht_zarr

  aliberts

  committed
  6ad84a6

  Copy full SHA for 6ad84a6

Commits on Nov 26, 2024

• Remove comment

  aliberts

  committed
  49bdcc0

  Copy full SHA for 49bdcc0

• Activate end-to-end tests

  aliberts

  committed
  56c01a2

  Copy full SHA for 56c01a2

• Merge remote-tracking branch 'origin/main' into user/aliberts/2024_09_25_reshape_dataset

  aliberts

  committed
  691d39a

  Copy full SHA for 691d39a

• Comment

  aliberts

  committed
  2945dca

  Copy full SHA for 2945dca

Commits on Nov 28, 2024

• Remove default root data dir, add fixes

  aliberts

  committed
  2556960

  Copy full SHA for 2556960

• Remove DATA_DIR references

  aliberts

  committed
  d6b4429

  Copy full SHA for d6b4429

• Remove remaining DATA_DIR reference

  aliberts

  committed
  82ff776

  Copy full SHA for 82ff776

• Remove commented code

  aliberts

  committed
  ea5009e

  Copy full SHA for ea5009e

Commits on Nov 29, 2024

• Remove unused arg

  aliberts

  committed
  0cb0af0

  Copy full SHA for 0cb0af0