Code

Add data loader for HF oasst1#2951

Code

Merged

andreaskoepf merged 17 commits intoLAION-AI:mainLAION-AI/Open-Assistant:mainfrom grgau:2828_add_oasst1_data_loadergrgau/Open-Assistant:2828_add_oasst1_data_loaderCopy head branch name to clipboardJun 13, 2023

+179-37

Commits

Commits on May 8, 2023

• build: add to config files params for use hf model

  

  grgau

  committed
  e2e8b16

  Copy full SHA for e2e8b16

• docs: update README with how to use HF dataset

  

  grgau

  committed
  6c4c533

  Copy full SHA for 6c4c533

• feat: HF data loader for oasst

  

  grgau

  committed
  bca2b6d

  Copy full SHA for bca2b6d

• Feature/add rm mode to dataset entry (LAION-AI#2867)

  Show description for 1bb1609
  

  2 peopleauthored and

  grgau

  committed
  1bb1609

  Copy full SHA for 1bb1609

• update dataset entry to support list of list type (LAION-AI#2879)

  Show description for 9a3edb2
  

  CloseChoice

  authored and

  grgau

  committed
  9a3edb2

  Copy full SHA for 9a3edb2

Commits on May 10, 2023

• style: pre-commit lint fixes

  

  grgau

  committed
  c94a5a0

  Copy full SHA for c94a5a0

• WIP(feat): data loader from hugging face dataset, needs to rebuild message trees from flat dataset messages

  

  grgau

  committed
  771fcb0

  Copy full SHA for 771fcb0

• Merge branch 'main' of https://github.com/LAION-AI/Open-Assistant into 2828_add_oasst1_data_loader

  

  grgau

  committed
  a07a53b

  Copy full SHA for a07a53b

Commits on May 21, 2023

• feat: rebuild trees from hugging face dataset

  

  grgau

  committed
  c4cae88

  Copy full SHA for c4cae88

• solve conflicts after pull

  

  grgau

  committed
  91009b3

  Copy full SHA for 91009b3

• fix pre-commit issues

  

  grgau

  committed
  6316fe1

  Copy full SHA for 6316fe1

Commits on Jun 13, 2023

• restore original ExportMessageNode emojis & labels

  andreaskoepf

  committed
  2cdf837

  Copy full SHA for 2cdf837

• simplify label parsing

  andreaskoepf

  committed
  d8c0b45

  Copy full SHA for d8c0b45

• Support partial dataset loading, cleanup

  andreaskoepf

  committed
  1f4d942

  Copy full SHA for 1f4d942

• minor changes in README

  andreaskoepf

  committed
  2e8c1ec

  Copy full SHA for 2e8c1ec

• minor README changes

  andreaskoepf

  committed
  1f0e7e1

  Copy full SHA for 1f0e7e1

• minor config changes

  andreaskoepf

  committed
  c3e7717

  Copy full SHA for c3e7717