Code

Stream ModelOutputs#29545

Code

Closed

dmarx wants to merge 49 commits intohuggingface:mainhuggingface/transformers:mainfrom coreweave:dmarx.output_streamercoreweave/transformers:dmarx.output_streamerCopy head branch name to clipboard

Commits

Commits on Feb 29, 2024

• skeleton OutputStreamer

  dmarx

  committed
  f901ee0

  Copy full SHA for f901ee0

• hacky but passes tests

  dmarx

  committed
  57a77d1

  Copy full SHA for 57a77d1

Commits on Mar 1, 2024

• skeleton OutputIteratorStreamer, notes

  dmarx

  committed
  b34c0b2

  Copy full SHA for b34c0b2

• fleshed out OutputIteratorStreamer w test case

  dmarx

  committed
  5eb0e69

  Copy full SHA for 5eb0e69

• token_id streaming passes

  dmarx

  committed
  ff35e2b

  Copy full SHA for ff35e2b

• contrastive passes token_id test

  dmarx

  committed
  76ed5c2

  Copy full SHA for 76ed5c2

• randomize test seed

  dmarx

  committed
  3885ea1

  Copy full SHA for 3885ea1

• greedy scores pass test

  dmarx

  committed
  2958099

  Copy full SHA for 2958099

Commits on Mar 2, 2024

• ensure we're building the output incrementally

  dmarx

  committed
  976eb23

  Copy full SHA for 976eb23

Commits on Mar 4, 2024

• multinom sampling outputs match

  dmarx

  committed
  cabb5fe

  Copy full SHA for cabb5fe

Commits on Mar 5, 2024

• throw error if on_ready called on empty buffer

  dmarx

  committed
  9d075c9

  Copy full SHA for 9d075c9

Commits on Mar 6, 2024

• enforce list(values) in on_ready()

  dmarx

  committed
  ba48d71

  Copy full SHA for ba48d71

• POC output_constructor

  dmarx

  committed
  108d3c2

  Copy full SHA for 108d3c2

• DRY tests following on_ready() fix

  dmarx

  committed
  02ec905

  Copy full SHA for 02ec905

• fix function spelling

  dmarx

  committed
  ebe4798

  Copy full SHA for ebe4798

• moved output_constructor to class method

  dmarx

  committed
  528d79a

  Copy full SHA for 528d79a

• rename output_constructor -> _prepare_output

  dmarx

  committed
  97812bf

  Copy full SHA for 97812bf

• integrating _prepare_output

  dmarx

  committed
  2c03ae6

  Copy full SHA for 2c03ae6

• finished integrating _prepare_output

  dmarx

  committed
  64732bb

  Copy full SHA for 64732bb

• placeholder args for attention/hidden streaming

  dmarx

  committed
  e903f04

  Copy full SHA for e903f04

• cleanup

  dmarx

  committed
  ca4dc8d

  Copy full SHA for ca4dc8d

• explicit GenerateEncoderDecoderOutput support

  dmarx

  committed
  6e17323

  Copy full SHA for 6e17323

Commits on Mar 7, 2024

• parameterized OutputStreamer tests

  dmarx

  committed
  3e2ff84

  Copy full SHA for 3e2ff84

• fixed tests

  dmarx

  committed
  256884f

  Copy full SHA for 256884f

• assert same types on field, cleanup

  dmarx

  committed
  75bf3d7

  Copy full SHA for 75bf3d7

• tuple-of-tensors output type parity

  dmarx

  committed
  673837b

  Copy full SHA for 673837b

• cleanup, test type consistency yielded of stream

  dmarx

  committed
  8dd7973

  Copy full SHA for 8dd7973

• test emits helpful info on failure

  dmarx

  committed
  cd644fb

  Copy full SHA for cd644fb

• output_attentions streaming for greedy decoding

  dmarx

  committed
  2ad0ead

  Copy full SHA for 2ad0ead

• fix skipped arguments

  dmarx

  committed
  e523c53

  Copy full SHA for e523c53

• 'fixed' tests, but now very messy

  dmarx

  committed
  b6c2dc1

  Copy full SHA for b6c2dc1

• cleaned up

  dmarx

  committed
  7ce491f

  Copy full SHA for 7ce491f

Commits on Mar 8, 2024

• test checks all output attrs but past_key_values

  dmarx

  committed
  290b9cb

  Copy full SHA for 290b9cb

• test over model varieties

  dmarx

  committed
  41cb5e2

  Copy full SHA for 41cb5e2

• moved instantiation closer to use

  dmarx

  committed
  a7a8a93

  Copy full SHA for a7a8a93

• attention for multinom decoding

  dmarx

  committed
  76ad30d

  Copy full SHA for 76ad30d

• contrastive working after multinom supported

  dmarx

  committed
  21adb61

  Copy full SHA for 21adb61

• attention streaming passes all test cases

  dmarx

  committed
  916bf43

  Copy full SHA for 916bf43

• add assistive decoding to test parameterization

  dmarx

  committed
  0b28bab

  Copy full SHA for 0b28bab

• draft streaming assisted, exceeds max tokens

  dmarx

  committed
  b89dc83

  Copy full SHA for b89dc83

• back out assisted decoding changes for the moment

  dmarx

  committed
  1e02df6

  Copy full SHA for 1e02df6

• tuple consistency

  dmarx

  committed
  965c55c

  Copy full SHA for 965c55c

• refactor tests

  dmarx

  committed
  6c7422e

  Copy full SHA for 6c7422e

• refactored tests, new issues w contrastive

  dmarx

  committed
  314c5e1

  Copy full SHA for 314c5e1

• added note

  dmarx

  committed
  33fded0

  Copy full SHA for 33fded0

• suppress contrastive tests ftm

  dmarx

  committed
  2a4f657

  Copy full SHA for 2a4f657

• setting scores=logits, contrastive passes

  dmarx

  committed
  eb0d567

  Copy full SHA for eb0d567

Commits on Mar 11, 2024

• block debugging import (lovely_tensors)

  dmarx

  authored
  014c8f5

  Copy full SHA for 014c8f5

Commits on Mar 12, 2024

• Merge branch 'main' into dmarx.output_streamer

  dmarx

  authored
  9358cd0

  Copy full SHA for 9358cd0