Use pytorch-labs/tokenizers and remove tokenizer/ #3462

Job	Run time
gather-models-cpu	14s
compile-gguf (macos-14)	4m 27s
gather-models-gpu	11s
runner-aoti (16-core-ubuntu)	5m 54s
runner-et (16-core-ubuntu)	13m 21s
runner-aoti (macos-14-xlarge)	3m 38s
runner-et (macos-14-xlarge)	11m 29s
test-gguf-util (macos-14)	4m 30s
test-mps (macos-m1-stable)	1m 12s
test-mps-dtype (macos-m1-stable)	1m 56s
test-tinystories-executorch (16-core-ubuntu)	18m 35s
torchchat-command-load-test (macos-14)	1m 49s
test-tinystories-executorch (macos-14-xlarge)	14m 35s
test-torchao-experimental (macos-14-xlarge)	10m 56s
test-build-runner-et-android / linux-job	11m 46s
test-gpu-aoti-float16 (cuda, stories15M) / linux-job	13m 15s
test-gpu-aoti-float32 (cuda, stories15M) / linux-job	13m 18s
test-gpu-compile (cuda, stories15M) / linux-job	24m 7s
test-gpu-eval-sanity-check (cuda, stories15M) / linux-job	12m 44s
test-gpu-aoti-bfloat16 (cuda, stories15M) / linux-job	14m 15s
test-cpu-eval-sanity-check-float16 (x86_64, stories15M)	30m 25s
test-cpu-eval-sanity-check (x86_64, stories15M)	11m 40s
test-cpu-eval-sanity-check-float16 (aarch64, stories15M)	6m 44s
test-cpu-compile (x86_64, stories15M)	17m 4s
test-cpu-eval-sanity-check (aarch64, stories15M)	10m 59s
test-cpu-aoti (x86_64, stories15M)	18m 30s
test-cpu-compile (aarch64, stories15M)	9m 19s
test-cpu-aoti (aarch64, stories15M)	9m 9s
test-cpu-eval-sanity-check-float32 (x86_64, stories15M)	38m 9s
test-cpu-eval-sanity-check-float32 (aarch64, stories15M)	8m 37s
	5h 42m 48s

Provide feedback