Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validation #867

Merged
Merged
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
8cb6522
add validation script
xiaohanzhan-db Dec 23, 2023
c59c11f
update
xiaohanzhan-db Jan 3, 2024
66f34eb
change token count function
Jan 3, 2024
2cd387b
reorganize cells
Jan 5, 2024
3eac3bf
Add unit tests
xiaohanzhan-db Jan 5, 2024
d2d9767
Add a printout for CPT
xiaohanzhan-db Jan 6, 2024
be25591
update question
xiaohanzhan-db Jan 6, 2024
4651be7
Add questions
Jan 8, 2024
5cd6a94
Fix lints
xiaohanzhan-db Jan 8, 2024
8e2c1f4
Merge branch 'main' into validation
XiaohanZhangCMU Jan 8, 2024
e6e4a81
update format
xiaohanzhan-db Jan 8, 2024
34c5690
Merge branch 'validation' of github.com:XiaohanZhangCMU/llm-foundryX …
xiaohanzhan-db Jan 8, 2024
1668b9a
update
xiaohanzhan-db Jan 8, 2024
2219135
nb source
xiaohanzhan-db Jan 8, 2024
86c6e87
add validation script
xiaohanzhan-db Dec 23, 2023
678b376
update
xiaohanzhan-db Jan 3, 2024
297e057
change token count function
Jan 3, 2024
09d0ebb
reorganize cells
Jan 5, 2024
460df65
Add unit tests
xiaohanzhan-db Jan 5, 2024
3ffd200
Add a printout for CPT
xiaohanzhan-db Jan 6, 2024
9362886
update question
xiaohanzhan-db Jan 6, 2024
898e5ac
Add questions
Jan 8, 2024
a4bef71
Fix lints
xiaohanzhan-db Jan 8, 2024
4ca9cc6
update format
xiaohanzhan-db Jan 8, 2024
d636a0f
update
xiaohanzhan-db Jan 8, 2024
827d155
nb source
xiaohanzhan-db Jan 8, 2024
6bbf3fc
Remove license insert for validation notebook
xiaohanzhan-db Jan 8, 2024
4f6a4fb
Merge branch 'validation' of github.com:XiaohanZhangCMU/llm-foundryX …
xiaohanzhan-db Jan 8, 2024
5966b68
Add validation utils
xiaohanzhan-db Jan 11, 2024
da17813
Merge branch 'main' into validation
xiaohanzhan-db Jan 11, 2024
a7c36bc
Minor cleanups (#858)
mvpatel2000 Jan 11, 2024
55e4626
update utils/__init__.py to include extra validation functions
xiaohanzhan-db Jan 11, 2024
45544a1
update notebook
Jan 11, 2024
d2797b3
update
xiaohanzhan-db Jan 11, 2024
019da77
Merge branch 'validation' of github.com:XiaohanZhangCMU/llm-foundryX …
xiaohanzhan-db Jan 11, 2024
756fdae
update
xiaohanzhan-db Jan 11, 2024
6de8c37
Read UC delta table (#773)
XiaohanZhangCMU Jan 11, 2024
93b5a9f
Add download remote function to util
xiaohanzhan-db Jan 11, 2024
b47c878
update
xiaohanzhan-db Jan 11, 2024
fa8f3d9
remove fused layernorm (#859)
mvpatel2000 Jan 11, 2024
13fd34c
update
xiaohanzhan-db Jan 11, 2024
610f669
update
xiaohanzhan-db Jan 11, 2024
9f2e51b
update
xiaohanzhan-db Jan 11, 2024
ec68f10
update
xiaohanzhan-db Jan 11, 2024
1e76068
update
xiaohanzhan-db Jan 11, 2024
7a5c164
update
xiaohanzhan-db Jan 11, 2024
e76038f
Merge branch 'main' into validation
xiaohanzhan-db Jan 11, 2024
5b413f5
update
xiaohanzhan-db Jan 11, 2024
a1aa31f
update
xiaohanzhan-db Jan 11, 2024
d24fd5c
update
xiaohanzhan-db Jan 11, 2024
da3bea1
Remove hardcoded combined.jsonl with a flag (#861)
XiaohanZhangCMU Jan 12, 2024
936e3a1
bump (#828)
mvpatel2000 Jan 12, 2024
55fce37
Add dask and dataframe_to_mds
xiaohanzhan-db Jan 12, 2024
86e2412
update
xiaohanzhan-db Jan 12, 2024
bbfec65
update
xiaohanzhan-db Jan 12, 2024
b2e880d
update
xiaohanzhan-db Jan 12, 2024
596443a
update
xiaohanzhan-db Jan 12, 2024
ea65187
Add notebook
xiaohanzhan-db Jan 12, 2024
378a4e0
update
xiaohanzhan-db Jan 12, 2024
af6e9aa
update
Jan 12, 2024
4e286ec
remove script and tests, keep notebook
xiaohanzhan-db Jan 12, 2024
09c4892
update
xiaohanzhan-db Jan 12, 2024
c82da6c
update
xiaohanzhan-db Jan 12, 2024
e5f83cc
update
xiaohanzhan-db Jan 12, 2024
17d2b9f
update
xiaohanzhan-db Jan 12, 2024
6579d55
Merge branch 'main' into validation
xiaohanzhan-db Jan 12, 2024
56308ff
Merge branch 'byod/data_validation' into validation
XiaohanZhangCMU Jan 12, 2024
6517a30
Always initialize dist (#864)
mvpatel2000 Jan 12, 2024
4daa324
updated notebook
Jan 12, 2024
b809691
Merge branch 'main' into validation
xiaohanzhan-db Jan 12, 2024
8b75f94
remove scripts keep notebook
xiaohanzhan-db Jan 12, 2024
99bf2cd
merge with byod/data_validation
xiaohanzhan-db Jan 12, 2024
22014d6
update notebook. rephrase.
Jan 12, 2024
d9f28aa
merged
xiaohanzhan-db Jan 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading