Add Bloom preprocessor #1424
Add Bloom preprocessor #1424
Conversation
|
@mattdangerw The preset files needs to be updated on kaggle, because the sequence length of the model is 2048 not 512.I set default to 512 instead of 2048. I am sorry for that. here is a link to the variation : https://www.kaggle.com/models/mohamedabuelnasr/bloom/frameworks/keras/variations/bloom_560m_multi |
@abuelnasr0 all good! definitely expected to keep working out kinks as we keep landing a new model. Sounds like we just need to copy version 2 here -> https://www.kaggle.com/models/mohamedabuelnasr/bloom/frameworks/keras/variations/bloom_560m_multi into the keras team bloom https://www.kaggle.com/models/keras/bloom, is that right? |
| `y` and `sample_weight` are both optional, can have any format, and will be | ||
| passed through unaltered. | ||
|
|
||
| `BloomPreprocessor` forces the input to have only one segment, as BLOOM is |
There was a problem hiding this comment.
I think we can just delete this paragraph, it's not very helpful (I can delete elsewhere too). People are doing all sort of "multi segment" stuff with decoder models.
There was a problem hiding this comment.
just deleting this? or enabling the multisegment packing by replacing StartEndPacker with MultiSegmentPacker?
There was a problem hiding this comment.
oh just delete this comment for now!
There was a problem hiding this comment.
I deleted it but I left the ValueError that checks that the number of segments is only one. because otherwise the packer will raise an error.
@mattdangerw Yes that is right. The only changed file is config.json. where |
|
updated https://www.kaggle.com/models/keras/bloom, take a look! will merge this whenever the that last nit is done, no rush! |
No description provided.