Update to Transformers v4.3.3 #1266
Conversation
jeswan
force-pushed
the
js/maint/update_transformers_v4.2.1
branch
from
e24c84a to
20913e4
Compare
Codecov Report
@@ Coverage Diff @@
## js/feature/easy_add_model #1266 +/- ##
=============================================================
+ Coverage 55.51% 55.56% +0.05%
=============================================================
Files 149 148 -1
Lines 10903 10884 -19
=============================================================
- Hits 6053 6048 -5
+ Misses 4850 4836 -14
Continue to review full report at Codecov.
|
|
… manager in models.
| attention_mask=input_mask, | ||
| output_hidden_states=output_hidden_states, | ||
| ) | ||
| return output.pooler_output, output.last_hidden_state, output |
There was a problem hiding this comment.
@zphang Do you know what the expected contents of other is? I'm currently looking at https://huggingface.co/transformers/_modules/transformers/modeling_bert.html. I think I need to make an additional change so that other contains the expected values (used here: https://github.com/nyu-mll/jiant/blob/master/jiant/proj/main/modeling/taskmodels.py#L240)
There was a problem hiding this comment.
Looking at this again...it looks like other should be the hidden_states. Let me know if I am missing anything.
There was a problem hiding this comment.
Hidden states/attention weights. One major benefit of using return_dict is to be able to do away with "other".
jeswan
force-pushed
the
js/maint/update_transformers_v4.2.1
branch
from
be42ff0 to
c897a62
Compare
jeswan
force-pushed
the
js/maint/update_transformers_v4.2.1
branch
from
c897a62 to
d6d989c
Compare
|
Hey guys, do you have a merge ETA for this PR? TIA |
|
@vblagoje Planning to get this PR into master within a week |
|
@vblagoje please checkout the feature branch |
* Update to Transformers v4.3.3 (#1266) * use default return_dict in taskmodels and remove hidden state context manager in models. * return hidden states in output of model wrapper * Switch to task model/head factories instead of embedded if-else statements (#1268) * Use jiant transformers model wrapper instead of if-else. Use taskmodel and head factory instead of if-else. * switch to ModelArchitectures enum instead of strings * Refactor get_output_from_encoder() to be member of JiantTaskModel (#1283) * refactor getting output from encoder to be member function of jiant model * switch to explicit encode() in jiant transformers model * fix simple runscript test * update to tokenizer 0.10.1 * Add tests for flat_strip() (#1289) * add flat_strip test * add list to test cases flat_strip * mlm_weights(), feat_spec(), flat_strip() if-else refactors (#1288) * moves remaining if-else statments to jiant model or replaces with model agnostic method * switch from jiant_transformers_model to encoder * fix bug in flat_strip() * Move tokenization logic to central JiantModelTransformers method (#1290) * move model specific tokenization logic to JiantTransformerModels * implement abstract methods for JiantTransformerModels * fix tasks circular import (#1296) * Add DeBERTa (#1295) * Add DeBERTa with sanity test * fix tasks circular import * [WIP] add deberta tests * Revert "fix tasks circular import" This reverts commit f924640. * deberta tests passing with transformers 6472d8 * switch to deberta-v2 * fix get_mlm_weights_dict() for deberta-v2 * update to transformers 4.5.0 * mark deberta test_export as slow * Update test_tokenization_normalization.py * add guide to add a model * fix test_expor_model tests * minor pytest fixes (add num_labels for rte, overnight flag fix) * bugfix for simple api notebook * bugfix for #1310 * bugfix for #1306: simple api notebook path name * squad running * 2nd bugfix for #1310: not all tasks have num_labels property * simple api notebook back to roberta-base * run test matrix for more steps to compare to master * save last/best model test fix Co-authored-by: Jesse Swanson <js11133Wnyu.edu>
Updating to Transformers v4.3.3 enables fast (Rust) tokenizers by default. Breaking changes found here are addressed in this PR including:
return_dict(as opposed to returned tuples)