Add TF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING
#18469
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
ydshieh
changed the title
TF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING
| @@ -1388,6 +1399,9 @@ def test_loss_computation(self): | |||
|
|
|||
| self.assertTrue(loss.shape.as_list() == expected_loss_size or loss.shape.as_list() == [1]) | |||
|
|
|||
| def check_keras_fit_results(self, val_loss1, val_loss2, atol=1e-2, rtol=1e-3): | |||
There was a problem hiding this comment.
For TFSegformerForSemanticSegmentation, we need higher tolerances. See the comment for the change in that model file.
Adding check_keras_fit_results here to avoid overwrite test_keras_fit entirely.
| @@ -336,6 +337,9 @@ def recursive_check(tuple_object, dict_object): | |||
| def test_dataset_conversion(self): | |||
| super().test_dataset_conversion() | |||
|
|
|||
| def check_keras_fit_results(self, val_loss1, val_loss2, atol=2e-1, rtol=2e-1): | |||
There was a problem hiding this comment.
Use higher tolerance for TFSegformerForSemanticSegmentation: this model
-has BatchNormalization layer,
- also have several dropout layers,
- as well as a layer
TFSegformerDropPathwhich has random operation during training.
These factors together cause the statistic of moving_average and moving_variance different, and we have larger validation loss.
There was a problem hiding this comment.
I have found that we need larger tolerances for dense prediction tasks like semantic segmentation. It was the case for TFData2VecVisionForSemanticSegmentation as well.
| hidden_states = self.dense(hidden_states) | ||
| hidden_states = self.dropout(hidden_states) | ||
| hidden_states = self.dropout(hidden_states, training=training) |
There was a problem hiding this comment.
We still pass this argument, right?
There was a problem hiding this comment.
We can avoid it since these are supposed to be set automatically during training by the Keras engine. But I find that explicitly specifying it gives me mental peace.
There was a problem hiding this comment.
Thank you for the information!
| @@ -773,9 +773,9 @@ def call(self, encoder_hidden_states): | |||
| all_hidden_states += (encoder_hidden_state,) | |||
|
|
|||
| hidden_states = self.linear_fuse(tf.concat(all_hidden_states[::-1], axis=-1)) | |||
| hidden_states = self.batch_norm(hidden_states) | |||
| hidden_states = self.batch_norm(hidden_states, training=training) | |||
|
Test failures are |
|
Thank you, @ydshieh for this. I appreciate the help. |
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
What does this PR do?
The original goal is to fix
TFSegformerModelTest.test_keras_fit, but it ends up the followingTF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPINGto some__init__files.trainingarguments in a few layers forTFSegformerModel_prepare_for_classto deal with 2 more image tasksTFData2VecVisionForSemanticSegmentationloss: we need batch dimension (without this,test_dataset_conversionfailed - this was previously skipped due to the lack of labels)