Further fix for encoder_only

[zphang]

Handles cases of loading across different taskmodel configurations

[codecov]

Codecov Report

Merging #1242 (58fefbd) into master (1f66050) will not change coverage.
The diff coverage is 0.00%.

@@           Coverage Diff           @@
##           master    #1242   +/-   ##
=======================================
  Coverage   56.49%   56.49%           
=======================================
  Files         147      147           
  Lines       10597    10597           
=======================================
  Hits         5987     5987           
  Misses       4610     4610           

Impacted Files	Coverage Δ
jiant/proj/main/modeling/model_setup.py	36.68% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1f66050...58fefbd. Read the comment docs.

[jeswan]

Thanks!