Update tests for asdf 3.0 #1004
Conversation
| 0 warnings""" | ||
| else: | ||
| expected_out = f"""Certifying '{jwst_data}/invalid.asdf' (1/1) as 'ASDF' relative to context 'jwst.pmap' | ||
| instrument='UNKNOWN' type='UNKNOWN' data='{jwst_data}/invalid.asdf' :: Validation error : Does not appear to be a ASDF file. |
There was a problem hiding this comment.
This warning message was changed as asdf 3.0 no longer support FITS files: https://asdf.readthedocs.io/en/3.0.0/asdf/whats_new.html#removed-features
| Setting 'META.EXPOSURE.TYPE [EXP_TYPE]' = None to value of 'META.EXPOSURE.P_EXPTYPE [P_EXPTYP]' = 'FGS_IMAGE|FGS_FOCUS|FGS_INTFLAT|FGS_SKYFLAT|' | ||
| File written with dev version of asdf library: 2.0.0.dev1213 | ||
| Checking JWST datamodels. | ||
| ######################################## | ||
| 0 errors | ||
| 1 warnings""" | ||
| else: | ||
| expected_out = f""" Certifying '{jwst_data}/jwst_fgs_distortion_bad_asdf_version.asdf' (1/1) as 'ASDF' relative to context 'jwst_0591.pmap' |
There was a problem hiding this comment.
This file is invalid and no longer (in asdf 3.0) loads as a custom object (and instead is returned as a simple tagged dictionary). The main issue is that the file:
#ASDF 1.0.0
#ASDF_STANDARD 1.2.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0claims to be asdf-standard 1.2.0 which only supports core/asdf-1.1.0 not core/asdf-1.0.0:
https://asdf-standard.readthedocs.io/en/1.0.3/generated/asdf-format.org/core/core-1.2.0.html
The expected output was updated to capture this.
There was a problem hiding this comment.
This file contains an invalid block index. This does not prevent the file from loading (in asdf 3.0 or earlier versions). However, in asdf 3.0 a warning is issued (to indicate that the file has this issue):
AsdfBlockIndexWarning: Invalid block index contents for block 0, falling back to serial reading: Invalid block magic
The addition of this new warning conflicts with the strict output checking done within the crds tests. As the block index is optional, the block index was removed from this file so it will load in asdf 3.0 and older versions with no warnings.
There was a problem hiding this comment.
This file has an invalid block index and received the same fix as jwst_nircam_specwcs_1_4_0.asdf.
| @@ -177,11 +179,25 @@ def test_diff_asdf(capsys, jwst_shared_cache_state, jwst_data): | |||
|
|
|||
| output = capsys.readouterr().out | |||
|
|
|||
| assert output == """ ndarrays differ by contents | |||
| ndarrays differ by contents | |||
There was a problem hiding this comment.
The output of asdftool diff changed in asdf 3 and no longer matches this expected output. asdf-format/asdf#1672 fixes the issue that caused this test to raise an exception and improved the output to better match the other tree differences.
braingram
changed the title
This PR addresses a number of crds test and asdf 3.0 incompatibilities. See comments below for details.