Stored ImageStreams are invalid

[blschatz]

When not using compression, libaff4 generates the wrong metadata for the relevant ImageStream. Eg:

aff4:compressionMethod aff4:NullCompressor ;

In this case it should not store a compressionMethod. From the spec v1

"Where compression is used within the Image Stream, the object must have a property aff4:compressionMethod set to a resource identifying the compression algorithm. Where there is no compressionMethod set, it is assumed that chunks are stored."

[scudette]

Reading that paragraph does not seem to suggest that this is incorrect behavior:

1. If we use compression we have to set the property
2. If the property is not set then we assume the null compressor (i.e. the default setting is null compressor).

We are setting the null compressor explicitly which is not forbidden. It is also clearer since we are less open to interpretations of the default value.

[scudette]

Ahh sorry - this commit actually fixes issue #94

[scudette]

So the main issue here is that aff4:NullCompressor is actually not defined in the spec at all. Instead the spec suggests that if the property is omitted it indicates that null compression is implied: i.e. if the writer intends to indicate no compression was used they should not include the property at all.

We have a couple of options:

1. Change the code to never emit a property if there is no compression at all. This is similar to the evimetry behavior. Evimetry currently chokes on the aff4:NullCompressor as an unknown compressor and the only way to imply no compression is to not have the property at all.

2. Extend the Evimetry tool to understand aff4:NullCompressor as an extension to the spec, while at the same time introducing the aff4:NullCompressor in the 1.1 spec revision. I feel it is better to have a way to properly specify no compression rather than relying on an absence of specification and an implied default value.

3. For c-aff4 we will continue recognizing the aff4:NullCompressor anyway so we can continue reading images which we produced in the past and specify it. While if the property is not specified at all we set aff4:NullCompressor as the default - this way we can continue reading Evimetry images too.

In a sense the issue is that Evimetry is more strict in the type of images it is willing to read so we need to produce images that follow the exact interpretation of the spec to make sure our images are properly supported by Evimetry readers. We should continue to be flexible in the type of images we can read though and accept either way if we can.

I feel that the path of least resistance is for us to change to increase interoperability - moving forward Evimentry can read our stored images which it can not now and we can read our stored images as normal either way (old ones or new ones). The discussion about the 1.1 standard is orthogonal and separate.

[blschatz]

it turns out that we have been using the NullCompressor lexicon element for this in Evimetry all along - it is only missing from pyaff4. Given this is the case, I no longer think we need to take the spec so literally, and think it is safe to treat the NullCompressor as errata to the 1.0 spec and fix pyaff4.