0.12.0

0.12.0

New Features

Type Checking Compatibility

Type checker-compatible annotations are finally here! The old dataclass annotation way would cause type checkers to report errors when you try assigning an int to a field marked as UInt8. You can now get around this by using typing.Annotated.

The old way:

@binobj.dataclass
class MyStruct(binobj.Struct):
    foo: UInt16
    bar: StringZ = ""
    sub_struct: MyOtherStruct
    baz: Timestamp64(signed=False)

To save you headaches with MyPy, the same struct can be declared like so:

@binobj.dataclass
class MyStruct(binobj.Struct):
    foo: Annotated[int, UInt16]
    bar: Annotated[str, StringZ] = ""
    sub_struct: MyOtherStruct    # Note no change necessary here
    baz: Annotated[datetime, Timestamp64(signed=False)]

Full Mapping-like Behavior

Two new struct wrappers, StructMappingProxy and MutableStructMappingProxy, allow you to use a Struct exactly as you would a Mapping or MutableMapping:

ro_proxy = binobj.StructMappingProxy(struct_instance)
assert ro_proxy.struct is struct_instance  # Original struct still available

for key, value in ro_proxy.items():
    print(f"{k!r} = {v!r}")

cm = collections.ChainMap({}, binobj.MutableStructMappingProxy(struct_instance))

These are typed as Mapping[str, Any] and MutableMapping[str, Any], respectively.

Other New Features

Now testing on Python 3.13-rc1.

Deprecations

• Support for Python 3.9 will be removed in the next backwards-incompatible release.
• Using Field instances as bare annotations is deprecated; use typing.Annotated instead.

Bugfixes

• On Python ≤ 3.10, using Field instances as type annotations completely broke if deferred annotation evaluation was enabled with from __future__ import annotations. This can now be worked around by using Annotated, or with normal field assignment.
• When reading fixed data, if exact was true the error message would be one byte off when saying how much it expected to read.
• Better type annotations for containers.
• Error messages now use __qualname__ for classes, instead of __name__. This will only change the output of nested classes.

Breaking Changes

Dropped support for EOL Python 3.7 and 3.8.

Other Changes

• Refactored Struct class initialization and pushed it into a factory method on StructMetadata. The eventual goal is to completely eliminate the need for inheriting Struct.
• Switched from Black to Ruff.
• Minimum version of typing_extensions is now 4.4.
• Upgraded test dependencies.

0.11.4

Bugfixes

Add all bugfixes from 0.11.2. (They were accidentally removed in 0.11.3 due to a borked merge.)

New Features

• Added support for Python 3.12.
• The binobj package now exports the binobj.pep526.dataclass decorator directly.

Other Changes

• Stricter linting, remove dead code and make tests prettier.
• Upgrade test dependencies.
• Add typing-extensions and importlib-metadata as explicit dependencies. We were relying on other libraries to install them before.

0.11.3

0.11.2 was broken and has been yanked. The following fixes are present in 0.11.3.

Bugfixes

• Don't assume __doc__ always exists. (This is removed when Python is run with optimization flags.)
• Fix wrong name in default error message for CannotDetermineNullError. It was using the repr of the exception instead of the name of the field.
• Throw exceptions instead of crashing when an invalid decider is passed to a Union field.

Other Changes

• Test on PyPy 3.10
• Un-deprecate the Field.computes decorator.
• Remove dead code.

0.11.1

Bugfixes

• Fixed #38. Computing the length of a struct when it has a nested variable-length struct no longer crashes. Thank you @kirill-varchenko for the bug report.
• Fixed a bunch of wrong typing in the Union field as well as in the prototype of Field._do_dump(). Nested was also fixed.
• to_dict() crashed in certain cases on Python 3.7 and 3.8 due to implementation details of ChainMap in those versions. This has been fixed.

Deprecations

to_dict() currently swallows all binobj exceptions. Because this can hide bugs in a field implementation or other some other problem, this behavior is deprecated for all exceptions not directly related to serialization. A future release will only catch serialization-related errors; until then, a warning is issued but the exception is still swallowed.

Other Changes

Nested is now annotated as type-invariant, meaning Nested(X) will only load and dump instances of X. Before, it was erroneously annotated such that X and any subclass of it were valid. This is now in line with its intended usage. Static type checking for users' code may break but the class still behaves the same.

0.11.0

New Features

• New Field!: UUID4: Store a UUID4 in four different formats: variant 1 (the most common), Microsoft format (variant 2), the canonical string representation, or as a hexadecimal string.
• Official support for CPython 3.10, 3.11, PyPy 3.8, and PyPy 3.9.
• New exception BuggyFieldImplementationError to give better information to people who are implementing custom fields.
• Field implementations now no longer need to handle None in _size_for_value(). The method is now guaranteed to never be called with None.
• Field.compute_value_for_dump() gained a context argument, so you can now pass your context object in and it'll get passed through to the present check.

Breaking Changes

• Dropped support for Python 3.5 and 3.6 as per the deprecation policy (when Pip drops support for a Python version, I no longer guarantee support).
• Switching to pyproject.toml breaks support for Pip older than 19.0. I consider this acceptable because Pip 19.0 is over four years old at this point, and predates the sunsetting of 3.5 and 3.6.

Deprecations

Callable Defaults

Specifying a callable as a default argument was a terrible idea in retrospect. Even at the time in the release notes I said it "[...] looks confusing and is not recommended. This may throw an exception in the future if I decide it's too egregious."

Thus, please don't do this:

    @binobj.dataclass
    class MyStruct(binobj.Struct):
        username: StringZ = getpass.getusername

For now it will only issue a DeprecationWarning, but will crash in the future. Instead, use the new factory argument:

    @binobj.dataclass
    class MyStruct(binobj.Struct):
        username: StringZ(factory=getpass.getusername)

Mixing Naive/Aware Timestamps

Passing a timezone-aware timestamp to a Timestamp field that was naive, or passing a naive timestamp to a timezone-aware Timestamp field is deprecated. In the future, doing so will trigger an error.

Bugfixes

• Field.compute_value_for_dump() now returns NOT_PRESENT in call cases where the field should be omitted. Before, it only checked to see if the field should be omitted if a value wasn't explicitly set for that field.

• Circular references involving a computed field are now handled properly and won't crash with a MissingRequiredValueError.

• Fixed a bug where StringZ would trigger a stack overflow when dumping if all the following conditions were met:

  • The default value was None (either from default or factory)
  • The default was used
  • null_value was set to a string (not bytes).

Other Changes

• Attempting to change the name of an already-bound field will throw a ConfigurationError.
• If a field depends on another field to give its size, and that other field has a non-integer value, a TypeError is thrown upon loading or dumping.
• Codec information for variable-length integers now uses dataclasses instead of dicts. This gives us the ability to add in stricter typing information and remove a number of typecasts.
• Endianness arguments to functions now have stricter, more accurate typing. If you use
  MyPy, it may get angry.

Internal Changes

• Removed struct metaclass in favor of __init_subclass__().
• Dropping 3.5 support allowed for some changes to the type declarations so they're more accurate.
• Minor code hygiene changes.
• Removed attrs as a dependency in favor of dataclasses from the Python standard library.
• Moved to Poetry for handling installation.

0.10.5

Released 2021-11-20

Bugfixes

Installation breaks on Python 3.5 due to this bug in more-itertools. I've put a new upper bound on that dependency (only on 3.5) to fix this. I plan on dropping support for 3.5 in the 0.11.0 release, but I figured it's the polite thing to do and doesn't require much effort.

Other Changes

Loosened upper bound on the typing-inspect dependency to allow the latest versions.

Internal Changes

• Bumped test dependencies.
• Fixed Tox envs for linting and type checking, which were silently broken.
• Added some more code comments.

0.10.4

Released 2021-09-04

Deprecations

• Passing a callable as a default value in dataclass mode is deprecated. It was a bad idea to begin with. I even expressed misgivings in the comments when I wrote it.
• Using an instance of a Field as a type annotation is also deprecated, as it breaks in Python 3.10. The next version of binobj will move away from a Marshmallow style and more towards Python 3.7-esque dataclasses.

Bugfixes

Fixed incorrect type annotation for the return value of the present callback to Field.

Other Changes

Fixed build status badge in README.

0.10.2

Released 2021-05-08

New Features

The documentation is now available online!

Bugfixes

• Creating a dataclass with no fields now throws NoDefinedFieldsError. Unfortunately, because dataclass is a decorator that executes after the class is created, we can't do the same with normal assigned fields.
• Fixed wrong docstring for NoDefinedFieldsError that said it was thrown when only assignments were used on a class marked with dataclass. It was supposed to be a MixedDeclarationsError.
• Fixed formatting and broken links in docstrings.

Internal Changes

• Upgraded test dependencies.
• Added a few more flake8 plugins for stricter linting of things.
• Set minimum test coverage to 95%.
• Split out packages used for unit tests and linting into separate files, as we were installing a bunch of stuff for the unit tests that weren't needed.
• Standardized order and placement of the "New in version X" and "Changed in version X" directives. They are now always at the bottom of the thing being documented, in chronological order.

0.10.1

The dev, test, and lint extras were removed from the package dependencies and instead moved into separate requirements files. The original structure was confusing sites like libraries.io which were including test dependencies as part of the normal package dependencies. It's not considered a breaking change since those extras were never documented or part of the public interface.

The 0.10.x series is the last release supporting Python 3.5.

Python 3.5 went EOL at the beginning of this year and is no longer maintained. Furthermore, I plan on moving from a Marshmallow-like structure to something closer to Python's dataclasses, and Python 3.5 support would make that very difficult.

0.10.0

Released 2021-01-12

New Features

Customize Struct Creation!

You can customize how a Struct is created by nesting a class named Meta into it, like so:

    class MyStruct(binobj.Struct):
        class Meta:
            # Options in here

        # Define your fields out here as before

For now we only support passing fallback values for arguments not passed to a field, such as defaults, null values, etc.

Before...

    class Person(binobj.Struct):
        first_name = StringZ(encoding="ibm500")
        middle_name = StringZ(encoding="ibm500")
        last_name = StringZ(encoding="ibm500")
        id = StringZ(encoding="ascii")

Now, you can pass a dictionary in a nested class called Meta with the names of the argument you wish to override and the value:

    class Person(binobj.Struct):
        class Meta:
            argument_defaults = {
                # All strings will use EBCDIC as the text encoding if they don't
                # get passed an explicit value.
                "encoding": "ibm500"
            }

        first_name = StringZ()
        middle_name = StringZ()
        last_name = StringZ()
        id = StringZ(encoding="ascii")

You can use the field class names as a prefix to provide different values for different field types. Suppose I want all integers to have a default value of 0, and all strings to have a default value of "":

    class Person:
        class Meta:
            argument_defaults = {
                "encoding": "ibm500",
                "StringZ__default": "",
                "Int8__default": 0
            }

        id = StringZ(encoding="ascii")
        first_name = StringZ()
        middle_name = StringZ()
        last_name = StringZ()
        age = Int8()
        num_children = Int8()

Bugfixes

• Fixed wrong type annotations for validate and present arguments of Field.
• Fixed outdated docstring for null_value argument of Field.