A comprehensive library designed for declarative parsing of messages at the bit level. This tool allows developers to efficiently define and interpret the structure and content of binary data streams, facilitating tasks such as network protocol analysis, embedded system communication, and data serialization. By using a high-level declarative approach, the library simplifies the process of specifying complex message formats, enabling precise and reliable extraction of information from raw binary data. All you have to do is write a DTO that represents your message and annotate it. That's all. Boxon will take care of the rest for you.
If you want to use the parser straight away, just go here.
This project adheres to the Zero Bugs Commitment. |
---|
(Like Preon — currently not maintained anymore —, but the code is understandable, shorter, easier to extend, uses the more powerful (and maintained) SpEL expressions, and the documentation is really free...)
Boxon...
- Is easily extensible through the use of converters.
- Contains a minimal set of annotations capable of handling "all" the primitive data (aside
char
, but this could be easily handled with a converter). - Contains a set of special annotations that handles the various messages peculiarities (defining message header properties, conditional choosing of converter, or object while reading an array, skip bits, checksum, 'constant' assignments)
- Is capable of handle concatenation of messages, using the correct template under the hood.
- The template is selected in a clever way, i.e. selecting the one with the longest
start
parameter that matches the message. - Can handle SpEL expressions on certain fields, thus more powerful and simpler than Limbo1 (but less than janino, that has other problems).
- Can decode and encode data on the fly with a single annotated class (thus avoiding separate decoder and encoder going out-of-sync).
- Supported data types are:
- Integers: 1-, 4-, 8-, 16-, 32- and 64-bit signed and unsigned integers, little- or big-endian.
- Floating point numbers: 32- and 64-bit floating point values.
- Bit fields: bit fields with length from 1 to 2,147,483,647 bits.
- Strings: fixed-length, variable-length and zero terminated strings with various encodings.
- Arrays: fixed-length and variable-length arrays of built-in or user-defined element types.
- Objects: custom-type DTOs.
- Choices: supports integer keys.
- User defined types (arbitrary combination of built-in types)
- Has templates (annotated classes) that are not complex: they do not call each other uselessly complicating the structure (apart, necessarily, for
@BindObject
and a few others), no complicated chains of factories: it's just a parser that works. - Supports SLF4J.
- Hides the complexities of encoding and decoding, thus simplifying the changes to be made to the code due to frequent protocol changes.
- Can automatically scan and loads all the binding annotations and/or templates from a package.
1 Currently Limbo is merged with Preon... thus rendering Preon not only a parser, but also an evaluator, over-complicating and cluttering the code.
Boxon differs from Preon in...
- Does not have a generic
Bound
annotation: it uses converters instead. - Does not need the "native byte order" constant. This is because the bytes of the message have little chance to be generated from the very same machine that will parse them, what if a message consider 24 bits as an Integer? If the code should be portable and installed and run everywhere it should not rely on the native properties of any machine.
Moreover,
@Bound boolean visible;
is 1 bit- or 1 byte-length? - Does not have
BoundList
: since the message is a finite sequence of bytes, then any array is of finite length, and thus the standard java array ([]
) is sufficient. If someone wants aList
a converter can be used. - Does not rely on the type of the annotated variable (because of the existence of the converters); in fact, the annotation, eventually, serves the purpose to pass a predefined type of data to a converter.
For this reason too, there is no need for theInit
annotation, thus the annotated file can contain the least amount of data necessary for its decoding (moreover, this annotation has NOT the inverse operation -- so it seems to me... so it's pretty useless anyway). - (By personal experience) enumerations can have different representations, or change between a version and the next of a protocol, even inside the same protocol (!), so having an annotation that tells the value of a particular element of this enum is at least risky. So, for this reason, the
BoundEnumOption
is not present in this library. - Does read and write more than 64 bits at a time (
BitBuffer.readBits
)
Get them here.
In order to include Boxon in a Maven project add the following dependency to your pom.xml (Java 21 required).
Replace x.y.z
below int the version tag with the latest release number.
<dependency>
<groupId>io.github.mtrevisan</groupId>
<artifactId>boxon</artifactId>
<version>x.y.z</version>
</dependency>
You can get pre-built JARs (usable on JRE 21 or newer) from Sonatype.
- Basic annotations
- Special annotations
- Protocol description
- Configuration annotations
- Describer
- Extractor
- Generator
- Comparator
- How to write SpEL expressions
- How to extend the functionalities
- Digging into the code
- Examples
- Contributing
- Changelog
- version 7.0.0
- version 6.0.1
- version 6.0.0
- version 5.0.0
- version 4.0.0
- version 3.6.0
- version 3.5.1
- version 3.5.0
- version 3.4.0
- version 3.3.0
- version 3.2.0
- version 3.1.3
- version 3.1.2
- version 3.1.1
- version 3.1.0
- version 3.0.2
- version 3.0.1
- version 3.0.0
- version 2.1.2
- version 2.1.1
- version 2.1.0
- version 2.0.0
- version 1.1.0
- version 1.0.0
- version 0.0.2
- version 0.0.1
- version 0.0.0
- License
Here the build-in basic annotations are described.
You can use them as a starting point to build your own customized readers.
Here is a brief summary of the parameters (described in detail below) for each annotation.
condition | type | charset | terminator / consumeTerminator |
size | byteOrder | selectFrom / selectDefault |
validator | converter / selectConverterFrom |
||
---|---|---|---|---|---|---|---|---|---|---|
BindObject | ☑ | ☑ | ☑ | ☑ | ☑ | BindObject | ||||
BindArray | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | BindArray | |||
BindArrayPrimitive | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | BindArrayPrimitive | |||
BindList | ☑ | ☑ | ☑ | ☑ | ☑ | BindList | ||||
BindBitSet | ☑ | ☑ | ☑ | ☑ | ☑ | BindBitSet | ||||
BindInteger | ☑ | ☑ | ☑ | ☑ | ☑ | BindInteger | ||||
BindString | ☑ | ☑ | ☑ | ☑ | ☑ | BindString | ||||
BindStringTerminated | ☑ | ☑ | ☑ | ☑ | ☑ | BindStringTerminated |
condition | start / end |
charset | value | consumeTerminator | byteOrder | skipStart / skipEnd |
algorithm | valueDecode / valueEncode |
name | ||
---|---|---|---|---|---|---|---|---|---|---|---|
TemplateHeader | ☑ | ☑ | TemplateHeader | ||||||||
SkipBits | ☑ | ☑ | SkipBits | ||||||||
SkipUntilTerminator | ☑ | ☑ | ☑ | SkipUntilTerminator | |||||||
Checksum | ☑ | ☑ | ☑ | ☑ | Checksum | ||||||
Evaluate | ☑ | ☑ | Evaluate | ||||||||
PostProcess | ☑ | ☑ | ProcessField | ||||||||
ContextParameter | ☑ | ☑ | ContextParameter |
shortDescription | longDescription | minProtocol / maxProtocol |
start / end |
charset | terminator | unitOfMeasure | minValue / maxValue |
pattern | enumeration | defaultValue | radix | composition | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ConfigurationHeader | ☑ | ☑ | ☑ | ☑ | ☑ | ConfigurationHeader | ||||||||
ConfigurationSkip | ☑ | ☑ | ConfigurationSkip | |||||||||||
ConfigurationField | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | ConfigurationField | |||
CompositeConfigurationField | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | CompositeConfigurationField | ||||||
CompositeSubField | ☑ | ☑ | ☑ | ☑ | ☑ | CompositeSubField | ||||||||
AlternativeConfigurationField | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | AlternativeConfigurationField | |||||||
AlternativeSubField | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | ☑ | AlternativeSubField |
condition
: The SpEL expression that determines if this field has to be read.type
: the Class of the Object of the single element of the array (defaults toObject
).selectFrom
: the selection from which to choose the instance type using anObjectChoices
with a prefix of a predetermined length.selectDefault
: the default selection if none can be chosen fromselectFrom
(defaults tovoid.class
).selectFromList
: the selection from which to choose the instance type using anObjectChoicesList
with a prefix consisting in a terminated string.validator
: the Class of a validator (applied BEFORE the converter).converter
: the converter used to convert the read value into the value that is assigned to the annotated variable.selectConverterFrom
: the selection from which to choose the converter to apply (theconverter
parameter can be used as a default converter whenever no converters are selected from this parameter).
Reads a single Object.
This annotation is bounded to a variable.
class Version{
@BindInteger(size = "8")
public byte major;
@BindInteger(size = "8")
public byte minor;
public byte build;
}
@BindBitSet(size = "1", converter = BitSetToBooleanConverter.class)
private boolean versionPresent;
@BindObject(condition = "versionPresent", type = Version.class)
private Version version;
size
: the size of the array (can be a SpEL expression).
Reads an array of Objects.
This annotation is bounded to a variable.
class Version{
@BindInteger(size = "8")
public byte major;
@BindInteger(size = "8")
public byte minor;
public byte build;
}
@BindObject(type = Version.class)
@BindAsArray(size = "2")
private Version[] versions;
@BindInteger(size = "8")
private byte positionsCount;
@BindObject(type = Position.class,
selectFrom = @ObjectChoices(prefixLength = 8,
alternatives = {
@ObjectChoices.ObjectChoice(condition = "#prefix == 0", prefix = "0", type = PositionInvalid.class),
@ObjectChoices.ObjectChoice(condition = "#prefix == 1", prefix = "1", type = PositionAbsolute.class),
@ObjectChoices.ObjectChoice(condition = "#prefix == 2", prefix = "2", type = PositionRelative.class),
@ObjectChoices.ObjectChoice(condition = "#prefix == 3", prefix = "3", type = PositionSameAsPrevious.class)
}
),
converter = PositionsConverter.class)
@BindAsArray(size = "positionsCount")
private Position[] positions;
condition
: The SpEL expression that determines if this field has to be read.type
: the Class of primitive of the single element of the array.size
: the size of the array (can be a SpEL expression).byteOrder
: the byte order,ByteOrder.BIG_ENDIAN
orByteOrder.LITTLE_ENDIAN
(used for primitives other thanbyte
).validator
: the Class of a validator (applied BEFORE the converter).converter
: the converter used to convert the read value into the value that is assigned to the annotated variable.selectConverterFrom
: the selection from which to choose the converter to apply (theconverter
parameter can be used as a default converter whenever no converters are selected from this parameter).
Defines a parameter as an array.
This annotation is bounded to a variable.
@BindInteger(size = "8")
@BindAsArray(size = "2")
private byte[] array;
@BindBitSet(size = "1", converter = BitSetToBooleanConverter.class)
private boolean angularDataPresent;
@BindInteger(condition = "angularDataPresent", size = "8",
selectConverterFrom = @ConverterChoices(
alternatives = {
@ConverterChoices.ConverterChoice(condition = "angularDataPresent", converter = CrashDataWithAngularDataConverter.class),
@ConverterChoices.ConverterChoice(condition = "!angularDataPresent", converter = CrashDataWithoutAngularDataConverter.class)
})
)
@BindAsArray(size = "dataLength")
private BigDecimal[][] crashData;
None.
Defines a parameter as a list.
This annotation is bounded to a variable.
@BindObject(type = TestType3.class, selectFromList = @ObjectChoicesList(terminator = ',',
alternatives = {
@ObjectChoices.ObjectChoice(condition = "#prefix == '1'", prefix = "1", type = TestType4.class),
@ObjectChoices.ObjectChoice(condition = "#prefix == '2'", prefix = "2", type = TestType5.class)
}))
@BindAsList
private List<TestType3> value;
condition
: The SpEL expression that determines if this field has to be read.size
: the number of bits to read (can be a SpEL expression).validator
: the Class of a validator (applied BEFORE the converter).converter
: the converter used to convert the read value into the value that is assigned to the annotated variable.selectConverterFrom
: the selection from which to choose the converter to apply (theconverter
parameter can be used as a default converter whenever no converters are selected from this parameter).
Reads a java BitSet
.
This annotation is bounded to a variable.
@BindBitSet(size = "2")
private BitSet bitmap;
condition
: The SpEL expression that determines if this field has to be read.size
: the number of bits to read (can be a SpEL expression).byteOrder
: the byte order,ByteOrder.BIG_ENDIAN
orByteOrder.LITTLE_ENDIAN
.validator
: the Class of a validator (applied BEFORE the converter).converter
: the converter used to convert the read value into the value that is assigned to the annotated variable.selectConverterFrom
: the selection from which to choose the converter to apply (theconverter
parameter can be used as a default converter whenever no converters are selected from this parameter).
Reads a long number (primitive or not) or a BigInteger given the amount of bits.
This annotation is bounded to a variable.
@BindInteger(size = "3")
private BigInteger number;
@BindInteger(size = "Long.SIZE + 10")
private BigInteger number;
condition
: The SpEL expression that determines if this field has to be read.charset
: the charset to be interpreted the string into (SHOULD BE the charset name, e.g.UTF-8
(the default),ISO-8859-1
, etc.).size
: the size of the string (can be a SpEL expression).validator
: the Class of a validator (applied BEFORE the converter).converter
: the converter used to convert the read value into the value that is assigned to the annotated variable.selectConverterFrom
: the selection from which to choose the converter to apply (theconverter
parameter can be used as a default converter whenever no converters are selected from this parameter).
Reads a String.
This annotation is bounded to a variable.
@BindString(size = "4")
public String text;
condition
: The SpEL expression that determines if this field has to be read.charset
: the charset to be interpreted the string into (SHOULD BE the charset name, e.g.UTF-8
(the default),ISO-8859-1
, etc.).terminator
: the byte that terminates the string (defaults to\0
).consumeTerminator
: whether to consume the terminator (defaults totrue
).validator
: the Class of a validator (applied BEFORE the converter).converter
: the converter used to convert the read value into the value that is assigned to the annotated variable.selectConverterFrom
: the selection from which to choose the converter to apply (theconverter
parameter can be used as a default converter whenever no converters are selected from this parameter).
Reads a String.
This annotation is bounded to a variable.
@BindStringTerminated(terminator = ',')
public String text;
Here are described the build-in special annotations.
start
: an array of possible start sequences (as string) for this message (defaults to empty).end
: a possible end sequence (as string) for this message (default to empty).charset
: the charset to be interpreted thestart
andend
strings into (SHOULD BE the charset name, e.g.UTF-8
(the default),ISO-8859-1
, etc.).
Marks a DTO as an annotated message.
This annotation is bounded to a class.
@TemplateHeader(start = "+", end = "-")
private class Message{
...
}
condition
: The SpEL expression that determines if this field has to be read.value
: the number of bits to be skipped (can be a SpEL expression).
Skips size
bits.
If this should be placed at the end of the message, then a placeholder variable (that WILL NOT be read, and thus can be of any type) should be added.
This annotation is bounded to a variable.
@SkipBits("3")
@SkipBits("1")
@BindString(size = "4")
public String text1;
@SkipBits("10")
public Void lastUnreadPlaceholder;
condition
: The SpEL expression that determines if this field has to be read.value
: the byte that terminates the skip.consumeTerminator
: whether to consume the terminator (defaults totrue
).
Skips bits until a terminator is found.
If this should be placed at the end of the message, then a placeholder variable (that WILL NOT be read, and thus can be of any type) should be added.
This annotation is bounded to a variable.
@SkipUntilTerminator('x')
@BindString(size = "10")
public String text2;
@SkipUntilTerminator(value = '\0', consumeTerminator = false)
public Void lastUnreadPlaceholder;
condition
: The SpEL expression that determines if this field has to be read.byteOrder
: the byte order,ByteOrder.BIG_ENDIAN
orByteOrder.LITTLE_ENDIAN
(used for primitives other thanbyte
).skipStart
: how many bytes are to be skipped from the start of the message for the calculation of the checksum (defaults to 0).skipEnd
: how many bytes are to be skipped from the end of the message for the calculation of the checksum (default to 0).algorithm
: the algorithm to be applied to calculate the checksum.checksumSize
: the size of the checksum field in the stream (must match the size of the field type size).
Reads a checksum.
Compute the message checksum and compare it to the read variable once a message has been completely read. The amount of bytes read depends on the output size of the checksum algorithm.
This annotation is bounded to a variable.
@Checksum(skipStart = 4, skipEnd = 4, algorithm = CRC16CCITT_FALSE.class, checksumSize = 16)
private short checksum;
condition
: The SpEL expression that determines if this field has to be read.value
: The value to be assigned, or calculated (can be a SpEL expression).
Assign a constant, calculated value to a field.
Note that the evaluations are done AFTER parsing the entire message.
This annotation with runLast
to true
can be used to force a parameter to have a certain value, after perhaps using it to calculate other parameters.
Its effect cannot be undone.
This annotation is bounded to a variable.
@BindString(size = "4")
private String messageHeader;
@Evaluate("T(java.time.ZonedDateTime).now()")
private ZonedDateTime receptionTime;
@Evaluate("messageHeader.startsWith('+B')")
private boolean buffered;
//from the variable `deviceTypes` passed in the context
@Evaluate("#deviceTypes.getDeviceTypeName(deviceTypeCode)")
private String deviceTypeName;
condition
: The SpEL expression that determines if this field has to be processed (both in the decode and encode phases).valueDecode
: The value to be assigned, or calculated, at the decode phase (can be a SpEL expression).valueEncode
: The value to be assigned, or calculated, at the encode phase (can be a SpEL expression).
Assign a constant, or calculated value, to a field after all the other annotations are processed.
Note that the evaluations are done AFTER parsing the entire message in the decode phase, or BEFORE in the encode phase.
This annotation is bounded to a variable.
@BindString(size = "4")
//this annotation restores the '+ACK' value after all the fields and evaluations are done (this is because the evaluation of `buffered`
//requires the read value of `messageHeader`, and '+BCK' means a buffered message)
@PostProcess(condition = "buffered", valueDecode = "'+ACK'", valueEncode = "'+BCK'")
private String messageHeader;
name
: The name of the parameter that will be inserted into the context (both in the decode and encode phases).value
: The value to be assigned, or calculated, to the parameter with the given name (can be a SpEL expression).
Assigns a constant, or calculated value, to a parameter that will be added to the context before processing.
Note that the parameter ceases to live after the field is decoded/encoded.
This annotation is bounded to a variable.
@ContextParameter(name = "valueCondition", value = "#self.readValue")
@ContextParameter(name = "valueSize", value = "8")
@BindString(condition = "valueCondition", size = "valueSize")
private String value;
A description of the protocol can be obtained through the methods Describer.describeTemplates
and Describer.describeTemplate
.
These returns a JSON with a description of all the annotations of the loaded templates.
Example:
DeviceTypes<Byte> deviceTypes = DeviceTypes.<Byte>create()
.with((byte)0x46, "QUECLINK_GB200S");
Core core = CoreBuilder.builder()
.withContextPair("deviceTypes", deviceTypes)
.withContext(ParserTest.class.getDeclaredMethod("headerLength"))
.withDefaultCodecs()
.withTemplate(ACKMessageHex.class)
.build();
Describer describer = Describer.create(core);
List<Map<String, Object>> descriptions = describer.describeTemplates();
gives as output the following
{
"header": {
"start": ["+ACK"],
"charset": "UTF-8"
},
"fields": [
{
"charset": "UTF-8",
"size": "#headerLength()",
"name": "messageHeader",
"annotationType": "io.github.mtrevisan.boxon.annotations.bindings.BindString",
"fieldType": "java.lang.String"
},
{
"converter": "MessageTypeConverter",
"name": "messageType",
"annotationType": "io.github.mtrevisan.boxon.annotations.bindings.BindInteger",
"size": "8",
"fieldType": "java.lang.String"
},
{
"condition": "mask.hasProtocolVersion()",
"size": "2",
"converter": "io.github.mtrevisan.boxon.core.codecs.queclink.QueclinkHelper$VersionConverter",
"name": "protocolVersion",
"annotationType": "io.github.mtrevisan.boxon.annotations.bindings.BindArrayPrimitive",
"type": "byte",
"fieldType": "java.lang.String",
"byteOrder": "BIG_ENDIAN"
}
],
"context": {
"headerLength": "private static int io.github.mtrevisan.boxon.core.ParserTest.headerLength()",
"deviceTypes": ["QUECLINK_GB200S(0x46)"]
}
}
Configurations are mainly used to compose a message.
Note that currently only composing in ASCII text format is supported.
Firstly, load the configuration as shown below:
//add the custom codec to the list of available codecs
//(use one of the lines below)
core.withConfiguration(ConfigurationCustomTest.class); //loads the given configuration
core.withConfigurationsFrom(ConfigurationCustomTest.class); //loads all configuration from the package where the given class resides
Then, to retrieve all the possible protocol version boundaries, call
Configurator configurator = Configurator.create(core);
List<String> protocolVersionBoundaries = configurator.getProtocolVersionBoundaries();
Then, to retrieve all the messages for a given protocol version, simply call
Configurator configurator = Configurator.create(core);
List<Map<String, Object>> configurationMessages = configurator.getConfigurations("1.35");
Moreover, to compose a configuration message (remember to also load the codecs), call
Configurator configurator = Configurator.create(core);
Map<String, Object> configurationData = new HashMap<>();
configurationData.put("Weekday", "TUESDAY|WEDNESDAY");
...
Response<String, byte[]> composedMessage = configurator.composeConfiguration("1.20", "AT+", configurationData);
shortDescription
: a short description of the field, mandatory, used as an identifier (and thus must be unique for every configuration message).longDescription
: a more expressive description, optional.minProtocol
: minimum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).maxProtocol
: maximum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).start
: starting text of the message, mandatory, used as an identifier (and thus must be unique for every configuration message).end
: ending text of the message, optional.charset
: charset of the message, optional.
Marks a DTO as an annotated configuration message.
This annotation is bounded to a class.
@ConfigurationHeader(shortDescription = "A configuration message", start = "+", end = "-")
private class ConfigurationMessage{
...
}
minProtocol
: minimum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).maxProtocol
: maximum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).terminator
: the string that terminates the skip (defaults to empty string), optional.
Skips a field.
This annotation is bounded to a variable.
@ConfigurationSkip(minProtocol = "1.2", maxProtocol = "1.3")
@ConfigurationSkip
@ConfigurationField(shortDescription = "A field")
public String text;
shortDescription
: a short description of the field, mandatory, used as an identifier (and thus must be unique inside every configuration message).longDescription
: a more expressive description, optional.unitOfMeasure
: the unit of measure, optional (the format should follow UCUM/ISO 80000 standard).minProtocol
: minimum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).maxProtocol
: maximum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).minValue
: minimum value this field can assume, optional (alternative topattern
andenumeration
).maxValue
: maximum value this field can assume, optional (alternative topattern
andenumeration
).pattern
: regex pattern this field must obey, optional (alternative tominValue
/maxValue
andenumeration
).enumeration
: enumeration for this field, optional (alternative topattern
andminValue
/maxValue
). If the field is not an array, then each value of this enum is mutually exclusive.defaultValue
: default value, optional. If the variable is an array, then this field may represent anor
between values (e.g.ONE|TWO|THREE
), otherwise can be a single value (e.g.TWO
). If not present, then the field is mandatory.charset
: charset of the field (if string value), optional.radix
: radix of the number field when written to the message, optional.terminator
: the string that terminates the skip (defaults to empty string), optional.
Defines a field of the configuration message.
This annotation is bounded to a variable.
@ConfigurationField(shortDescription = "Report interval", terminator = ",", minProtocol = "1.19", maxProtocol = "1.20",
minValue = "90", maxValue = "86400", defaultValue = "3600", unitOfMeasure = "s")
public int motionlessReportInterval;
value
: a set of CompositeSubFieldshortDescription
: a short description of the field, mandatory, used as an identifier (and thus must be unique inside every configuration message).longDescription
: a more expressive description, optional.minProtocol
: minimum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).maxProtocol
: maximum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).pattern
: regex pattern this field must obey, optional.composition
: the FreeMarker pattern used to compose the field. The short description of each subfield is used as identifier.charset
: charset of the field (if string value), optional.terminator
: the string that terminates the skip (defaults to empty string), optional.
Defines a composite field of the configuration message.
This annotation is bounded to a string variable.
@CompositeConfigurationField(
value = {
@CompositeSubField(shortDescription = "URL", pattern = "https?://.{0,92}"),
@CompositeSubField(shortDescription = "username", pattern = ".{1,32}"),
@CompositeSubField(shortDescription = "password", pattern = ".{1,32}")
},
shortDescription = "Download URL",
composition = "${URL}<#if username?has_content && password?has_content>@${username}@${password}</#if>",
terminator = ",",
pattern = ".{0,100}"
)
public String downloadURL;
shortDescription
: a short description of the field, mandatory, used as an identifier (and thus must be unique inside every configuration message).longDescription
: a more expressive description, optional.unitOfMeasure
: the unit of measure, optional (the format should follow UCUM/ISO 80000 standard).pattern
: regex pattern this field must obey, optional.defaultValue
: default value, optional. If the variable is an array, then this field may represent anor
between values (e.g.ONE|TWO|THREE
), otherwise can be a single value (e.g.TWO
). If not present, then the field is mandatory.
Defines a subfield of a composite field of the configuration message.
This annotation is bounded to a string variable.
@CompositeConfigurationField(
value = {
@CompositeSubField(shortDescription = "URL", pattern = "https?://.{0,92}"),
@CompositeSubField(shortDescription = "username", pattern = ".{1,32}"),
@CompositeSubField(shortDescription = "password", pattern = ".{1,32}")
},
shortDescription = "Download URL",
composition = "${URL}<#if username?has_content && password?has_content>@${username}@${password}</#if>",
terminator = ",",
pattern = ".{0,100}"
)
public String downloadURL;
value
: a set of AlternativeSubFieldshortDescription
: a short description of the field, mandatory, used as an identifier (and thus must be unique inside every configuration message).longDescription
: a more expressive description, optional.unitOfMeasure
: the unit of measure, optional (the format should follow UCUM/ISO 80000 standard).minProtocol
: minimum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).maxProtocol
: maximum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).enumeration
: enumeration for this field, optional. If the field is not an array, then each value of this enum is mutually exclusive.terminator
: the string that terminates the skip (defaults to empty string), optional.
Defines an alternative field of the configuration message.
This annotation is bounded to a variable.
@AlternativeConfigurationField(
shortDescription = "Download protocol", terminator = ",", enumeration = DownloadProtocol.class,
value = {
@AlternativeSubField(maxProtocol = "1.35", defaultValue = "HTTP"),
@AlternativeSubField(minProtocol = "1.36", defaultValue = "HTTPS")
}
)
private DownloadProtocol downloadProtocol;
longDescription
: a more expressive description, optional.unitOfMeasure
: the unit of measure, optional (the format should follow UCUM/ISO 80000 standard).minProtocol
: minimum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).maxProtocol
: maximum protocol for which this configuration message is valid, optional (should follow Semantic Versioning).minValue
: minimum value this field can assume, optional (alternative topattern
andenumeration
).maxValue
: maximum value this field can assume, optional (alternative topattern
andenumeration
).pattern
: regex pattern this field must obey, optional.defaultValue
: default value, optional. If the variable is an array, then this field may represent anor
between values (e.g.ONE|TWO|THREE
), otherwise can be a single value (e.g.TWO
). If not present, then the field is mandatory.charset
: charset of the field (if string value), optional.radix
: radix of the number field when written to the message, optional.
Defines a subfield of an alternative field of the configuration message.
This annotation is bounded to a variable.
@AlternativeConfigurationField(
shortDescription = "Download timeout", terminator = ",", unitOfMeasure = "min",
value = {
@AlternativeSubField(maxProtocol = "1.18", minValue = "5", maxValue = "30", defaultValue = "10"),
@AlternativeSubField(minProtocol = "1.19", minValue = "5", maxValue = "30", defaultValue = "20")
}
)
private int downloadTimeout;
Return a description of the loaded templates and configuration. It basically provides a description of the annotations in JSON format.
Describer describer = Describer.create(core);
List<Map<String, Object>> templateDescriptions = describer.describeTemplates();
List<Map<String, Object>> configurationDescriptions = describer.describeConfiguration();
Extract values from a DTO using RFC6901 syntax.
Parser parser = Parser.create(core);
List<Response<byte[], Object>> result = parser.parse(payload);
ACKMessageASCII parsedMessage = (ACKMessageASCII)result.get(0).getMessage();
Extractor extractor = Extractor.create(parsedMessage);
String messageHeader = extractor.get("/messageHeader");
int protocolVersionMinor = extractor.get("/protocolVersion/minor");
Generate template or configuration messages, along with enumerations, starting from a description map generated by the Describer.
Generator generator = Generator.create(core);
Class<?> dynamicType = generator.generateTemplate(description);
core.addTemplate(dynamicType);
Uses Levenshtein metric to calculate the distance and the similarity between two templates.
Useful for building dendrograms and Hierarchical clustering, for example (see also Cluster analysis).
Comparator comparator = Comparator.create(core);
int distance = comparator.distance(ACKMessageHex.class, ACKMessageHexByteChecksum.class);
double similarity = comparator.similarity(ACKMessageHex.class, ACKMessageHexByteChecksum.class);
Care should be taken in writing SpEL expressions for the fields condition
, and size
.
The root object is the outermost object.
In order to evaluate a variable of a parent object the complete path should be used, from the root to the desired property, as in object1.variable1
.
In order to evaluate a variable of a children object, that is the object currently scanned, the relative path should be used introduced by the special keyword #self
, as in #self.variable2
.
Note that the #
prefix is intended to refer to the context (whether it is an object or a method), and the current object is stored in the context under the name self
.
See also Spring Expression Language (SpEL) Primer.
class A{
@BindInteger(size = "8")
private byte value;
@BindObject(type = OtherClass.class)
private OtherClass other;
@BindString(condition = "value == 2", size = "1")
private String var3;
}
class OtherClass{
@BindString(condition = "value == 1", size = "1")
private String var1;
@BindString(condition = "#self.var1.equals('2')", size = "1")
private String var2;
}
Boxon can handle array of primitives, bit, byte, short, int, long, float, double, and their object counterpart, as long as Object, BigInteger, string (with a given size, or with a terminator), and the special "checksum".
You can extend the basic functionalities through the application of converters as shown below in some examples. Here lies the power of Boxon.
Boxon already provides some build-in converters for your convenience: BitSetToBoolean, IntegerToFloat, LongToDouble, ShortToCharacter, StringToBigDecimal, UnsignedByteToShort, UnsignedShortToInteger, and UnsignedIntegerToLong.
NOTE that decode
and encode
MUST BE the inverse of each other, that is they MUST BE invertible (injective), or partly invertible, that is, otherwise said, decode(x) = y iff encode(y) = x
(eventually in a restricted domain).
@BindInteger(size = "64", converter = UnixTimestampConverter.class)
private LocalDateTime eventTime;
public class UnixTimestampConverter implements Converter<Long, LocalDateTime>{
@Override
public LocalDateTime decode(final Long unixTimestamp){
return LocalDateTime.ofInstant(Instant.ofEpochMilli(value), ZoneOffset.UTC);
}
@Override
public Long encode(final LocalDateTime value){
return value.toInstant(ZoneOffset.UTC).toEpochMilli();
}
}
@BindInteger(size = "8", converter = DateTimeYYYYMMDDHHMMSSConverter.class)
@BindAsArray(size = "7")
private ZonedDateTime eventTime;
public class DateTimeYYYYMMDDHHMMSSConverter implements Converter<byte[], ZonedDateTime>{
@Override
public ZonedDateTime decode(final byte[] value){
final ByteBuffer bb = ByteBuffer.wrap(value);
final int year = bb.getShort();
final int month = bb.get();
final int dayOfMonth = bb.get();
final int hour = bb.get();
final int minute = bb.get();
final int second = bb.get();
return DateTimeUtils.createFrom(year, month, dayOfMonth, hour, minute, second);
}
@Override
public byte[] encode(final ZonedDateTime value){
return ByteBuffer.allocate(7)
.putShort((short)value.getYear())
.put((byte)value.getMonthValue())
.put((byte)value.getDayOfMonth())
.put((byte)value.getHour())
.put((byte)value.getMinute())
.put((byte)value.getSecond())
.array();
}
}
IMEI converter (from 'nibble' array to String, that is, each nibble represents a character of the IMEI)
@BindInteger(size = "8", converter = IMEIConverter.class, validator = IMEIValidator.class)
@BindAsArray(size = "8")
private String imei;
public class IMEIConverter implements Converter<byte[], String>{
@Override
public String decode(final byte[] value){
final StringBuilder sb = new StringBuilder(15);
for(int i = 0; i < 7; i ++)
sb.append(String.format("%02d", value[i] & 255));
sb.append(ByteHelper.applyMaskAndShift(value[7], Byte.SIZE, (byte)0x0F));
return sb.toString();
}
@Override
public byte[] encode(final String value){
final byte[] imei = new byte[8];
final String[] components = value.split("(?<=\\G\\d{2})", 8);
for(int i = 0; i < 8; i ++)
imei[i] = (byte)Integer.parseInt(components[i]);
return imei;
}
}
@BindInteger(size = "8", converter = RSSIConverter.class)
private short rssi;
/**
* input: output:
* -----------------------
* 0: < -133 dBm
* 1: -111 dBm
* 2-30: -109 - -53 dBm
* 31: > -51 dBm
* 99: unknown
*/
public class RSSIConverter implements Converter<Byte, Short>{
public static final int RSSI_UNKNOWN = 0;
@Override
public Short decode(final Byte value){
if(value == null)
return null;
if(value == 0)
//< -133 dBm
return (byte)-133;
if(value == 99)
return RSSI_UNKNOWN;
//31 is > -51 dBm
return (short)(value * 2 - 133);
}
@Override
public Byte encode(final Short value){
if(value == null)
return null;
if(value == -133)
return 0;
if(value == RSSI_UNKNOWN)
return 99;
return (byte)((value + 133) / 2);
}
}
An annotation can be written by 'extending' an existing default annotation through composition, annotating the new annotation with the default 'parent' one.
NOTE: If the extended annotation uses some elements of the parent one, these MUST BE reported as they are defined in the parent annotation.
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
@Documented
@BindStringTerminated(terminator = ',')
public @interface BindStringCommaTerminated{
//NOTE that only the parameters used by this annotation should be *copied as-is* from the parent annotation!
String condition() default "";
String charset() default "UTF-8";
byte terminator() default 0;
boolean consumeTerminator() default true;
}
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
@Repeatable(SkipTwoBytes.Skips.class)
@Documented
@SkipBits("2*8")
public @interface SkipTwoBytes{
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
@Documented
@SkipBits.Skips
@interface Skips{
SkipTwoBytes[] value();
}
}
You can also define your own annotation by define a plain annotation from scratch and implementing Codec
as in the following example.
Optionally, the method String condition()
could be defined.
... and remember to add it to the Parser
!
//annotation
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
@interface VarLengthEncoded{}
import io.github.mtrevisan.boxon.io.Evaluator;
//codec
//the number of bytes to read is determined by the leading bit of each individual bytes
//(if the first bit of a byte is 1, then another byte is expected to follow)
class VariableLengthByteArray implements Codec{
private static TemplateParser TEMPLATE_PARSER = TemplateParser.getInstance();
public Class<?> type(){
return VarLengthEncoded.class;
}
public Object decode(TemplateParser templateParser, BitBuffer reader, VarLengthEncoded annotation, Object data){
Evaluator.evaluate("1+2");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
boolean continuing = true;
while(continuing){
byte b = reader.getByte();
baos.write(b & 0x7F);
continuing = ((b & 0x80) != 0x00);
}
return baos.toByteArray();
}
public void encode(TemplateParser templateParser, BitWriter writer, VarLengthEncoded annotation, Object data, Object value){
int size = Array.getLength(value);
for(int i = 0; i < size; i++)
writer.put((byte)((byte)Array.get(value, i) | (i < size - 1? (byte)0x80: 0x00)), ByteOrder.BIG_ENDIAN);
}
}
//add the custom codec to the list of available codecs
//(use one of the lines below)
core.withDefaultCodecs(); //loads all codecs from the library itself
core.withCodecsFrom(CodecCustomTest.class); //this class resides in the package where the custom codec(s) are
core.withCodec(new VariableLengthByteArray());
Almost for each base annotation there is a corresponding class defined into Template.java
that manages the encoding and decoding of the underlying data.
The other annotations are managed directly into TemplateParser.java
, that is the main class that orchestrates the parsing of a single message with all of its annotations.
If an error occurs an AnnotationException
(an error occurs on an annotation definition), CodecException
(an error occurs while finding the appropriate codec), TemplateException
(an error occurs if a template class is badly annotated), or DecodeException
/EncodeException
(a container exception for the previous ones for decoding and encoding respectively) is thrown.
Messages can be concatenated, and the Parser.java
class manages them, returning a DTO, ParserResponse.java
, which contains a list of all successfully read messages and a list of all errors from problematic messages.
Each annotated class is processed by Template.class
, that is later retrieved by Parser.java
depending on the starting header.
For that reason each starting header defined into TemplateHeader
annotation MUST BE unique. This class can also accept a context.
All the SpEL expressions are evaluated by Evaluator.java
.
All the annotated classes are conveniently loaded using the Loader.java
as is done automatically in the Parser.java
.
If you want to provide your own classes you can use the appropriate with...
method of Parser
.
The Parser
is also used to encode a message.
BitBuffer.java
has the task to read the bits, whereas BitWriter.java
has the task to write the bits.
BitSet.java
is the container for the bits (like java.utils.BitSet
, but enhanced for speed).
ByteOrder.java
is the enum that is used to indicate the byte order.
All you have to care about, for a simple example on multi-message automatically-loaded templates, is the Parser
.
//optionally create a context
Map<String, Object> context = ...
Core core = CoreBuilder.builder()
.withContext(context)
.withContext(VersionHelper.class, "compareVersion", String.class, String.class)
.withContext(VersionHelper.class.getDeclaredMethod("compareVersion", new Class[]{String.class, String.class}))
.withDefaultCodecs()
.withTemplate(...)
.build();
Parser parser = Parser.create(core);
//parse the message
byte[] payload = ...
List<Response<byte[], Object>> result = parser.parse(payload);
//process the successfully parsed messages and errors
for(int index = 0; index < result.size(); index ++){
Response<byte[], Object> response = result.get(index);
Object parsedMessage = response.getMessage();
Exception error = response.getError();
if(error != null){
LOGGER.error("An error occurred while parsing:\r\n {}", response.getSource());
}
else if(parsedMessage != null){
...
}
}
The inverse of parsing is composing, and it's simply done as follows.
//compose the message (`Template` should be a recognized template)
Template data = ...;
Response<Template, byte[]> composeResult = composer.compose(data);
//process the composed messages
byte[] composedMessage = response.getMessage();
Exception error = response.getError();
if(error != null){
LOGGER.error("An error occurred while composing:\r\n {}", response.getSource());
}
else if(composedMessage != null){
...
}
Please report issues to the issue tracker if you have any difficulties using this module, found a bug, or request a new feature.
Pull requests are welcomed.
- Added
checksumSize
parameter toChecksum
annotation to specify the size of the CRC field in the stream, allowing for different checksum algorithms to be coded. - Enhanced
Checksummer
capabilities to encompass more CRC algorithms.
- Fixed
SpelCompilerMode
toMIXED
to address compilation problems.
- Improve error messages and handling in encoding and decoding process.
- Now it's possible to extends a codec.
- Now it's possible to annotate a custom annotation, creating a default.
- Added
ContextParameter
annotation which assigns a constant or calculated value to a parameter that will be added to the context before processing; each context parameter is created before a field is decoded or encoded and removed as soon as the field processing completes. - Added
enumerations
key in template and configuration descriptions. - Added
Generator
, to create an annotatedTemplate
andConfigurationMessage
from a description generated by theDescriber
(an enumeration is represented as an array of strings, each string is a pair<name>(<value>)
, wherename
is the name of the enum element, andvalue
is a number associated with the element; context data are ignored). - Added long description and unit of measure attribute to
BindBitSet
,BindInteger
,BindObject
,BindString
,BindStringTerminated
,Evaluate
, andPostProcess
annotations. - Fixed bug on missing
annotationType
in alternatives' description. - 2× in speed with respect to previous version due to efficient use of (configurable) memoizers (see
Parser
) and other improvements. - Custom
DeviceTypes
code length (no longer bounded to be abyte
).
- Add support for
BindAsArray
andBindAsList
instead of manyBindArray
/BindArrayPrimitive
andBindList
, now removed. - Cleaning and refactoring of various codecs.
- Major reorganization of the code, along with refactor to make it more modular and cohesive, and removal of duplicated code.
- Simplified the exception handling.
- Improved annotation validation by skipping validations for non-library annotations.
- Revised and corrected the reader.
- Added custom validation for custom codecs.
- Added similarity/distance calculation between descriptions.
- Incompatibility between read type, converter input, converter output, validator, and field type is now resolved during core construction, rather than at runtime.
- Fixed error while handling of
SkipBits
andSkipUntilTerminator
to allow processing multiple instances. - Removed useless
BindFloat
andBindDouble
has they are rarely used and can be substituted by the appropriate converters (IntegerToFloatConvert
andLongToDoubleConverter
). - Removed useless
BindByte
,BindShort
,BindInt
, andBindLong
as they can be substituted byBindInteger
. - Renamed
Descriptor
into the more meaningfulDescriber
. - Added builder to
Version
. - Many smells removed, major code refactor.
- Specialized
Skip
annotation intoSkipBits
andSkipUntilTerminator
. - Added subtypes describer on
BindObject
,BindArray
,BindList
,ObjectChoices.ObjectChoice
, andObjectChoicesList.ObjectChoiceList
. - Some minor improvements.
- Fix error while assessing size value.
- Corrected
NullObjectChoice
andNullObjectChoiceList
type value. - Fix error while putting numeric value in
NumberWriterManager
. - Removed the default on
Skip
size: it was intended to be mandatory. - Fixed bug on evaluating an expression on a deep nested object.
- (minor) Fix missing field name in parser context in
TemplateParser.encode
.
- Added condition on
Checksum
annotation. - Removed
type
andstartValue
fromChecksum
: the same information can be retrieved from the algorithm used, plus, the start value can be embedded in the implemented class. - Fixed a problem while converting a numeric string to number.
- Make record classes work with annotations.
- Slightly reduced memory footprint and improved execution time.
- Removed
runLast
fromEvaluator
, added a specialized annotation that can work in both decoding and encoding phases. - Added method
describeParser
toDescriptor
to also include the description of evaluation and post-processing fields.
- Added description of configuration.
- Changed the key to reference a configuration (from
start
toshortDescription
). - Corrected generation of parameter
MutuallyExclusive
on configuration description. - Corrected log text on enumeration error.
- Fixed a test that occasionally fails.
- Added
runLast
toEvaluator
to run an evaluation after all other evaluations have been made (used, for example, to restore a default value after being used by another evaluator).
- Fixed duplicated descriptions.
- Fixed validation on max value while composing a message.
- Fixed number not written with the correct radix.
- Made
shortDescription
mandatory in the annotation, as it should have been. - Added method to map a DTO into a
Map<String, Object>
inReflectionHelper
. - Added method
Configurator.composeConfiguration
accepting a DTO. - Corrected errors in the documentation.
- Migrated from java 11 to java 21 for performance.
- Improvement on handling values inside big decimal converter.
- Improvement on error reporting.
- Renamed
Composer.composeMessage
intocompose
. - Corrected error while showing the start array of message header in the description.
- Fix size validation of array and list (now it can be zero).
- Fixed an error if annotating with
@Skip
as the last annotation of the DTO.
- Added
@BindList
, the equivalent of@BindArray
for messages with separators.
- Fixed a bug on
@ConfigurationHeader
where the protocol range check was incorrectly done considering the minimum protocol as the maximum.
- Updated library versions.
- Added
CoreBuilder
to facilitate the creation of aCore
: now it is no longer necessary to remember the order in which the methods should be called. - Added missing javadoc. Enhanced existing javadoc.
- Added
@BindBitSet
binding for java@BitSet
. - Added
Extractor
, used to programmatically extract values from a DTO. - Removed
Bits
. - Enhanced binding validation.
- Fixed a concurrency bug on the validation of alternatives.
- Reordered some packages to better reflect usage.
- Added missing javadoc.
- No more cycles between classes or packages.
- Bug fix:
Evaluator
class is now exportable. - Removed a package cycle.
- General cleaning of the code (removed duplicated code, useless templates, etc.).
- Made library thread-safe.
- Added methods to retrieve a description of the protocol (in JSON format).
- Decomposed and simplified
Parser
class.
- Completely revised the packages, solving a lot of structural problems and refactorings that have to be done.
- Added methods to retrieve the configurations, a.k.a. a JSON that tells the configuration parameters of certain annotated messages.
- Better handling of NOP logger.
- Abandoned Reflections in favor of ClassGraph.
- Added
BindArray.selectDefault
andBindObject.selectDefault
to cope with default selector that has no prefix. - Added some feasibility checks on annotation data.
- Added public constructor to
Parser
to allow for extensions. - Changed the signature of
Checksummer.calculateChecksum
returning short instead of long. - Changed method
Validator.validate
intoValidator.isValid
. - Changed method
ParserResponse.getMessageForError
intoParserResponse.getErrorMessageAt
to align it to other method name's conventions. - Moved classes
ParserResponse
andComposerResponse
fromio.github.mtrevisan.boxon.external
toio.github.mtrevisan.boxon.core
in order to hide add methods; the constructors are also hidden. - Minor refactorings.
- Added
originator
variable (and its getter) toComposerResponse
to hold the given objects used to create the message. - Added/modified javadocs to better explain some classes.
- Removed
ComposerResponse.getErrors
,BindInteger.unsigned
andBitReader.getInteger(int, ByteOrder, boolean)
as they are useless. - Removed
BitWriter.putText(String, byte, boolean)
because of the Boolean Trap. - Removed useless
match()
parameter from bindings. - Enhanced the exception message thrown if the type of
BitReader.get(Class, ByteOrder)
is not recognized. - Renamed
BindChecksum
intoChecksum
. - Relocated all binding annotations inside
annotations.bindings
(Bind*
and*Choices
). - Corrected bug while reading skips in
TemplateParser.decode
.
- Speed-up execution.
- Revision of the packages with removal of cycles.
- Better handling of class retrieval (codecs and templates).
- Final revision.
- First revision.
- Some more thoughts on how it should work.
- First version.
This project is licensed under MIT license. For the full text of the license, see the LICENSE file.