Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.4.0 #13

Merged
merged 6 commits into from
Dec 12, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: basic support for Ctx{Get,Set} decorators
`Ctx` decorators are a way to share informations across
objects during the reading of the binary file definition.

In the following example, a streaming protocol that receives records
of arbitrary length is defined.
Records have two different type a 'definition' or a 'data' both use an
'id' to identify themself. The definition defines the size of the data
message they define by using `CtxSet` to store that size into the
context. The data message uses `CtxGet` to fetch its size defined
previously by the definition.

```typescript
class RecordDefinition {
    @relation(PrimitiveSymbol.u8)
    id: number

    @CtxSet(_ => `Definition.${_.id}`)
    @relation(PrimitiveSymbol.u8)
    size: number
}

class RecordMessage {
    @relation(PrimitiveSymbol.u8)
    id: number

    @CtxGet(_ => `Definition.${_.id}`)
    _size: number

    @count('_size')
    @relation(PrimitiveSymbol.u8)
    data
}

class Record {
    @relation(PrimitiveSymbol.u8)
    type: number

    @choice('type', {
        0x00: RecordDefinition,
        0x01: RecordMessage,
    })
    message: RecordDefinition | RecordMessage
}

class Protocol {
    @until(EOF)
    records: Record[]
}
```
tperale committed Dec 12, 2024
commit b450ed2c1e6f639c4dca4e9bcb128ea7342839ca
30 changes: 27 additions & 3 deletions src/__tests__/parser.test.ts
Original file line number Diff line number Diff line change
@@ -4,9 +4,10 @@ import { EOF, PrimitiveSymbol, type InstantiableObject } from '../types'
import { binread } from '../reader'
import { withBinspectorContext } from '../context'
import { BinaryReader, BinaryCursorEndianness } from '../cursor'
import { CtxGet, CtxSet } from '../decorators/context'

function expectReadTest<Target> (buffer: Array<number>, ObjectDefinition: InstantiableObject<Target>, endian: BinaryCursorEndianness = BinaryCursorEndianness.BigEndian, ...args: any[]) {
return expect(binread(new BinaryReader(new Uint8Array(buffer).buffer, endian), ObjectDefinition, ...args))
function expectReadTest<Target> (buffer: Array<number>, ObjectDefinition: InstantiableObject<Target>, endian: BinaryCursorEndianness = BinaryCursorEndianness.BigEndian, ctx = {}, ...args: any[]) {
return expect(binread(new BinaryReader(new Uint8Array(buffer).buffer, endian), ObjectDefinition, ctx, ...args))
}

function expectReadTestToThrow<Target> (buffer: Array<number>, ObjectDefinition: InstantiableObject<Target>) {
@@ -536,7 +537,7 @@ describe('Reading binary definition with PrePost decorators', () => {
}
}

expectReadTest([0x01, 0x02, 0x03, 0x04], Protocol, BinaryCursorEndianness.BigEndian, 2).toMatchObject({
expectReadTest([0x01, 0x02, 0x03, 0x04], Protocol, BinaryCursorEndianness.BigEndian, {}, 2).toMatchObject({
value: 0x03,
})
})
@@ -652,6 +653,29 @@ describe('Reading binary definition with PrePost decorators', () => {
})
})

describe('Reading binary definition with Ctx decorators', () => {
it('should ', () => {
class Protocol {
@CtxGet('Settings.Count')
data_type: number

@CtxSet('Settings.Value')
@Count('data_type')
@Relation(PrimitiveSymbol.u8)
foo: number
}

const ctx = { Settings: { Count: 3 } }

expectReadTest([0x01, 0x02, 0x03], Protocol, BinaryCursorEndianness.LittleEndian, ctx).toMatchObject({
data_type: 3,
foo: [1, 2, 3],
})

expect(ctx).toMatchObject({ Settings: { Count: 3, Value: [1, 2, 3] } })
})
})

describe('Reading a relation to an empty definition', () => {
it('should throw an error', () => {
class Header {
72 changes: 72 additions & 0 deletions src/decorators/__tests__/context.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
import { describe, expect } from '@jest/globals'
import { CtxSet, CtxGet, useContextGet, useContextSet } from '../context'
import Meta from '../../metadatas'

function testContextGet (TargetClass: new () => any, field: string, ctx: object, expected: any) {
const instance = new TargetClass()

const contexts = Meta.getContext(TargetClass[Symbol.metadata] as DecoratorMetadataObject, field)
expect(useContextGet(contexts, instance, ctx)).toEqual(expected)
}

function testContextSet (TargetClass: new () => any, field: string, value: any, ctx: object) {
const instance = new TargetClass()

const contexts = Meta.getContext(TargetClass[Symbol.metadata] as DecoratorMetadataObject, field)
useContextSet(contexts, value, instance, ctx)
}

describe('@Ctx: functions', () => {
it('should modify the ctx object', () => {
class TestClass {
@CtxSet('test')
data: number
}
const ctx: any = {}
const VALUE = 1
testContextSet(TestClass, 'data', VALUE, ctx)
expect(ctx.test).toEqual(VALUE)
})
it('should modify the ctx object with recursive accessors', () => {
class TestClass {
@CtxSet('foo.bar.1')
data: number

@CtxSet('foo.bar.2')
data_2: number
}
const ctx: any = {}
const VALUE = 1
testContextSet(TestClass, 'data', VALUE, ctx)
testContextSet(TestClass, 'data_2', VALUE, ctx)
expect(ctx.foo.bar[1]).toEqual(VALUE)
expect(ctx.foo.bar[2]).toEqual(VALUE)
})
it('should get value from the ctx object', () => {
class TestClass {
@CtxGet('test')
data: number
}
const VALUE = 1
const ctx = { test: VALUE }
testContextGet(TestClass, 'data', ctx, VALUE)
})
it('should get value from the ctx object with recursive accessors', () => {
class TestClass {
@CtxGet('foo.bar')
data: number
}
const VALUE = 1
const ctx = { foo: { bar: VALUE } }
testContextGet(TestClass, 'data', ctx, VALUE)
})
it('should throw when accessing non existing properties', () => {
class TestClass {
@CtxGet('test.foo')
data: number
}
const VALUE = 1
const ctx = { test: VALUE }
expect(() => testContextGet(TestClass, 'data', ctx, VALUE)).toThrow()
})
})
264 changes: 264 additions & 0 deletions src/decorators/context.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,264 @@
/**
* Module definition of {@link Context} property decorators.
*
* The {@link Context} decorators provides functions to read/write the context
* shared among the binary objects during the reading phase.
*
* @module Context
*/
import { Context, DecoratorType } from '../types'
import { createPropertyMetaDescriptor, PropertyMetaDescriptor, recursiveGet, StringFormattedRecursiveKeyOf } from './common'
import { Relation } from './primitive'
import Meta from '../metadatas'

export const CtxSymbol = Symbol('context')

export type GlobalCtx = object

export type CtxKeyFunction<This> = (targetInstance: This) => string

export enum CtxType {
CtxGetter,
CtxSetter,
}

export interface CtxOptions {
/**
* Ensures that a relation exists before defining the Transformer decorator.
*/
base_type: 'array' | undefined
}

export const CtxOptionsDefault = {
base_type: undefined,
}

/**
* Ctx metadata type definition.
*
* @extends {PropertyMetaDescriptor}
*/
export interface Ctx<This> extends PropertyMetaDescriptor<This> {
options: CtxOptions
/**
* Function that retrieve the key to access the context
*/
keyGetter: CtxKeyFunction<This>
/**
* Context type: retrieve a value or set a value
*/
func_type: CtxType
}

function ctxPropertyFunctionDecoratorFactory<This extends object, Value> (name: string, func_type: CtxType, keyGetter: string | CtxKeyFunction<This>, opt: Partial<CtxOptions> = CtxOptionsDefault): DecoratorType<This, Value> {
const options = { ...CtxOptionsDefault, ...opt }

return function (_: any, context: Context<This, Value>) {
const propertyKey = context.name as keyof This
if (!Meta.isFieldDecorated(context.metadata, propertyKey)) {
// Create an empty relation that wont be read.
Relation()(_, context)
}
const ctx: Ctx<This> = {
...createPropertyMetaDescriptor(CtxSymbol, name, context.metadata, propertyKey),
func_type,
options,
keyGetter: typeof keyGetter === 'string' ? () => keyGetter : keyGetter,
}

Meta.setContext(context.metadata, propertyKey, ctx)
}
}

/**
* `@CtxGet` decorator retrieve a value based on the key passed as argument
* from a 'context' shared during the reading phase.
*
* @example
*
* In the following example, a streaming protocol that receives records
* of arbitrary length is defined.
* Records have two different type a 'definition' or a 'data' both use an
* 'id' to identify themself. The definition defines the size of the data
* message they define by using `CtxSet` to store that size into the
* context. The data message uses `CtxGet` to fetch its size defined
* previously by the definition.
*
* ```typescript
* class RecordDefinition {
* @Relation(PrimitiveSymbol.u8)
* id: number
*
* @CtxSet(_ => `Definition.${_.id}`)
* @Relation(PrimitiveSymbol.u8)
* size: number
* }
*
* class RecordMessage {
* @Relation(PrimitiveSymbol.u8)
* id: number
*
* @CtxGet(_ => `Definition.${_.id}`)
* _size: number
*
* @Count('_size')
* @Relation(PrimitiveSymbol.u8)
* data
* }
*
* class Record {
* @Relation(PrimitiveSymbol.u8)
* type: number
*
* @Choice('type', {
* 0x00: RecordDefinition,
* 0x01: RecordMessage,
* })
* message: RecordDefinition | RecordMessage
* }
*
* class Protocol {
* @Until(EOF)
* records: Record[]
* }
* ```
*
* @param {CtxKeyFunction<This>} keyGetter Either a string formatted as
* recursive key or a function that returns that string based on the
* instance value.
* @param {Partial<CtxOptions>} [opt] Optional configuration.
* @returns {DecoratorType<This, Value>} The property decorator function.
*
* @category Decorators
*/
export function CtxGet<This extends object, Value> (keyGetter: CtxKeyFunction<This> | string, opt?: Partial<CtxOptions>): DecoratorType<This, Value> {
return ctxPropertyFunctionDecoratorFactory<This, Value> ('ctx-get', CtxType.CtxGetter, keyGetter, opt)
}

/**
* `@CtxSet` decorator set the value of the decorated property into a
* shared 'context' during the reading phase.
*
* @example
*
* In the following example, a streaming protocol that receives records
* of arbitrary length is defined.
* Records have two different type a 'definition' or a 'data' both use an
* 'id' to identify themself. The definition defines the size of the data
* message they define by using `CtxSet` to store that size into the
* context. The data message uses `CtxGet` to fetch its size defined
* previously by the definition.
*
* ```typescript
* class RecordDefinition {
* @Relation(PrimitiveSymbol.u8)
* id: number
*
* @CtxSet(_ => `Definition.${_.id}`)
* @Relation(PrimitiveSymbol.u8)
* size: number
* }
*
* class RecordMessage {
* @Relation(PrimitiveSymbol.u8)
* id: number
*
* @CtxGet(_ => `Definition.${_.id}`)
* _size: number
*
* @Count('_size')
* @Relation(PrimitiveSymbol.u8)
* data
* }
*
* class Record {
* @Relation(PrimitiveSymbol.u8)
* type: number
*
* @Choice('type', {
* 0x00: RecordDefinition,
* 0x01: RecordMessage,
* })
* message: RecordDefinition | RecordMessage
* }
*
* class Protocol {
* @Until(EOF)
* records: Record[]
* }
* ```
*
* @param {CtxKeyFunction<This>} keyGetter Either a string formatted as
* recursive key or a function that returns that string based on the
* instance value.
* @param {Partial<CtxOptions>} [opt] Optional configuration.
* @returns {DecoratorType<This, Value>} The property decorator function.
*
* @category Decorators
*/
export function CtxSet<This extends object, Value> (keyGetter: CtxKeyFunction<This> | string, opt?: Partial<CtxOptions>): DecoratorType<This, Value> {
return ctxPropertyFunctionDecoratorFactory<This, Value> ('ctx-set', CtxType.CtxSetter, keyGetter, opt)
}

/**
* useContextGet execute an array of `Ctx` decorator metadata.
*
* @typeParam This The type of the class the decorator is applied to.
*
* @param {Array<Ctx<This>>} metaCtx An array of context function to apply.
* @param {This} targetInstance The target class instance containing the property.
* @param {GlobalCtx} ctx The shared context reference.
* @returns {any} The context retrieved.
*
* @category Advanced Use
*/
export function useContextGet<This> (metaCtx: Array<Ctx<This>>, targetInstance: This, ctx: GlobalCtx): any {
const values = metaCtx.filter(x => x.func_type === CtxType.CtxGetter).map((x) => {
const key = x.keyGetter(targetInstance)

// TODO future version should pass some typing to the context
return key.split('.').reduce((acc: any, key: string) => {
if (Object.prototype.hasOwnProperty.call(acc, key) === false) {
throw new ReferenceError(`Can't retrieve key: '${key}' from ctx: ${JSON.stringify(ctx)}.`)
}
return acc[key]
}, ctx)
})

return values.length === 1 ? values[0] : values
}

/**
* useContextSet execute an array of `Ctx` decorator metadata.
*
* @typeParam This The type of the class the decorator is applied to.
*
* @param {Array<Ctx<This>>} metaCtx An array of context function to apply.
* @param {This} targetInstance The target class instance containing the property.
* @param {GlobalCtx} ctx The shared context reference.
* @returns {any} The context retrieved.
*
* @category Advanced Use
*/
export function useContextSet<This> (metaCtx: Array<Ctx<This>>, propertyValue: any, targetInstance: This, ctx: GlobalCtx): void {
metaCtx.filter(x => x.func_type === CtxType.CtxSetter).forEach((x) => {
const key = x.keyGetter(targetInstance)
const accessors = key.split('.')
const lastKey = accessors[accessors.length - 1]
const ref = accessors.slice(0, -1).reduce((acc: any, key: string) => {
if (Object.prototype.hasOwnProperty.call(acc, key) === false) {
if (x.options.base_type == 'array') {
Object.defineProperty(acc, key, {
value: []
})
} else {
Object.defineProperty(acc, key, {
value: {}
})
}
}
return acc[key]
}, ctx)
ref[lastKey] = propertyValue
})
}
1 change: 1 addition & 0 deletions src/decorators/index.ts
Original file line number Diff line number Diff line change
@@ -10,3 +10,4 @@ export * from './primitive'
export * from './prepost'
export * from './bitfield'
export * from './transformer'
export * from './context'
19 changes: 19 additions & 0 deletions src/metadatas.ts
Original file line number Diff line number Diff line change
@@ -4,6 +4,7 @@ import { type Controller, ControllerSymbol } from './decorators/controller'
import { type Transformer, TransformerSymbol } from './decorators/transformer'
import { type Validator, ValidatorSymbol } from './decorators/validator'
import { type BitField, BitFieldSymbol } from './decorators/bitfield'
import { type Ctx, CtxSymbol } from './decorators/context'
import { type PrePost, PreFunctionSymbol, PostFunctionSymbol, PreClassFunctionSymbol, PostClassFunctionSymbol, PrePostSymbols, PrePostClass } from './decorators/prepost'
import { type DecoratorMetadataObject } from './types'

@@ -252,6 +253,22 @@ function setBitField<This> (
return bitfields
}

function getContext<This> (metadata: DecoratorMetadataObject, propertyKey: keyof This): Array<Ctx<This>> {
return getMetadata(
metadata,
propertyKey,
CtxSymbol,
)
}

function setContext<This> (
metadata: DecoratorMetadataObject,
propertyKey: keyof This,
ctx: Ctx<This>,
): Array<Ctx<This>> {
return setMetadata(metadata, propertyKey, ctx.type, ctx)
}

export default {
getMetadata,
setMetadata,
@@ -276,4 +293,6 @@ export default {
getBitField,
getBitFields,
setBitField,
getContext,
setContext,
}
14 changes: 12 additions & 2 deletions src/reader.ts
Original file line number Diff line number Diff line change
@@ -21,6 +21,7 @@ import { useValidators } from './decorators/validator'
import { useConditions } from './decorators/condition'
import { usePrePost } from './decorators/prepost'
import { useBitField } from './decorators/bitfield'
import { useContextGet, useContextSet, CtxType } from './decorators/context'

/**
* binread.
@@ -44,7 +45,7 @@ import { useBitField } from './decorators/bitfield'
* `ObjectDefinition` passed in param.
* You can create self refering field by using conditionnal decorator.
*/
export function binread<Target> (content: Cursor, ObjectDefinition: InstantiableObject<Target>, ...args: any[]): Target {
export function binread<Target> (content: Cursor, ObjectDefinition: InstantiableObject<Target>, ctx = {}, ...args: any[]): Target {
const ObjectDefinitionName = ObjectDefinition.name
function getBinReader (field: PropertyType<Target>, instance: Target): ControllerReader {
if (isPrimitiveRelation(field)) {
@@ -63,7 +64,7 @@ export function binread<Target> (content: Cursor, ObjectDefinition: Instantiable
}

try {
return binread(content, field.relation, ...[...finalArgs, instance])
return binread(content, field.relation, ctx, ...[...finalArgs, instance])
} catch (error) {
// We need to catch the EOF error because the binread function
// can't return it so it just throw it EOFError.
@@ -105,6 +106,14 @@ export function binread<Target> (content: Cursor, ObjectDefinition: Instantiable
Meta.getFields<Target>(metadata).forEach((field) => {
usePrePost(Meta.getPre(metadata, field.propertyName), instance, content, ExecutionScope.OnRead)

const metaCtx = Meta.getContext(field.metadata, field.propertyName)

const ctxGetter = metaCtx.filter(x => x.func_type === CtxType.CtxGetter)
if (ctxGetter.length) {
instance[field.propertyName] = useContextGet(metaCtx, instance, ctx)
return
}

// TODO [Cursor] Pass the field name information to add to the namespace
const finalRelationField = isUnknownProperty(field) ? useConditions(Meta.getConditions(field.metadata, field.propertyName), instance) : field
if (finalRelationField !== undefined) {
@@ -127,6 +136,7 @@ export function binread<Target> (content: Cursor, ObjectDefinition: Instantiable
useValidators(Meta.getValidators(metadata, field.propertyName), transformedValue, instance, content)

instance[field.propertyName] = transformedValue
useContextSet(metaCtx, transformedValue, instance, ctx)
}
usePrePost(Meta.getPost(metadata, field.propertyName), instance, content, ExecutionScope.OnRead)
})

Unchanged files with check annotations Beta

}
function asObjectDtb (structs: DTBStructBlock[]): object {
function setObject (o: Record<string, any>, current: string[], key: string, value: any) {

Check warning on line 134 in example/devicetree/devicetree.ts

GitHub Actions / lint

Unexpected any. Specify a different type

Check warning on line 134 in example/devicetree/devicetree.ts

GitHub Actions / lint

Unexpected any. Specify a different type
const currentObj = current.reduce((obj, k) => obj[k], o)
currentObj[key] = value
@Relation(PrimitiveSymbol.char)
strings: string[]
asObject (): Record<string, any> {

Check warning on line 191 in example/devicetree/devicetree.ts

GitHub Actions / lint

Unexpected any. Specify a different type
return Reflect.get(asObjectDtb(this.structs), '')
}
}
test('', async () => {
const data = await fs.readFile(path.join(__dirname, 'am335x-bone.dtb'))
const dts = binread(new BinaryReader(data.buffer), DTB).asObject()

Check warning on line 10 in example/devicetree/index.test.ts

GitHub Actions / lint

'dts' is assigned a value but never used. Allowed unused vars must match /^_/u
// console.log(JSON.stringify(dts, null, 2))
})
}
it('should retrieve the member of the instance in the same order of definition', () => {
const testMatchInstance = new TestMatch()

Check warning on line 22 in src/__tests__/decorators.test.ts

GitHub Actions / lint

'testMatchInstance' is assigned a value but never used. Allowed unused vars must match /^_/u
const fields = Meta.getFields(TestMatch[Symbol.metadata] as DecoratorMetadataObject)
expect(fields.map(x => x.propertyName)).toStrictEqual([
'test',
])
})
it('should retrieve the correct metadata content', () => {
const testMatchInstance = new TestMatch()

Check warning on line 31 in src/__tests__/decorators.test.ts

GitHub Actions / lint

'testMatchInstance' is assigned a value but never used. Allowed unused vars must match /^_/u
const testMatchPropertyMetadatas = Meta.getValidators(
TestMatch[Symbol.metadata] as DecoratorMetadataObject,
'test',
const testSymbol = Symbol('test')
function Decorator<This, Value> (_: any, context: Context<This, Value>): void {

Check warning on line 13 in src/__tests__/metadatas.test.ts

GitHub Actions / lint

Unexpected any. Specify a different type
Meta.setMetadata(context.metadata, context.name, testSymbol, context.name)
}
describe('Set metadata information through the metadata API', () => {
it('should manage to retrieve the type information set by the decorator from the Reflect API', () => {
const c = new MyClass()

Check warning on line 27 in src/__tests__/metadatas.test.ts

GitHub Actions / lint

'c' is assigned a value but never used. Allowed unused vars must match /^_/u
expect(Meta.getMetadata(MyClass[Symbol.metadata] as DecoratorMetadataObject, 'field1', testSymbol)).toStrictEqual(['field1'])
expect(Meta.getMetadata(MyClass[Symbol.metadata] as DecoratorMetadataObject, 'field2', testSymbol)[0]).toStrictEqual('field2')
})
it('should store the validator', () => {
const c = new MyClass()
const validator: Validator<any, boolean> = {

Check warning on line 33 in src/__tests__/metadatas.test.ts

GitHub Actions / lint

Unexpected any. Specify a different type
id: 0,
type: ValidatorSymbol,
name: 'test',
metadata: MyClass[Symbol.metadata] as DecoratorMetadataObject,
propertyName: 'field1',
options: ValidatorOptionsDefault,
validator: (_: any) => true,

Check warning on line 40 in src/__tests__/metadatas.test.ts

GitHub Actions / lint

Unexpected any. Specify a different type
}
expect(Meta.setValidator(c, 'field1', validator)).toStrictEqual([
validator,