How to parse bits

[Gerschtli]

Hey,

I played around with you library and wanted to figure out how I can parse a sequence like one byte, then 3 bits and then one byte with Big Endian mode.

I created the following JBBP script:

byte byte1;
bit:3 bits;
byte byte2;

which generated

package org.jbbp;

import com.igormaznitsa.jbbp.model.*;
import com.igormaznitsa.jbbp.io.*;
import com.igormaznitsa.jbbp.compiler.*;
import com.igormaznitsa.jbbp.compiler.tokenizer.*;
import java.io.IOException;
import java.util.*;

/**
 * Generated from JBBP script by internal JBBP Class Source Generator
 */
public class Example {
	
	/**
	 * The Constant contains parser flags
	 * @see JBBPParser#FLAG_SKIP_REMAINING_FIELDS_IF_EOF
	 * @see JBBPParser#FLAG_NEGATIVE_EXPRESSION_RESULT_AS_ZERO
	 */
	protected static final int _ParserFlags_ = 0;

	public byte byte1;
	public byte bits;
	public byte byte2;
	

	public Example () {
	}

	public Example read(final JBBPBitInputStream In) throws IOException {
		this.byte1 = (byte)In.readByte();
		this.bits = In.readBitField(JBBPBitNumber.BITS_3);
		this.byte2 = (byte)In.readByte();
		
		return this;
	}

	public Example write(final JBBPBitOutputStream Out) throws IOException {
		Out.write(this.byte1);
		Out.writeBits(this.bits,JBBPBitNumber.BITS_3);
		Out.write(this.byte2);
		
		return this;
	}
}

To test this, I used this data: 0x01a100

Binary representation:
00000001 101 00001000 00000
________ ___ ________
 ^        ^    ^ 
 |        |    byte2
 |        bits
 byte1

I want to get

byte1: 1
bits:  5
byte2: 8

but get

byte1: 1
bits:  1
byte2: 20

I used this JUnit Test:

package org.example;

import static org.assertj.core.api.Assertions.assertThat;

import com.igormaznitsa.jbbp.io.JBBPBitInputStream;
import java.io.ByteArrayInputStream;
import java.util.HexFormat;
import org.jbbp.Example;
import org.junit.jupiter.api.Test;

class ExampleTest {
  @Test
  void test() throws Exception {
    // 00000001 10100001 00000000
    var data = HexFormat.of().parseHex("01a100");

    var example = new Example();
    example.read(new JBBPBitInputStream(new ByteArrayInputStream(data)));

    var expected = new Example();
    expected.byte1 = 1; // 00000001
    expected.bits = 5;  // 00000101
    expected.byte2 = 8; // 00001000

    System.out.println("byte1: " + example.byte1);
    System.out.println("bits:  " + example.bits);
    System.out.println("byte2: " + example.byte2);

    assertThat(example).usingRecursiveComparison().isEqualTo(expected);
  }
}

Can you help me to use your library to reach my expected outcome? Thank you!

[raydac]

by default JBBP reads from zero bit, and in your case you should read from 7th bit and reverse values

    byte [] data = new byte[]{0b00000001, (byte)0b10100001, 0b00000000};
    JBBPBitInputStream in = new JBBPBitInputStream(new ByteArrayInputStream(data), JBBPBitOrder.MSB0);
    assertEquals(1, JBBPFieldByte.reverseBits((byte)in.read()));
    assertEquals(5, JBBPFieldBit.reverseBits((byte)in.readBits(JBBPBitNumber.BITS_3), JBBPBitNumber.BITS_3));
    assertEquals(8, JBBPFieldByte.reverseBits((byte)in.read()));

    JBBPParser parser = JBBPParser.prepare("byte a; bit:3 b; byte c;", JBBPBitOrder.MSB0);
    JBBPFieldStruct struct = parser.parse(data);

    assertEquals(1, struct.findFieldForNameAndType("a", JBBPFieldByte.class).getAsInvertedBitOrder());
    assertEquals(5, struct.findFieldForNameAndType("b", JBBPFieldBit.class).getAsInvertedBitOrder());
    assertEquals(8, struct.findFieldForNameAndType("c", JBBPFieldByte.class).getAsInvertedBitOrder());

[Gerschtli]

Thank you for the quick response.

I am not that familiar with Big vs Little Endian but shouldn't this property solve this issue?

Also, I kind of struggle to understand the reasoning why JBBP starts on the last bit instead of the first bit when extracting n bits of the byte stream. For me it is not clear, why this library is "jumping around" between the bytes in the array.

IIUC JBBP seems to split this example like

00000001 10100001 00000000

00000001
              001
         10100    000 -> bytes are swaped and being merged into 00010100

[raydac]

at present it works in such way

    final byte[] data = new byte[] {0b0000_0001};

    JBBPBitInputStream in = new JBBPBitInputStream(new ByteArrayInputStream(data));
    assertEquals(0b0001, in.readBits(BITS_4));
    assertEquals(0b0000, in.readBits(BITS_4));
    assertEquals(0b0000_0001, new JBBPBitInputStream(new ByteArrayInputStream(data)).read());

    in = new JBBPBitInputStream(new ByteArrayInputStream(data), JBBPBitOrder.LSB0);
    assertEquals(0b0001, in.readBits(BITS_4));
    assertEquals(0b0000, in.readBits(BITS_4));
    assertEquals(0b0000_0001,
        new JBBPBitInputStream(new ByteArrayInputStream(data), JBBPBitOrder.LSB0).read());

    in = new JBBPBitInputStream(new ByteArrayInputStream(data), JBBPBitOrder.MSB0);
    assertEquals(0b0000, in.readBits(BITS_4));
    assertEquals(0b1000, in.readBits(BITS_4));
    assertEquals(0b1000_0000,
        new JBBPBitInputStream(new ByteArrayInputStream(data), JBBPBitOrder.MSB0).read());

so we get reversed data for MSB0 because it depends on filling of internal bit buffer during read, but looks that it is good idea to provide way to read and non reversed data, I will investigate how to add mode without reverse

[raydac]

I have added MSB0_DIRECT mode which doesn't make reverse during read and write, you can try 3.0.1-SNAPSHOT

[Gerschtli]

Awesome, thank you very much! I tested it and it works perfectly fine for my use case :)