HOB format: remaining unknown fields

[BenChampion]

Starting a thread for findings on the remaining unknown fields in the HOB format.

dbg/dbg_HOB is very useful since it is a small file with only 3 simple objects. (#9 brings it into the model_viewer)

With reference to https://github.com/dpethes/rerogue/blob/master/doc/file_hob_spec.txt

Object [116B] 
{
  16B array of char : name
  4B int   : facegroups offset
  4B int   : object parts offset
  4B int   : facegroup header 2 offset
  12B zero

These ints seem to maybe be offsets, but to/from what?

  4B int   : ? 94, 0xD4
  4B int   : ? 98, 0x100
  4B int   : ? 9C, 0x104

  4B float : ?
      -bark_moon, cldcar: 1.0
      -sky, e_cor: 0.2 
      -wmvwing: 1.25
  12B zero
  5x 4B float : ?
  4B int offset before 0xFFFFFFFF header end marker

I am pretty sure these last 6 floats are bounding box corners (x_min, y_min, z_min) (x_max, y_max, z_max)

  6x 4B float : ?`
}

I think the 4th byte in each vertex is just padding? Is it ever anything except 0?

[...]
Vertices
{
  2B int X
  2B int Y
  2B int Z
  2B int ?
}

[BenChampion]

I wrote a Python parser using construct as a tool to investigate the file further. It works for data/dbg/dbg_HOB but not for other files - although the headers can be successfully parsed in some cases (I hope to add more functionality to make it able to handle stuff it doesn't understand yet, rather than just stopping - maybe I need to rewrite it in a more "offset-based" way).

A nice feature of construct is that it is symmetric, so (in theory) the same data structure can be used for parsing or for writing a file. For now it is just a computer-readable version of the spec...

Using this parser (and manual work with a hex editor), some interesting things I've noticed:

• object_effects: this array (based on the strings) looks like it might describe parts of the object with reflective/emissive properties, like cockpit canopy glass, engine/engine glow, and the chrome on /data2/koelsch_HOB.
• unknown_matrices_maybe_orientation: these structs don't seem to contain any offsets and have strings like "ab", "scnd", "zf", "zt", "zb", "zr", "zl", and "trl" (these names look similar for the player craft I looked at, but are different in e.g. data/imp_stuff/atat_HOB). I call them "matrices" but I don't know what they are for sure.
• imp_stuff/tieinter_HOB seems to have another header following the 0xFFFFFFFF of the first one. The offsets to the two parts of this header are right after the offsets for the "normal" header, they are the first 8B of what is normally 12B of zeros in the Object struct.

[SubDrag]

I agree with the notes here, though I am focusing on the N64 version, which are similar (though not 100% identical), they match my investigation. I don't think it's extra 8 offset - it matches num indexes of 16-bits. At least on N64, it also pads some sections to 4, so that explains often times extra 0000s.

And there are more to these bitflags for face flags. Still investigating, but parsing is definitely more going on than what is currently identified. For example, separate vertex colors can be on (and include colors), with vertex colors off.

Are either of you still looking into this? I was thinking of looking into this some more.

[BenChampion]

Go for it! I haven't touched this for four years.

[SubDrag]

Separate vertexes and has UVs are always true flags, at end. Has color seems to be incorrectly identified, and all faces either have separate vertex flag on, or single vertex if it's off.

So the only confirmed flags to me are:
hasUVs (1 << 2)
isQuad (1 << 3)
separateVertexColors (1 << 4)
extraIndexes (1 << 6)

The real issue seems to be in determining the extra indexes, which I assume are normals (but didn't fully confirm). There is a flag, as you note, but it's only for sure used if face flags & 0x00000060. Otherwise, some are always off, some are on or off, with same face flags. Something determines it, outside of face I presume, just have been really unable to figure it out sadly (examples 000002BE, 0000023E, tat_sun.HOB, ywing.HOB). I suppose it's possible has to do with the material associated with a face, haven't checked that out too much; isn't anything else in the single group/face, as far as I can tell.