Skip to content

Commit db79955

Browse files
mjao1mjao1
committed
update readme and SHORT_DELTA
1 parent a56ce83 commit db79955

File tree

2 files changed

+52
-107
lines changed

2 files changed

+52
-107
lines changed

‎README.md

+51-106
Original file line numberDiff line numberDiff line change
@@ -83,117 +83,64 @@ Once a tree is created, the common operation is to delete nodes, and at most
8383
insert phi nodes (SSA). This means that the insertion is for a few entries, not
8484
large subtrees.
8585

86-
87-
The traversal is a topological sort equivalent where parents of leafs are
88-
visited first, and then to process them the leafs are accessed. In a way, the
89-
following is a typical AST traversal:
90-
86+
### Key Features
87+
88+
- Efficient sibling traversal with next/previous pointers
89+
- Optimized for append operations rather than random insertions
90+
- Support for subtree references and sharing between trees via Forest container
91+
- Three traversal modes: pre-order, post-order, and sibling-order
92+
- Memory-efficient storage using chunks of 8 nodes
93+
- Delta-compressed child pointers for common cases
94+
- Tombstone deletion (IDs are not reused)
95+
- Leaf-focused operations (delete_leaf is optimized)
96+
97+
### Data Structure Design
98+
99+
The tree uses a chunked storage approach where nodes are stored in fixed-size chunks of 8 entries. Each chunk contains:
100+
101+
- Long pointers (49-bit) for first/last child when needed
102+
- Delta-compressed short pointers (18-bit) for child references within nearby chunks
103+
- Next/Previous sibling pointers for efficient sibling traversal
104+
- Parent pointers for upward traversal
105+
- Leaf flag for quick leaf checks
106+
- Support for subtree references
107+
108+
The structure is particularly optimized for:
109+
1. Append operations (adding siblings or children at the end)
110+
2. Leaf deletion
111+
3. Sibling traversal
112+
4. Child traversal for small families (< 8 children)
113+
114+
### Traversal Support
115+
116+
The tree provides three iterator types:
117+
```cpp
118+
// Pre-order traversal (parent then children)
119+
for(auto node : tree.pre_order(start_pos)) { ... }
120+
121+
// Post-order traversal (children then parent)
122+
for(auto node : tree.post_order(start_pos)) { ... }
123+
124+
// Sibling-order traversal (iterate through siblings)
125+
for(auto node : tree.sibling_order(start_pos)) { ... }
91126
```
92-
Index: first Child, parent
93-
94-
01: 00,03 │ +── 1.1.1
95-
02: 00,03 │ |── 1.1.2
96-
03: 01,32 -── 1.1
97-
04: 00,06 │ +── 1.2.1.1.1
98-
05: 00,06 │ |── 1.2.1.1.2
99-
06: 04,13 │ +── 1.2.1.1
100-
07: 00,08 │ +── 1.2.1.2.1
101-
08: 07,13 │ |── 1.2.1.2
102-
09: 00,12 │ +── 1.2.1.3.1
103-
10: 00,12 │ |── 1.2.1.3.2
104-
11: 00,12 │ |── 1.2.1.3.3
105-
12: 09,13 │ |── 1.2.1.3
106-
13: 04,27 │ +── 1.2.1
107-
14: 00,16 │ +── 1.2.2.1
108-
15: 00,16 │ |── 1.2.2.2
109-
16: 14,27 │ ├── 1.2.2
110-
17: 00,19 │ +── 1.2.3.1.1
111-
18: 00,19 │ |── 1.2.3.1.2
112-
19: 17,23 │ +── 1.2.3.1
113-
20: 00,22 │ +── 1.2.3.2.1
114-
21: 00,22 │ |── 1.2.3.2.2
115-
22: 20,23 │ +── 1.2.3.2
116-
23: 19,27 │ |── 1.2.3
117-
24: 00,26 │ +── 1.2.4.1
118-
25: 00,26 │ |── 1.2.4.2
119-
26: 24,27 │ |── 1.2.4
120-
27: 13,32 ├── 1.2
121-
28: 00,30 │ |── 1.3.1
122-
29: 00,30 │ |── 1.3.2
123-
30: 00,32 ├── 1.3
124-
31: 00,32 ├── 1.4
125-
32: 00,00 | 1
126-
```
127-
128-
An alternative data structure.
129127
128+
Each traversal mode also supports following subtree references via an optional flag:
129+
```cpp
130+
// Follow subtree references during traversal
131+
for(auto node : tree.pre_order(start_pos, true)) { ... }
130132
```
131-
Index: last_child, parent
132-
// 01: 16,00 | 1
133-
// 02: 00,01 -── 1.1
134-
// 03: 04,01 ├── 1.2
135-
// 04: 05,03 │ -── 1.2.1
136-
// 05: 00,04 │ -── 1.2.1.1
137-
// 06: 00,04 │ ├── 1.2.1.2
138-
// 07: 10,01 ├── 1.3
139-
// 08: 00,07 │ -── 1.3.1
140-
// 09: 00,07 │ ├── 1.3.2
141-
// 10: 13,07 │ ├── 1.3.3
142-
// 11: 00,10 │ -── 1.3.1.1
143-
// 12: 00,10 │ |── 1.3.1.2
144-
// 13: 15,10 │ |── 1.2.1.3
145-
// 14: 00,13 │ -── 1.2.1.3.1
146-
// 15: 00,13 │ -── 1.2.1.3.2
147-
// 16: 20,01 ├── 1.4
148-
// 17: 00,16 │ -── 1.4.1
149-
// 18: 19,16 │ ├── 1.4.2
150-
// 19: 00,18 │ │ -── 1.4.2.1
151-
// 20: 21,16 │ ├── 1.4.3
152-
// 21: 00,20 │ │ -── 1.4.3.1
153-
```
154-
155-
The previous data structure fits quite well with most AST (LiveHD) tree operations with the exception of two popular ones: get_next_sibling and add_child.
156133

157-
An alternative data structure. The siblings are consecutive, the descendent from the siblings are after all the siblings.
134+
### Forest Support
158135

136+
Trees can share subtrees via the Forest container:
137+
```cpp
138+
Forest<int> forest;
139+
Tree_pos subtree = forest.create_tree(root_data);
140+
tree.add_subtree_ref(node_pos, subtree);
159141
```
160-
Index: first_child, last_child, parent
161-
// 01: 02,05,00 | 1
162-
// 02: 00,00,01 -── 1.1
163-
// 03: 06,06,01 ├── 1.2
164-
// 04: 12,14,01 ├── 1.3
165-
// 05: 17,19,01 ├── 1.4
166-
// 06: 07,09,03 │ -── 1.2.1
167-
// 07: 00,00,06 │ -── 1.2.1.1
168-
// 08: 00,00,06 │ ├── 1.2.1.2
169-
// 09: 10,11,06 │ |── 1.2.1.3
170-
// 10: 00,00,09 │ -── 1.2.1.3.1
171-
// 11: 00,00,09 │ -── 1.2.1.3.2
172-
// 12: 15,16,04 │ -── 1.3.1
173-
// 13: 00,00,04 │ ├── 1.3.2
174-
// 14: 00,00,04 │ ├── 1.3.3
175-
// 15: 00,00,12 │ -── 1.3.1.1
176-
// 16: 00,00,12 │ |── 1.3.1.2
177-
// 17: 00,00,05 │ -── 1.4.1
178-
// 18: 20,21,05 │ ├── 1.4.2
179-
// 19: 00,00,05 │ ├── 1.4.3
180-
// 20: 00,00,18 │ │ -── 1.4.2.1
181-
// 21: 00,00,18 │ │ -── 1.4.3.1
182-
```
183-
184-
The previous structure handles everything quite fast with the exception of add_child
185-
186-
add_child tends to add nodes close to the end. In which case, and append or just shifting a few nodes works fast.
187-
188-
Some solutions for random location add_child:
189-
190-
-Support a small hashmap with "other children" that did not fit. Only insert starting from last_child, and change last_child to 0 to indicate that last children are in a hashmap.
191-
192-
-Allow it BUT trigger a message to fix. This should be rare and easy to fix so that calls are not in that order
193-
194-
-Chunk the vector to blocks of 16K entries. This allows smaller pointers (first_child,last_child, parent) using shorts (16bits). WHen a pointer needs to point to other table (16K entry max per table),
195-
it can have a all ones value (-1) to indicate that the following entry uses the 3 fields (16 bits) as pointer to the larger table (32 bits) and 16 bits as offset.
196142

143+
The Forest manages reference counting and cleanup of shared subtrees.
197144

198145
### Related
199146

@@ -202,5 +149,3 @@ Not same, but similar idea and different representation:
202149
Meyerovich, Leo A., Todd Mytkowicz, and Wolfram Schulte. "Data parallel programming for irregular tree computations." (2011).
203150

204151
Vollmer, Michael, Sarah Spall, Buddhika Chamith, Laith Sakka, Chaitanya Koparkar, Milind Kulkarni, Sam Tobin-Hochstadt, and Ryan R. Newton. "Compiling tree transforms to operate on packed representations." (2017): 26.
205-
206-

‎hhds/tree.hpp

+1-1
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ static constexpr Tree_pos INVALID = 0; // This is invalid for al
5454
static constexpr Tree_pos ROOT = 1 << CHUNK_SHIFT; // ROOT ID
5555

5656
static constexpr int CHUNK_BITS = 49;
57-
static constexpr int SHORT_DELTA = 17;
57+
static constexpr int SHORT_DELTA = 18;
5858

5959
static constexpr uint64_t MAX_TREE_SIZE = 1LL << CHUNK_BITS; // Maximum number of chunks in the tree
6060

0 commit comments

Comments
 (0)