how to realize a 3-level cached

[cery999]

hi,

I hope to use halide to simulate a three-level cache architecture for cpu. and the memory size is divided by each top layer. such as

     L3.    ->     L2       ->     L1         ->       L2      ->      L3
(256*256) -> 16 * (16*256) -> 16 * 4 * (4*256) -> 16 * (16*256) -> (256*256)

I use this cpp function to run

Func l1, l2, l3, l2_out, l3_out;
l3.store_in(MemoryType::L3);
l2.store_in(MemoryType::L2);
l1.store_in(MemoryType::L1);
l3_out.store_in(MemoryType::L3);
l2_out.store_in(MemoryType::L2);
auto l2_size = 16*256, l1_size = 4*256; 
for (auto i = 0; i < 16; i++) { // 16 times l3->l2
    RDom r_l2(0, l2_size, "l2_reduce");
    l2(x2) = l3(x2);
    l2(r_l2) = l3(i * l2_size + r_l2);
    for (auto j = 0; j < 4; j++) { // 4 times l1->l2
        RDom r_l1(0, l1_size, "l1_reduce");
        l1(x1) = l2(j * l1_size + r_l1);
        l2_out(j*l1_size+r_l1) = l1(r_l1);
    }
    l3_out(i*l2_size + r_l2) = l2_out(r_l2);
}

it seems I can't define reduction in pure fuction definition.

terminate called after throwing an instance of 'Halide::CompileError'
  what():  Error: In pure definition of Func "l1$0":
Reduction domain referenced in pure function definition.

is there any way to run it?

[abadams]

Those for loops make me worry you're misunderstanding the metaprogramming aspect of Halide. Each loop iteration you have adds new definitions for those Funcs, rather than actually moving any data around. I'd suggest starting with the tutorials here: https://halide-lang.org/tutorials/tutorial_introduction.html

[cery999]

maybe I mistaken the for loop defination. that's was the new version to illustrate, the reason why I add new for loop in the previous version is that, if the memory is 1K -> 4K ,there will be 4 times sizes, so I add 4 defination for that case.

For Now, Halide is designed for a cache-architecture device, but how to use it for cache-less design?

the previous paper have try expand halide to support IPU: 《Extending Halide to Improve Software Development for Imaging DSPs》

we want to try expand halide to support our RsicV architecture. what we have defined is HBM -> L3 cache, CSB -> L2 cache, SIP -> L1 cache. different from cuda and OpenCL, the real compute is happen in the L1 cache, so we need to manipulate DMA transfer for good locality. our device is just like GPU, names DTU.
but what I confused was that how to transfer DMA process for different size

for example,

large (6M)-> middle(1M) -> small(64K) -> middle(1M) -> large(6M)

seeing the large -> middle -> small, this can be thought by reduce the image in little size. but how to explain 64k->1M. the size will overflow the buffer size. I look trough the sample code , the CUDA and arm,x86，their cache is just managed by the device, so there is no transfer for DMA operation. So our DMA transfer will be very complex, because there will be 5 pipeline stage for illustrate the process.

here is the example,

 // simple 3 level memory hierarchy for DTU
Func l1("l1"), l2("l2"), l3("l3"), l2_out("l2_out"), l3_out("l3_out");
auto l3_size = 256 * 256, l2_size = 16 * 256, l1_size = 4 * 256;
auto l3_scale = l3_size / l2_size, l2_scale = l2_size / l1_size;
Var x, y, z;
// HBM -> CSB -> SIP -> CSB -> HBM
// (256*256) -> 16 * (16*256) -> 16 * 4 * (4*256) -> 16 * (16*256) -> (256*256)
l3.store_in(MemoryType::HBM);
l2.store_in(MemoryType::CSB);
l1.store_in(MemoryType::SIP);
l2_out.store_in(MemoryType::CSB);
l3_out.store_in(MemoryType::HBM);

// value define
l3(x) = x;
l2(x) = l3(x);
l1(x) = l2(x);
l1(x) = l1(x) + 1;
l3_out(x) = l1(x);
l3_out(x) = l2_out(x);

Var tx, ty, bx, by, bxo, bxi;
// schedule
l3.bound(x, 0, 256 * 256).gpu_tile(x, bx, tx, 4 * 256, TailStrategy::Auto, DeviceAPI::EnflameDTU).split(bx, bxo, bxi, 16).compute_at(l2, Var::outermost());
l2.bound(x, 0, 16 * 256).gpu_tile(x, bx, tx, 4 * 256, TailStrategy::Auto, DeviceAPI::EnflameDTU).split(bx, bxo, bxi, 4).compute_at(l1, Var::outermost());
l1.bound(x, 0, 4 * 256).gpu_tile(x, bx, tx, 4 * 256, TailStrategy::Auto, DeviceAPI::EnflameDTU);

here is the Halide IR

produce l1:
  produce l2:
    produce l3:
      gpu_block v0.v5.v7 in [0, 3]<EnflameDTU:Dorado>:
        gpu_block v0.v5.v8 in [0, 15]<EnflameDTU:Dorado>:
          gpu_thread v0.v3 in [0, 1023]<EnflameDTU:Dorado>:
            l3(...) = ...
    consume l3:
      gpu_block v0.v5.v7 in [0, 0]<EnflameDTU:Dorado>:
        gpu_block v0.v5.v8 in [0, 3]<EnflameDTU:Dorado>:
          gpu_thread v0.v3 in [0, 1023]<EnflameDTU:Dorado>:
            l2(...) = ...
  consume l2:
    gpu_block v0.v5 in [0, 0]<EnflameDTU:Dorado>:
      gpu_thread v0.v3 in [0, 1023]<EnflameDTU:Dorado>:
        l1(...) = ...
  for v0 in [0, 1023]:
    l1(...) = ...
Success!

thanks for kindly support.

[cery999]

finally, I understand the meta function, so I need to add dma instruction with schedule functions , thanks.