gem5 gcn3 vega run database

[Yongze-zzz]

Hello, gem5-gcn3 ​​vega authors. I'm a database practitioner, trying to figure out how database software behaves under unified memory. I used an older version of gem5-gpu full-system mode to run the rodinia benchmark (but it produced unsolvable problems). I tried to run the square example using gem5-gcn3 ​​vega's apu_se.py, and it worked fine on my computer. I'm curious about how gem5-gcn3 ​​vega runs gpu full-system mode? It supports both cpu-side calling pthreads and gpu-side executing kernel functions. But I have a hard time understanding gem5, can you give me some advice?

[mattsinc]

@Yongze-zzz : I'm not sure I fully understand your question, but @abmerop put together instructions on installing GPUFS here: https://github.com/gem5/gem5-resources/blob/stable/src/x86-ubuntu-gpu-ml/BUILDING.md. I think this is what you are looking for?

[Yongze-zzz]

Running ./build.sh on my system produced the following problem：qemu.initialize: Error determining qemu version: exit status 1.

Then I try to run "qemu-system-x86_64 --version" ,it reported like below

[mattsinc]

@abmerop can you help with this? I am not an expert with your scripts yet.

[abmerop]

Do the other disk images also fail to build? I assume they would since this is a copy with additional packages. In that case it is not specific to gpu.

[Yongze-zzz]

Yes,abmerop is right. I tried running build.sh under x86-ubuntu and had the same problem. It seems that my qemu software is not a specific version, I have trouble finding a version that can support normal builds on my system.

[Yongze-zzz]

How to run gem5-gcn3-vega full system mode?

[abmerop]

There is a pre-built disk image that will be available on next release. You could try that on the develop branch. To do that you could try to follow the steps "To Reproduce" in this issue: #1195 .

[Yongze-zzz]

I run build/VEGA_X86/gem5.opt configs/example/gpufs/runfs.py --disk-image ../gem5-resources/src/x86-ubuntu-gpu-ml/disk-image/x86-ubuntu-gpu-ml --kernel ../gem5-resources/src/x86-ubuntu-gpu-ml/vmlinux-gpu-ml --gpu-mmio-trace ../gem5-resources/src/gpu-fs/vega_mmio.log. Than the system reports :panic: Need at least 2GB of system memory to load amdgpu module. My CPU is Intel 8360H and GPU is nvidia tesla t4. Does the simulator support to simulate amdgpu based on the nvidia physical device?

[abmerop]

No physical GPU is required, it is a simulated device.

I would recommend using the configs/example/gpufs/vega10_kvm.py helper script if you are on stable. If you are on the develop branch I would recommended using either configs/example/gpufs/mi200.py or configs/example/gpufs/mi300.py

Those helper scripts will set the memory size for you to avoid the error you are seeing. If you want to use runfs.py you'll have to add at least --mem-size=8GB or anything >2GB.

[Yongze-zzz]

Hi abmerop! Thanks for your reply! Your suggestion does work efficient.
After that, I try to add pthread function to support multi-thread in "gem5-resources/src/gpu/square/square.cpp" like below:

`#include <stdio.h>
#include "hip/hip_runtime.h"
#include <pthread.h>
#define CHECK(cmd)
{
hipError_t error = cmd;
if (error != hipSuccess) {
fprintf(stderr, "error: '%s'(%d) at %s:%d\n", hipGetErrorString(error), error,FILE, LINE);
exit(EXIT_FAILURE);
}
}
/*

• Thread function for printing "Hello, World!".
  /
  void print_hello(void* arg) {
  int thread_id = (int)arg;
  printf("Hello, World! from thread %d\n", thread_id);
  pthread_exit(NULL);
  }
  /*

• Square each element in the array A and write to array C.
  */
  template
  global void
  vector_square(T *C_d, const T *A_d, size_t N)
  {
  size_t offset = (hipBlockIdx_x * hipBlockDim_x + hipThreadIdx_x);
  size_t stride = hipBlockDim_x * hipGridDim_x ;

  for (size_t i=offset; i<N; i+=stride) {
  C_d[i] = A_d[i] * A_d[i];
  }
  }
  int main(int argc, char *argv[])
  {
  float A_h, C_h;
  size_t N = 1000000;
  size_t Nbytes = N * sizeof(float);
  hipDeviceProp_t props;
  CHECK(hipGetDeviceProperties(&props, 0/deviceID/));
  printf ("info: running on device %s\n", props.name);
  #ifdef HIP_PLATFORM_HCC
  printf ("info: architecture on AMD GPU device is: %d\n",props.gcnArch);
  #endif
  printf ("info: allocate host and device mem (%6.2f MB)\n", 2Nbytes/1024.0/1024.0);
  CHECK(hipHostMalloc(&A_h, Nbytes));
  CHECK(hipHostMalloc(&C_h, Nbytes));
  // Fill with Phi + i
  for (size_t i=0; i<N; i++)
  {
  A_h[i] = 1.618f + i;
  }
  const unsigned blocks = 512;
  const unsigned threadsPerBlock = 256;
  printf ("info: launch 'vector_square' kernel\n");
  hipLaunchKernelGGL(vector_square, dim3(blocks), dim3(threadsPerBlock), 0, 0, C_h, A_h, N);
  hipDeviceSynchronize();
  printf ("info: check result\n");
  for (size_t i=0; i<N; i++) {
  if (C_h[i] != A_h[i] * A_h[i]) {
  CHECK(hipErrorUnknown);
  }
  }
  printf ("AMDGPU PASSED!\n");
  const int NUM_THREADS = 4;
  pthread_t threads[NUM_THREADS];
  int thread_ids[NUM_THREADS];
  // Create threads to print "Hello, World!"
  for (int i = 0; i < NUM_THREADS; i++) {
  thread_ids[i] = i;
  int rc = pthread_create(&threads[i], NULL, print_hello, (void)&thread_ids[i]);
  if (rc) {
  fprintf(stderr, "Error: unable to create thread %d\n", rc);
  // exit(EXIT_FAILURE);
  }
  }
  // Wait for all threads to complete
  for (int i = 0; i < NUM_THREADS; i++) {
  pthread_join(threads[i], NULL);
  }
  return 0;
  }`

Then I run /root/gem5space/gem5/build/GCN3_X86/gem5.opt /root/gem5space/gem5/configs/example/gpufs/runfs.py --disk-image /root/gem5space/gem5-resources/src/x86-ubuntu-gpu-ml/disk-image/x86-ubuntu-gpu-ml --kernel /root/gem5space/gem5-resources/src/x86-ubuntu-gpu-ml/vmlinux-gpu-ml --gpu-mmio-trace /root/gem5space/gem5-resources/src/gpu-fs/vega_mmio.log --mem-size=8GB --script /root/gem5space/gem5-resources/src/gpu/square/square.rcS.

The script is cd ./bin /sbin/m5 resetstats ./square /sbin/m5 exit

The output likes below:

I'm not sure if it actually executes, as I don't see any output from the program.
Also, is the program emulating a unified memory APU, like the software virtualization architecture described in this slide （https://old.gem5.org/wiki/images/1/19/AMD_gem5_APU_simulator_isca_2018_gem5_wiki.pdf）?