Add NUMA support and optimizations #11871

tlichwala · 2024-11-15T13:50:07Z

Summary:
This pull request adds support for Non-Uniform Memory Access (NUMA) to Apache Traffic Server, enhancing performance on NUMA systems.

Key Changes:

Added CMake options to enable NUMA support and debugging.
Introduced new configuration options for NUMA optimizations.
Enhanced thread and memory management to be NUMA-aware.
Added RamCacheContainer for cache duplication across NUMA nodes.
Integrated NUMA debugging utilities.

Performance testing has shown increased throughput, reduced CPU usage, improved latency, decreased UPI bus load

These changes aim to optimize memory access patterns and reduce latency on NUMA systems.

To build the application with NUMA support, use the following CMake command:
cmake -B build -DCMAKE_BUILD_TYPE=Release -DENABLE_MIMALLOC=ON -DENABLE_NUMA=ON -DENABLE_HWLOC=ON -Dmimalloc_DIR=""

zwoop · 2024-11-15T16:53:56Z

Nice! The PR uses the old Debug() statements, which needs to be changed to the new Dbg / Ctl features:

../src/iocore/net/Server.cc:247:3: error: use of undeclared identifier 'Debug'
  Debug("numa", "[Server::listen] Attempting to create socket with family: %d, type: %d, protocol: %d", addr.sa.sa_family,
  ^
../src/iocore/net/Server.cc:256:3: error: use of undeclared identifier 'Debug'
  Debug("numa", "[Server::listen] Attempting to set up fd for listen with non_blocking: %d, options: %d", non_blocking, opt);
  ^

cmcfarlen

Thanks so much for this PR! It looks really interesting. Just a few comments to get ci happy. Mainly migrating Debug->Dbg. Looking forward to trying this out!

cmcfarlen · 2024-11-15T17:03:34Z

src/cripts/Lulu.cc

+{
+  return details::splitter<Cript::string_view>(input, delim);
+}
+


You can remove this chunk, perhaps a merge conflict issue.

cmcfarlen · 2024-11-15T17:05:08Z

src/iocore/net/Server.cc

+    int my_thread_id = this_ethread()->id;
+    int my_numa_node = this_ethread()->get_numa_node();
+
+    Debug("numa_sequencer", "[NUMASequencer] Thread %d (NUMA node %d) entered run_sequential.", my_thread_id, my_numa_node);


Debug has been removed, please use the Dbg macro instead.

cmcfarlen · 2024-11-15T17:05:58Z

src/iocore/net/Server.cc

  if (opt.f_mptcp) {
    Dbg(dbg_ctl_connection, "Define socket with MPTCP");
    prot = IPPROTO_MPTCP;
  }

+  // Create the socket
+  Debug("numa", "[Server::listen] Attempting to create socket with family: %d, type: %d, protocol: %d", addr.sa.sa_family,


These Debug lines were maybe for your debug? Could they be removed?

cmcfarlen · 2024-11-15T18:21:31Z

src/iocore/cache/RamCacheContainer.cc

+}
+
+RamCache *
+RamCacheContainer::get_cache(unsigned int my_node, unsigned int node)


Suggested change

RamCacheContainer::get_cache(unsigned int my_node, unsigned int node)

RamCacheContainer::get_cache(unsigned int /* my_node ATS_UNUSED */, unsigned int node)

cmcfarlen · 2024-11-15T18:23:05Z

src/iocore/cache/RamCacheContainer.cc

+
+// returns true if consistent
+static bool
+check_pages_consistency(void *data, size_t size, const char *name = "")


This function is unused unless NUMA_CONSISTENCY_CHECK is defined. Please add precompile check here.

cmcfarlen · 2024-11-15T18:23:21Z

src/iocore/cache/RamCacheContainer.cc

+}
+
+static void
+move_pages_to_current_numa_zone(void *data, size_t size)


This function is unused

cmcfarlen · 2024-11-15T18:24:11Z

src/iocore/net/Server.cc

+
+    // If all threads have been added (assuming their number is equal to eventProcessor.net_threads), sort the thread IDs and set
+    // ready_to_run to true
+    if (thread_ids.size() == eventProcessor.net_threads) {


I get a signed/unsigned compare error for this line.

cmcfarlen

The build is failing for systems without numa.h. I noted one place, but there could be other similar issues.

include/iocore/eventsystem/EThread.h

tlichwala · 2024-12-02T11:31:11Z

Hi all,
I've addressed the feedback and made the necessary changes. However, the automated tests are still showing build errors on some operating systems. Could someone please help review these issues?
Thank you!

cmcfarlen · 2024-12-02T22:21:23Z

It looks like its still related to when numa is not available:

../src/iocore/net/QUICPacketHandler.cc: In member function 'virtual void QUICPacketHandlerIn::_recv_packet(int, UDPPacket*)':
../src/iocore/net/QUICPacketHandler.cc:270:65: error: no matching function for call to 'EventProcessor::assign_thread(const int&)'
  270 |     eth                           = eventProcessor.assign_thread(ET_NET);
      |                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
In file included from ../src/iocore/net/../eventsystem/P_EventSystem.h:46,
                 from ../src/iocore/net/P_Net.h:92,
                 from ../src/iocore/net/QUICPacketHandler.cc:25:
../src/iocore/net/../eventsystem/P_UnixEventProcessor.h:54:1: note: candidate: 'EThread* EventProcessor::assign_thread(EventType, int)'
   54 | EventProcessor::assign_thread(EventType etype, int numa_node)
      | ^~~~~~~~~~~~~~
../src/iocore/net/../eventsystem/P_UnixEventProcessor.h:54:1: note:   candidate expects 2 arguments, 1 provided

tlichwala · 2024-12-04T09:38:12Z

include/tscore/ink_thread.h

 #include <sched.h>
+#if TS_USE_HWLOC
+#include <hwloc.h>
+#include <hwloc/glibc-sched.h>


I am unable to reproduce the compilation error that occurs when including the header file <hwloc/glibc-sched.h> on freebsd. The error message is: "Please make sure to include sched.h before including glibc-sched.h, and define _GNU_SOURCE before any inclusion of sched.h". I added the undesirable definition of the _GNU_SOURCE macro in the code, as shown below, to ensure that it is defined.

hmm, I'm not super familiar with freebsd, but I think this should be added to CXXFLAGS as part of the build setup, not in the source code. Is Freebsd able to find the numa library? This feature could be disabled for that OS for now.

The _GNU_SOURCE macro is defined in the CMakeLists.txt file as follows:
list(APPEND CMAKE_REQUIRED_DEFINITIONS -D_GNU_SOURCE -DATS_BUILD)
I added the macro in the source code only for testing purposes to ensure that it is indeed defined.
It appears that the error I am encountering is not related to the numa.h library but rather to hwloc.
Thank you for your suggestion. If you have any further insights or recommendations, I would appreciate it.

cmcfarlen · 2024-12-05T16:24:24Z

You might need to add more precompiler conditions for TS_USE_NUMA to keep the errors down for systems that don't support this.

cmcfarlen · 2024-12-18T22:12:56Z

Unfortunately CMakeLists.txt is conflicting. You can just accept both the new and old line when resolving.

- Enable NUMA optimizations via CMake options - Introduce NUMA-aware thread assignment and memory allocation - Add NUMA debugging utilities - Enhance cache management for NUMA systems

apache deleted a comment from ezelkow1 Nov 15, 2024

zwoop added Core Performance labels Nov 15, 2024

zwoop added this to the 10.1.0 milestone Nov 15, 2024

zwoop assigned tlichwala Nov 15, 2024

cmcfarlen requested changes Nov 15, 2024

View reviewed changes

cmcfarlen requested changes Nov 20, 2024

View reviewed changes

include/iocore/eventsystem/EThread.h Show resolved Hide resolved

tlichwala commented Dec 4, 2024

View reviewed changes

tlichwala force-pushed the Numa-awareness-patch branch 4 times, most recently from 3b23f05 to 5f061f2 Compare December 13, 2024 11:45

tlichwala force-pushed the Numa-awareness-patch branch from 5f061f2 to 9712d65 Compare January 9, 2025 08:51

tlichwala added 6 commits January 9, 2025 09:03

Add NUMA support and optimizations

441903c

- Enable NUMA optimizations via CMake options - Introduce NUMA-aware thread assignment and memory allocation - Add NUMA debugging utilities - Enhance cache management for NUMA systems

Update Debug message method.

1920f18

Add getcpu() only with ENABLE_NUMA option.

59a61ee

Add define to resolve issue with glibc-sched.h

e3b470e

Adding hwloc elements to numa optimizations

7d14513

Remove duplicated hwloc include

91a6cc6

tlichwala force-pushed the Numa-awareness-patch branch from 9712d65 to 91a6cc6 Compare January 9, 2025 09:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add NUMA support and optimizations #11871

Add NUMA support and optimizations #11871

tlichwala commented Nov 15, 2024

zwoop commented Nov 15, 2024

cmcfarlen left a comment

cmcfarlen Nov 15, 2024

cmcfarlen Nov 15, 2024

cmcfarlen Nov 15, 2024

cmcfarlen Nov 15, 2024

cmcfarlen Nov 15, 2024

cmcfarlen Nov 15, 2024

cmcfarlen Nov 15, 2024

cmcfarlen left a comment

tlichwala commented Dec 2, 2024

cmcfarlen commented Dec 2, 2024

tlichwala Dec 4, 2024

cmcfarlen Dec 5, 2024

tlichwala Dec 5, 2024

cmcfarlen commented Dec 5, 2024

cmcfarlen commented Dec 18, 2024

	RamCacheContainer::get_cache(unsigned int my_node, unsigned int node)
	RamCacheContainer::get_cache(unsigned int /* my_node ATS_UNUSED */, unsigned int node)

Add NUMA support and optimizations #11871

Are you sure you want to change the base?

Add NUMA support and optimizations #11871

Conversation

tlichwala commented Nov 15, 2024

zwoop commented Nov 15, 2024

cmcfarlen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmcfarlen left a comment

Choose a reason for hiding this comment

tlichwala commented Dec 2, 2024

cmcfarlen commented Dec 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmcfarlen commented Dec 5, 2024

cmcfarlen commented Dec 18, 2024