Option to build without the standard library

[barakugav]

🚀 The feature, motivation and pitch

ExecuTorch is intended for use in edge devices, even embedded systems. In these use cases it is required to reduce the size of the executable, and ExecuTorch are proud of the core library to be only few dozes of Kb. I think it will be very useful to be able compile without the standard CPP library.

The library already give a great flexibility for custom allocators, and I don't think there is some critical code that must have std.
From my attempt of compiling the lib without std I saw some use of std errors, clocks (which maybe I can override by providing another implementation for the platform/ headers), allocations in MemoryAllocator (maybe we can solve it with a cmake flag), and exceptions (rarely used, saw in some backends loading code).

I wrote Rust bindings for the library, and one of the features I would like to offer is exactly that, to compile without the std lib. In Rust you just add #![no_std] to the crate, in CPP you should do -nostdlib.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

[barakugav]

Tried to link an executable without the std lib, here is the linker error:

Undefined symbols for architecture arm64:
"std::logic_error::logic_error(char const*)", referenced from:
    std::length_error::length_error[abi:ue170006](char const*) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"std::length_error::~length_error()", referenced from:
    std::__1::__throw_length_error[abi:ue170006](char const*) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"std::bad_array_new_length::bad_array_new_length()", referenced from:
    std::__throw_bad_array_new_length[abi:ue170006]() in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"std::bad_array_new_length::~bad_array_new_length()", referenced from:
    std::__throw_bad_array_new_length[abi:ue170006]() in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"std::__1::chrono::steady_clock::now()", referenced from:
    _et_pal_init in libexecutorch_sys-8e629529a73be94e.rlib[22](Posix.cpp.o)
    _et_pal_current_ticks in libexecutorch_sys-8e629529a73be94e.rlib[22](Posix.cpp.o)
"std::terminate()", referenced from:
    ___clang_call_terminate in libportable_kernels.a[11](op_any.cpp.o)
"typeinfo for std::length_error", referenced from:
    std::__1::__throw_length_error[abi:ue170006](char const*) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"typeinfo for std::bad_array_new_length", referenced from:
    std::__throw_bad_array_new_length[abi:ue170006]() in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"vtable for __cxxabiv1::__class_type_info", referenced from:
    typeinfo for torch::executor::MemoryAllocator in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    typeinfo for torch::executor::DataLoader in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
"vtable for __cxxabiv1::__si_class_type_info", referenced from:
    typeinfo for torch::executor::util::MallocMemoryAllocator in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    typeinfo for torch::executor::util::FileDataLoader in libexecutorch_sys-8e629529a73be94e.rlib[24](file_data_loader.cpp.o)
    NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
"vtable for std::length_error", referenced from:
    std::length_error::length_error[abi:ue170006](char const*) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
"operator delete(void*)", referenced from:
    torch::executor::MemoryAllocator::~MemoryAllocator() in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    torch::executor::util::MallocMemoryAllocator::~MallocMemoryAllocator() in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    void std::__1::__libcpp_operator_delete[abi:ue170006]<void*>(void*) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    torch::executor::util::FileDataLoader::~FileDataLoader() in libexecutorch_sys-8e629529a73be94e.rlib[24](file_data_loader.cpp.o)
"operator delete(void*, std::align_val_t)", referenced from:
    void std::__1::__libcpp_operator_delete[abi:ue170006]<void*, std::align_val_t>(void*, std::align_val_t) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"operator new(unsigned long)", referenced from:
    void* std::__1::__libcpp_operator_new[abi:ue170006]<unsigned long>(unsigned long) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"operator new(unsigned long, std::align_val_t)", referenced from:
    void* std::__1::__libcpp_operator_new[abi:ue170006]<unsigned long, std::align_val_t>(unsigned long, std::align_val_t) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"___cxa_allocate_exception", referenced from:
    std::__1::__throw_length_error[abi:ue170006](char const*) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    std::__throw_bad_array_new_length[abi:ue170006]() in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"___cxa_begin_catch", referenced from:
    ___clang_call_terminate in libportable_kernels.a[11](op_any.cpp.o)
"___cxa_free_exception", referenced from:
    std::__1::__throw_length_error[abi:ue170006](char const*) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"___cxa_guard_abort", referenced from:
    torch::executor::getBackendRegistry() in libexecutorch_sys-8e629529a73be94e.rlib[8](interface.cpp.o)
    torch::executor::getOperatorRegistry() in libexecutorch_sys-8e629529a73be94e.rlib[17](operator_registry.cpp.o)
"___cxa_guard_acquire", referenced from:
    torch::executor::getBackendRegistry() in libexecutorch_sys-8e629529a73be94e.rlib[8](interface.cpp.o)
    torch::executor::getOperatorRegistry() in libexecutorch_sys-8e629529a73be94e.rlib[17](operator_registry.cpp.o)
"___cxa_guard_release", referenced from:
    torch::executor::getBackendRegistry() in libexecutorch_sys-8e629529a73be94e.rlib[8](interface.cpp.o)
    torch::executor::getOperatorRegistry() in libexecutorch_sys-8e629529a73be94e.rlib[17](operator_registry.cpp.o)
"___cxa_pure_virtual", referenced from:
    vtable for torch::executor::DataLoader in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    vtable for torch::executor::DataLoader in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"___cxa_throw", referenced from:
    std::__1::__throw_length_error[abi:ue170006](char const*) in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    std::__throw_bad_array_new_length[abi:ue170006]() in libexecutorch_sys-8e629529a73be94e.rlib[4](842ff64ee6e2bf22-api_utils.o)
"___gxx_personality_v0", referenced from:
    /Users/barak/code/executorch-rs/executorch-sys/third-party/executorch/cmake-out/kernels/portable/libportable_ops_lib.a[2](RegisterCodegenUnboxedKernelsEverything.cpp.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[dbort]

MallocMemoryAllocator and FileDataLoader both use malloc() under the hood. Systems without malloc() should use different allocators and data loaders

• BufferDataLoader for example points to a pre-allocated buffer. This would work for users who embed the .pte file data directly in their firmware image.
• The MemoryAllocator base class has a constructor that accepts a pre-allocated pointer and length. Unfortunately there's no way to ask the runtime how much memory is needed for a given .pte file and runtime version, but users could experiment with sizes to find a value that works. Or, if they have a non-malloc()-based dynamic memory source, they could write their own MemoryAllocator subclass.

Uses of clocks and stderr are centralize in the PAL (platform abstraction layer). The default PAL is posix.cpp, which uses the stdlib. But users can build with the (recently-added) CMake flag -DEXECUTORCH_PAL_DEFAULT=minimal to build with minimal.cpp, which does not depend on the standard library. But in that case the users will need to implement any of the et_pal_*() functions that they depend on; see https://pytorch.org/executorch/main/runtime-platform-abstraction-layer.html for more information about the PAL.

[dbort]

Hopefully this gives you what you need. If not, please re-open the issue.

[barakugav]

@dbort I can avoid using these structs and functions, but they still require linking to std as the code is still there.
My problem is mostly around alloc and delete, I can override the PAL. For example the delete symbol is not found here:

"operator delete(void*)", referenced from:
    torch::executor::MemoryAllocator::~MemoryAllocator() in libexecutorch_sys-e52bf4e4c0c4d75e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    torch::executor::util::FileDataLoader::~FileDataLoader() in libexecutorch_sys-e52bf4e4c0c4d75e.rlib[24](file_data_loader.cpp.o)

I would like to use MemoryAllocator with pre allocated buffer, but I cant get it to work

[barakugav]

@dbort any thoughts?

[swolchok]

@dbort I can avoid using these structs and functions, but they still require linking to std as the code is still there. My problem is mostly around alloc and delete, I can override the PAL. For example the delete symbol is not found here:

"operator delete(void*)", referenced from:
    torch::executor::MemoryAllocator::~MemoryAllocator() in libexecutorch_sys-e52bf4e4c0c4d75e.rlib[4](842ff64ee6e2bf22-api_utils.o)
    torch::executor::util::FileDataLoader::~FileDataLoader() in libexecutorch_sys-e52bf4e4c0c4d75e.rlib[24](file_data_loader.cpp.o)

I would like to use MemoryAllocator with pre allocated buffer, but I cant get it to work

I'm surprised that ~MemoryAllocator has an operator delete call. It's empty: https://github.com/pytorch/executorch/blob/main/runtime/core/memory_allocator.h#L163 and AFAICT the class has no members with non-trivial destructors.

Similarly, ~FileDataLoader calls std::free but I don't see why it's calling operator delete:

executorch/extension/data_loader/file_data_loader.cpp

Line 67 in ccd684f

std::free(const_cast<char*>(file_name_));

@barakugav do you have any idea where the operator delete calls might be coming from that I've missed?

[swolchok]

I inspected assembly for file_data_loader.cpp, and sure enough, _ZN5torch8executor4util14FileDataLoaderD0Ev has an operator delete call. The Itanium C++ ABI explains:

• the destructor ending in D0 is the "deleting destructor" (5.1.4.3)
• the deleting destructor of a class T is "A function that, in addition to the actions required of a complete object destructor, calls the appropriate deallocation function (i.e,. operator delete) for T." (1.1)
• deleting destructors are required to be emitted for virtual destructors (2.5.2 and implied in 5.2.5)

So, I think you have to include at least part of the standard library (specifically, operator delete) in order to link C++ executables that use the Itanium C++ ABI (which is what I would expect on ~all major platforms) and contain classes with virtual destructors. That said, I imagine you can probably just write an operator delete(void*) implementation that calls abort() or whatever the appropriate equivalent is in your environment?

Google says this is a known problem for embedding systems and C++; see P1105r1.