-
Notifications
You must be signed in to change notification settings - Fork 15k
Description
Using MCJIT, several projects have run into failures like
lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp:400: void llvm::RuntimeDyldELF::resolveAArch64Relocation(const llvm::SectionEntry&, uint64_t, uint64_t, uint32_t, int64_t): Assertion `isInt<33>(Result) && "overflow check failed for relocation"' failed.
- Impala runs into this with LLVM 5.0.1 (IMPALA-11542).
- Numba and Julia have also run into it on newer versions of LLVM (at least 13): https://discourse.llvm.org/t/llvm-rtdyld-aarch64-abi-relocation-restrictions/74616.
I believe this issue is unaddressed in newer releases. As summarized here, MCJIT with the SectionMemoryManager or TrivialMemoryManager allocates sections in separate memory allocation calls. On systems with lots of memory, this can wind up allocating them in different locations in a way that violates AArch64 ABI restrictions (that text and GOT segments must be within 4GB of each other). 574713c is insufficient to fix the issue (tested it in Impala).
RTDyldMemoryManager provides needsToReserveAllocationSpace/reserveAllocationSpace. One proposal is to reserve all necessary memory in a single allocation, ensuring it's one contiguous block. This puts slightly stricter requirements on contiguous memory, which may not be suitable for memory-constrained environments, but for most use-cases it switches from 2-3 allocations to 1 larger allocation. This approach seems to address the issue in Impala.