KEMBAR78
Detect if calloc calls malloc by apangin · Pull Request #1500 · async-profiler/async-profiler · GitHub
Skip to content

Conversation

@apangin
Copy link
Member

@apangin apangin commented Sep 21, 2025

Description

Use temporary malloc hook to detect if calloc implementation calls malloc under the hood. This is required for proper nativemem profiling.

Related issues

#1466

Motivation and context

Various malloc implementations may or may not do cross-calls. For example, default calloc in musl libc calls malloc internally. Async-profiler handles this case specially to prevent double-counting of native memory alocations. Previously, it simply checked OS::isMusl, but this was inaccurate, since musl libc could be built with different malloc libraries. For example, in Alpaquita Linux, musl is linked with mimalloc, where calloc does not calls malloc.

How has this been tested?

nativemem tests now pass on Alpaquita Linux as well as on vanilla Alpine.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@Baraa-Hasheesh
Copy link
Contributor

Baraa-Hasheesh commented Sep 21, 2025

While thinking about this issue, the solution that made most sense to me was something along the lines of what is implented here

However I have some doubts
It's assumeed that if calloc calls malloc, then the posix_memalign would also call aligned_alloc
I think it's better to check for that separately, as user might change calloc via LD_PRELOAD but keep posix_memalign the same

On a similar note, do we want to support other patterns the user might introduce via an LD_PRELOAD? (For example a user might change malloc to call calloc)

@apangin
Copy link
Member Author

apangin commented Sep 22, 2025

@Baraa-Hasheesh Checking only one type of relation (calloc calling malloc) was a conscious choice. This relation is a distinctive characteristic of musl's own memory allocator. Other popular allocators (glibc's malloc, jemalloc, tcmalloc, mimalloc) do not do cross-calls via PLT.

Checking relation between posix_memalign and aligned_alloc would require writing more code in vain (with no effect for the above allocators). At the same time, it would not be a complete solution for a custom user-provided allocator anyway.

Support for custom allocators that hook arbitrary malloc functions is out of scope of nativemem feature. Doing LD_PRELOAD hacks you've mentioned while running nativemem profiling sounds extraordinary. I think it is unreasonable to build a complicated solution for a theoretical problem that noone have probably seen.

static bool _recursive_malloc = false;

// Test if calloc() implementation calls malloc()
static void* recursive_malloc_hook(size_t size) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if "recursive" is the right naming, as function is not calling itself.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I had doubts about that too, but decided it's still recursive in the sense the malloc algorithm (as the whole) calls to itself via PLT. Do you have a suggestion how to name it better?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to nested

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Nested" sounds good, as we are calling all malloc-family functions malloc here.

@apangin apangin merged commit d97a7d3 into master Sep 23, 2025
43 checks passed
@apangin apangin deleted the recursive-malloc branch September 23, 2025 21:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants