Support for OTLP Profile signals #1188

fandreuz · 2025-03-21T14:31:20Z

Changes

New output mode (otlp) to dump data in OpenTelemetry format
New Java API to dump Async-Profiler data in OpenTelemetry format
protobuf.cpp: Allow specifying the maximum bit count for the varint encoding the length of a nested message

Motivation

OpenTelemetry recently added support for Profile signals. The end goal of the initiative is to provide a standard, vendor-agnostic format for profiling data. Adopting this protocol as one of our supported output formats will help our users integrating Async-Profiler in their workflow easily.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Co-authored-by: Francesco Andreuzzi <fandreuz@amazon.com>

src/profiler.cpp

src/index.h

src/profiler.cpp

apangin · 2025-06-10T12:43:07Z

src/profiler.cpp

+        otlp_buffer.field(Sample::locations_start_index, frames_seen);
+        otlp_buffer.field(Sample::locations_length, sampleInfo.num_frames);
+
+        protobuf_mark_t sample_value_mark = otlp_buffer.startMessage(Sample::value);


There are many tiny messages whose length can be encoded in 1 byte.
Do you want to provide a version of startMessage (or add an optional argument) to specify maximum length?

Yes, the idea is to have an optional argument eventually. Do you want this in this PR?

Yeah, that would be a more valuable optimization than reusing a buffer :)

Check 84d0ce1 and e52f0be

src/api/one/profiler/AsyncProfiler.java

src/index.h

src/profiler.cpp

apangin · 2025-06-10T14:56:15Z

src/profiler.cpp

+        otlp_buffer.field(Sample::locations_start_index, frames_seen);
+        otlp_buffer.field(Sample::locations_length, sampleInfo.num_frames);
+
+        protobuf_mark_t sample_value_mark = otlp_buffer.startMessage(Sample::value);


Yeah, that would be a more valuable optimization than reusing a buffer :)

src/profiler.cpp

src/profiler.h

This reverts commit be5f413.

src/api/one/profiler/AsyncProfiler.java

apangin · 2025-06-10T23:26:37Z

src/protobuf.cpp

+void ProtoBuffer::commitMessage(const protobuf_mark_t& mark) {
+    size_t actual_len = _offset - (mark.message_start + mark.expected_len_byte_count);
+    size_t actual_len_byte_count = varIntSize(actual_len);
+    if (actual_len_byte_count > mark.expected_len_byte_count) {


This should be an assertion. If reserved space was smaller than required, treat it as error.
Otherwise we may end up moving 100MB blocks without knowing there was a computation mistake.

I don't agree with this. We cannot estimate the size of the message without low level (and error prone) computations at runtime. And any such estimation would be fragile and hard to change later. Any fixed size we decide now may be not enough for long profiling sessions. But either case, I don't think it's a "computation mistake".

Are you implying we should have an exact estimate of the size of the message, before we start writing it to the buffer?

OK, let's call it "underestimation" rather than "computation mistake". I'm saying that startMessage should accept the maximum size of a message instead of an estimation. 5 bytes (which can encode up to 32 GiB messages) is sufficient for all use cases. So, let's use 5 bytes default for the top-level messages and provide an appropriate upper bound for inner ones. Exact estimate is not required.

apangin · 2025-06-11T00:18:54Z

src/protobuf.h

+struct protobuf_mark_t {
+    size_t message_start;
+    size_t expected_len_byte_count;
+};


It's not a good practice to return a structure larger than machine word from a function.
64-bit field for a value that needs at most 3 is too much anyway.
It would be natural (in system programming world) to fit protobuf_mark_t in a machine word or in u64 and pass it by value: 3 bits for len_bytes and remaining 61 bits for message_start. You may use either bit-field struct or just u64 and do manipulations by yourself.

Check 17084fe

src/protobuf.h

src/profiler.cpp

apangin · 2025-06-11T18:24:21Z

src/protobuf.h

-    protobuf_mark_t startMessage(protobuf_index_t index);
-    void commitMessage(protobuf_mark_t mark);
+    protobuf_mark_t startMessage(protobuf_index_t index, size_t max_len_byte_count = NESTED_FIELD_BYTE_COUNT);
+    void commitMessage(const protobuf_mark_t& mark);


It does not need to be a reference, right?

Right, e5e8d33

apangin · 2025-06-11T18:26:06Z

src/arguments.cpp

                _output = OUTPUT_TEXT;
                _dump_flat = value == NULL ? INT_MAX : atoi(value);

+            CASE("otlp")


Do you mind updating all relevant docs and examples with new option?

Sure, let me know if I forgot something: a90b347

apangin · 2025-06-11T18:27:29Z

src/profiler.cpp

 #include "frameName.h"
 #include "os.h"
+#include "otlp.h"
+#include "protobuf.h"


I guess protobuf.h is redundant here, since otlp.h already includes it.

apangin · 2025-06-11T18:29:00Z

src/protobuf.cpp

+
+    for (size_t i = 0; i < max_len_byte_count - 1; ++i) {
+        size_t idx = message_start + i;
+        _data[idx] = (unsigned char) (0x80 | (actual_len & 0x7f));


& 0x7f is redundant

apangin · 2025-06-12T00:14:04Z

I already merged PR, but can I ask you to update its description with background, motivation, summary of changes, etc., so we could refer it in the future. Thanks.

fandreuz · 2025-06-12T09:25:03Z

I already merged PR, but can I ask you to update its description with background, motivation, summary of changes, etc., so we could refer it in the future. Thanks.

Done, let me know if you'd like to have any changes or additions.

apangin · 2025-06-12T11:05:09Z

Looks good, thanks.

fandreuz and others added 16 commits March 20, 2025 10:32

Fix typo (async-profiler#1183)

d518251

Co-authored-by: Francesco Andreuzzi <fandreuz@amazon.com>

fix merge=false compilation error

964ebad

enable protobuf compilation via make

6f49bcd

implement otlp dump (JSON for now)

bcfea59

ap args for otlp

8d23bea

some more code comments

9b0fc48

more comments

53b692d

scope uuid

e7bf867

serialize to binary

b717f55

dump to file args. comments

784252d

todo comment

bddce21

delta

a326f5a

Merge branch 'master' into otel-hackaton

0fb114b

remove compiled proto stuff

f90fe58

nn

7e83108

comment

bb34323

fandreuz commented Mar 26, 2025

View reviewed changes

src/profiler.cpp Outdated Show resolved Hide resolved

fandreuz added the enhancement label Apr 23, 2025

fandreuz added 9 commits May 7, 2025 13:38

Merge branch 'master' into otel-hackaton

4a19452

nope

cc6eada

move away buffer

8a26c15

more stuff

271ba88

fix cmpl

a77b50a

more stuff

52bcd04

protected

fa263e9

Merge branch 'buffer-things' into otel-hackaton

5fa6e5b

wip

152f152

fandreuz mentioned this pull request May 8, 2025

Move out Buffer from flightRecorder.cpp #1290

Closed

fandreuz added 2 commits May 8, 2025 19:39

fix stuff

5167875

trigger

15c8831

multiple counters

5ac9ac6

apangin reviewed Jun 10, 2025

View reviewed changes

fandreuz added 3 commits June 10, 2025 13:31

review comments

4595172

nn

be5f413

cc

267d13b

apangin reviewed Jun 10, 2025

View reviewed changes

fandreuz added 6 commits June 10, 2025 15:24

Revert "nn"

3f10cd0

This reverts commit be5f413.

review comments

975bd6f

add to register natives

a552fde

comment

20c84a7

allow setting expected size

84d0ce1

set some len manually

e52f0be

apangin reviewed Jun 11, 2025

View reviewed changes

fandreuz added 5 commits June 11, 2025 08:39

1 bit

2d8648a

bigger

2c6bd72

64 bit for mark

17084fe

size_t

cdd72f8

max not expected

21d8211

apangin reviewed Jun 11, 2025

View reviewed changes

fandreuz and others added 6 commits June 11, 2025 20:45

no reference

6b69090

review comments

e5e8d33

update docs

a90b347

prettier

b491a51

Formatting

1c7568b

Formatting

cc4044e

apangin merged commit b3f5842 into async-profiler:master Jun 12, 2025
32 of 34 checks passed

roy-soumadipta pushed a commit to roy-soumadipta/async-profiler that referenced this pull request Jun 20, 2025

Support for OTLP Profile signals (async-profiler#1188)

6a7aac3

Support for OTLP Profile signals #1188

Support for OTLP Profile signals #1188

Uh oh!

Conversation

fandreuz commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Motivation

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fandreuz Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fandreuz Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apangin Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

apangin commented Jun 12, 2025

Uh oh!

fandreuz commented Jun 12, 2025

Uh oh!

apangin commented Jun 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

fandreuz commented Mar 21, 2025 •

edited

Loading

fandreuz Jun 10, 2025 •

edited

Loading

fandreuz Jun 11, 2025 •

edited

Loading

apangin Jun 11, 2025 •

edited

Loading