KEMBAR78
JFR to OTLP converter by fandreuz · Pull Request #1336 · async-profiler/async-profiler · GitHub
Skip to content

JFR to OTLP converter #1336

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 63 commits into from
Jul 4, 2025
Merged

JFR to OTLP converter #1336

merged 63 commits into from
Jul 4, 2025

Conversation

@fandreuz
Copy link
Contributor

@fandreuz fandreuz commented Jun 11, 2025 •
edited
Loading

Description

In this PR I add a JFR to OTLP converter.

Motivation and context

This piece of work was scheduled as part of the ongoing effort to integrate Async-Profiler with OTLP. Having an OTLP converter will allow our users to work with historical data.

How has this been tested?

Local testing: out_otlp.txt

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@fandreuz fandreuz marked this pull request as ready for review June 12, 2025 16:03
@fandreuz fandreuz requested a review from apangin June 12, 2025 16:03
apangin
apangin reviewed Jun 13, 2025
View reviewed changes
@fandreuz fandreuz marked this pull request as draft June 13, 2025 09:31
@fandreuz
Copy link
Contributor Author

fandreuz commented Jun 30, 2025 •
edited
Loading

Before this is merged, I'd like to tackle the problem which @Baraa-Hasheesh mentioned during the last demo. Moving to draft again.

I thought about this, I think we could have a flag allEvents which forces the JFR collector to collect all events. Then, each event from a given type will be part of a separate profile. This could be implemented in another PR. The current implementation handles only one type per call.

@fandreuz fandreuz marked this pull request as ready for review June 30, 2025 10:10
Baraa-Hasheesh
Baraa-Hasheesh reviewed Jul 2, 2025
View reviewed changes
fandreuz
fandreuz commented Jul 2, 2025
View reviewed changes
fandreuz added 2 commits July 2, 2025 09:03
@fandreuz
comment by bara
21dfb9b 
Signed-off-by: Francesco Andreuzzi <andreuzzi.francesco@gmail.com>
@fandreuz
Merge branch 'master' into jfr-to-otlp-converter
dbc5ecd
@fandreuz fandreuz mentioned this pull request Jul 2, 2025
Merged
apangin
apangin reviewed Jul 2, 2025
View reviewed changes
src/converter/one/convert/JfrToOtlp.java Outdated

proto.field(
SAMPLE_attribute_indices,
attributesPool.index(new KeyValue("thread.name", getThreadName(event.tid))));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My gut feeling tells me this can be a bottleneck due to lots of garbage created on the hot path.
I'm not asking you to rewrite it, though: it will be an interesting exercise to profile this later on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure will do, do you know any low overhead sampling profiler for Java that does not suffer from the Safepoint bias problem?

apangin
apangin reviewed Jul 4, 2025
View reviewed changes
apangin
apangin reviewed Jul 4, 2025
View reviewed changes
src/converter/one/convert/JfrToOtlp.java Outdated

public JfrToOtlp(JfrReader jfr, Arguments args) {
super(jfr, args);
ticksPerNanosecond = jfr.ticksPerSec / 1_000_000_000.0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, move this to OtlpEventToSampleVisitor. This should not be cached across chunks, since chunks may have different ticksPerSec

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ouch, I thought this was unique per JFR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

93679fc

src/converter/one/convert/JfrToOtlp.java Outdated
proto.field(PROFILE_duration_nanos, jfr.chunkDurationNanos());
}

public void dump(OutputStream out) throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method is lost somewhere in the middle between private methods.
Move it next to other public methods, either after constructor or before convert.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

93679fc

src/converter/one/convert/JfrToOtlp.java Outdated
}

// Write location table
for (Line l : linePool.keys()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You took my suggestion about shortening variable names too literally :)
line would be perfect here. l is not good as identifier - it resembles 1 or I.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

93679fc

@apangin apangin merged commit 2b8dfff into async-profiler:master Jul 4, 2025
18 checks passed
@apangin
Copy link
Member

apangin commented Jul 4, 2025

Good job, thanks!
Merged

visheshruparelia pushed a commit to visheshruparelia/async-profiler that referenced this pull request Jul 9, 2025
@fandreuz @visheshruparelia
JFR to OTLP converter (async-profiler#1336)
1326362
visheshruparelia added a commit to visheshruparelia/async-profiler that referenced this pull request Jul 9, 2025
@visheshruparelia
refactor to use hasDwarfTable()
08ee9c1 
Disable JFR OldObjectSample event in jfrsync mode (async-profiler#1350)

Fix invalid alignment in mallocTracer and zero-init buf in getTotalCpuTime (async-profiler#1351)

C++ linting via clang-tidy (async-profiler#1338)

Allow cstack=vmx for native applications (async-profiler#1354)

Correctly unwind stack for malloc events in VM stack walking mode (async-profiler#1357)

Simplify location handling in OTLP (async-profiler#1361)

Suppress javac warnings when compiling tests

Do not include excess files in test.jar

Fix nonjava test failure on Alpine

Auto-generated clang-tidy review comments (async-profiler#1360)

JFR to OTLP converter (async-profiler#1336)

Cancel redundant in-progress GHA runs (async-profiler#1363)

Ensure that only files under `src/` are checked in `cpp-lint-diff` (async-profiler#1365)

Publish clang-tidy comments only for non-draft PRs (async-profiler#1367)

Give tests unique suffix names (async-profiler#1371)

Test OTLP output format (async-profiler#1331)

Disable JFR OldObjectSample event in jfrsync mode (async-profiler#1350)

Fix invalid alignment in mallocTracer and zero-init buf in getTotalCpuTime (async-profiler#1351)

C++ linting via clang-tidy (async-profiler#1338)

Allow cstack=vmx for native applications (async-profiler#1354)

Correctly unwind stack for malloc events in VM stack walking mode (async-profiler#1357)

Simplify location handling in OTLP (async-profiler#1361)

Suppress javac warnings when compiling tests

Do not include excess files in test.jar

Fix nonjava test failure on Alpine

Auto-generated clang-tidy review comments (async-profiler#1360)

JFR to OTLP converter (async-profiler#1336)

Cancel redundant in-progress GHA runs (async-profiler#1363)

Ensure that only files under `src/` are checked in `cpp-lint-diff` (async-profiler#1365)

Publish clang-tidy comments only for non-draft PRs (async-profiler#1367)

Give tests unique suffix names (async-profiler#1371)

Test OTLP output format (async-profiler#1331)
visheshruparelia added a commit to visheshruparelia/async-profiler that referenced this pull request Jul 9, 2025
@visheshruparelia
refactor to use hasDwarfTable()
4ae1986 
Disable JFR OldObjectSample event in jfrsync mode (async-profiler#1350)

Fix invalid alignment in mallocTracer and zero-init buf in getTotalCpuTime (async-profiler#1351)

C++ linting via clang-tidy (async-profiler#1338)

Allow cstack=vmx for native applications (async-profiler#1354)

Correctly unwind stack for malloc events in VM stack walking mode (async-profiler#1357)

Simplify location handling in OTLP (async-profiler#1361)

Suppress javac warnings when compiling tests

Do not include excess files in test.jar

Fix nonjava test failure on Alpine

Auto-generated clang-tidy review comments (async-profiler#1360)

JFR to OTLP converter (async-profiler#1336)

Cancel redundant in-progress GHA runs (async-profiler#1363)

Ensure that only files under `src/` are checked in `cpp-lint-diff` (async-profiler#1365)

Publish clang-tidy comments only for non-draft PRs (async-profiler#1367)

Give tests unique suffix names (async-profiler#1371)

Test OTLP output format (async-profiler#1331)

Disable JFR OldObjectSample event in jfrsync mode (async-profiler#1350)

Fix invalid alignment in mallocTracer and zero-init buf in getTotalCpuTime (async-profiler#1351)

C++ linting via clang-tidy (async-profiler#1338)

Allow cstack=vmx for native applications (async-profiler#1354)

Correctly unwind stack for malloc events in VM stack walking mode (async-profiler#1357)

Simplify location handling in OTLP (async-profiler#1361)

Suppress javac warnings when compiling tests

Do not include excess files in test.jar

Fix nonjava test failure on Alpine

Auto-generated clang-tidy review comments (async-profiler#1360)

JFR to OTLP converter (async-profiler#1336)

Cancel redundant in-progress GHA runs (async-profiler#1363)

Ensure that only files under `src/` are checked in `cpp-lint-diff` (async-profiler#1365)

Publish clang-tidy comments only for non-draft PRs (async-profiler#1367)

Give tests unique suffix names (async-profiler#1371)

Test OTLP output format (async-profiler#1331)
visheshruparelia added a commit to visheshruparelia/async-profiler that referenced this pull request Jul 9, 2025
@visheshruparelia
refactor to use hasDwarfTable()
aa9aa5c 
Disable JFR OldObjectSample event in jfrsync mode (async-profiler#1350)

Fix invalid alignment in mallocTracer and zero-init buf in getTotalCpuTime (async-profiler#1351)

C++ linting via clang-tidy (async-profiler#1338)

Allow cstack=vmx for native applications (async-profiler#1354)

Correctly unwind stack for malloc events in VM stack walking mode (async-profiler#1357)

Simplify location handling in OTLP (async-profiler#1361)

Suppress javac warnings when compiling tests

Do not include excess files in test.jar

Fix nonjava test failure on Alpine

Auto-generated clang-tidy review comments (async-profiler#1360)

JFR to OTLP converter (async-profiler#1336)

Cancel redundant in-progress GHA runs (async-profiler#1363)

Ensure that only files under `src/` are checked in `cpp-lint-diff` (async-profiler#1365)

Publish clang-tidy comments only for non-draft PRs (async-profiler#1367)

Give tests unique suffix names (async-profiler#1371)

Test OTLP output format (async-profiler#1331)

Disable JFR OldObjectSample event in jfrsync mode (async-profiler#1350)

Fix invalid alignment in mallocTracer and zero-init buf in getTotalCpuTime (async-profiler#1351)

C++ linting via clang-tidy (async-profiler#1338)

Allow cstack=vmx for native applications (async-profiler#1354)

Correctly unwind stack for malloc events in VM stack walking mode (async-profiler#1357)

Simplify location handling in OTLP (async-profiler#1361)

Suppress javac warnings when compiling tests

Do not include excess files in test.jar

Fix nonjava test failure on Alpine

Auto-generated clang-tidy review comments (async-profiler#1360)

JFR to OTLP converter (async-profiler#1336)

Cancel redundant in-progress GHA runs (async-profiler#1363)

Ensure that only files under `src/` are checked in `cpp-lint-diff` (async-profiler#1365)

Publish clang-tidy comments only for non-draft PRs (async-profiler#1367)

Give tests unique suffix names (async-profiler#1371)

Test OTLP output format (async-profiler#1331)
visheshruparelia added a commit to visheshruparelia/async-profiler that referenced this pull request Jul 9, 2025
@visheshruparelia
refactor to use hasDwarfTable()
3c4c589 
Disable JFR OldObjectSample event in jfrsync mode (async-profiler#1350)

Fix invalid alignment in mallocTracer and zero-init buf in getTotalCpuTime (async-profiler#1351)

C++ linting via clang-tidy (async-profiler#1338)

Allow cstack=vmx for native applications (async-profiler#1354)

Correctly unwind stack for malloc events in VM stack walking mode (async-profiler#1357)

Simplify location handling in OTLP (async-profiler#1361)

Suppress javac warnings when compiling tests

Do not include excess files in test.jar

Fix nonjava test failure on Alpine

Auto-generated clang-tidy review comments (async-profiler#1360)

JFR to OTLP converter (async-profiler#1336)

Cancel redundant in-progress GHA runs (async-profiler#1363)

Ensure that only files under `src/` are checked in `cpp-lint-diff` (async-profiler#1365)

Publish clang-tidy comments only for non-draft PRs (async-profiler#1367)

Give tests unique suffix names (async-profiler#1371)

Test OTLP output format (async-profiler#1331)

Disable JFR OldObjectSample event in jfrsync mode (async-profiler#1350)

Fix invalid alignment in mallocTracer and zero-init buf in getTotalCpuTime (async-profiler#1351)

C++ linting via clang-tidy (async-profiler#1338)

Allow cstack=vmx for native applications (async-profiler#1354)

Correctly unwind stack for malloc events in VM stack walking mode (async-profiler#1357)

Simplify location handling in OTLP (async-profiler#1361)

Suppress javac warnings when compiling tests

Do not include excess files in test.jar

Fix nonjava test failure on Alpine

Auto-generated clang-tidy review comments (async-profiler#1360)

JFR to OTLP converter (async-profiler#1336)

Cancel redundant in-progress GHA runs (async-profiler#1363)

Ensure that only files under `src/` are checked in `cpp-lint-diff` (async-profiler#1365)

Publish clang-tidy comments only for non-draft PRs (async-profiler#1367)

Give tests unique suffix names (async-profiler#1371)

Test OTLP output format (async-profiler#1331)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@apangin apangin apangin left review comments

+1 more reviewer

@Baraa-Hasheesh Baraa-Hasheesh Baraa-Hasheesh left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@fandreuz @apangin @Baraa-Hasheesh