[coreclr] Initial support of R2R builds #10007

ivanpovazan · 2025-04-04T14:23:29Z

Description

ReadyToRun deployments are enabled when:

<PublishReadyToRun>true</PublishReadyToRun> is added to the project file
App is published via: dotnet publish

Once enabled, the SDK injects necessary build tasks to perform AOT compilation of assemblies after their trimming and creation of R2R images during app publish.

Problem

Currently, Android app build runs two assembly postprocessing build tasks (after R2R image generation): RewriteMarshalMethods and RemoveRegisterAttribute that are modifying assemblies with Mono.Cecil. However, altering R2R assemblies is not supported with Mono.Cecil and build fails via:

System.NotSupportedException: Writing mixed-mode assemblies is not supported
         at Mono.Cecil.ModuleWriter.Write(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters)
         at Mono.Cecil.ModuleWriter.WriteModule(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters)
         at Mono.Cecil.ModuleDefinition.Write(Stream stream, WriterParameters parameters)
         at Mono.Cecil.ModuleDefinition.Write(WriterParameters parameters)
         at Mono.Cecil.ModuleDefinition.Write()
         at Mono.Cecil.AssemblyDefinition.Write()
         at Xamarin.Android.Tasks.RemoveRegisterAttribute.RunTask() in /Users/runner/work/1/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAtt
      ribute.cs:line 37
         at Microsoft.Android.Build.Tasks.AndroidTask.Execute() in /Users/runner/work/1/s/xamarin-android/external/xamarin-android-tools/src/Microsoft.Android.Build.
      BaseTasks/AndroidTask.cs:line 25

Changes

To enable R2R builds, this PR:

disables running RewriteMarshalMethods and RemoveRegisterAttribute build tasks in ReadyToRun deployments
adds new unit tests for R2R builds
reenables unit tests for regular CoreCLR builds

Future work

Investigate running R2R image generation build task after RewriteMarshalMethods and RemoveRegisterAttribute so that R2R builds could also benefit from the performance improvements these build tasks bring.

grendello · 2025-04-08T16:33:41Z

I think I'd rather wait with merging this PR until marshal methods can be used again. Maybe it's a simple matter of ordering? R2R should be called, like Mono AOT, after we're done modifying the assemblies.

jonathanpeppers · 2025-04-15T16:49:12Z

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs

+
+			proj.SetProperty ("RuntimeIdentifier", rid);
+			proj.SetProperty ("UseMonoRuntime", "false"); 	// Enables CoreCLR
+			proj.SetProperty ("_IsPublishing", "true"); 	// Make "dotnet build" act as "dotnet publish"


Can you update this line for when $(PublishReadyToRun) is true:

android/src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.DefaultProperties.targets

Line 34 in 747041d

<_IsPublishing Condition=" '$(_IsPublishing)' == '' and '$(_AndroidRuntime)' == 'NativeAOT' ">true</_IsPublishing>

I would expect dotnet build -c Release -p:PublishReadyToRun=true to work by default on Android.

We could make this as a temporary workaround for R2R as well.
However, on the long run we shouldn't force dotnet build -c Release -p:PublishReadyToRun=true to do publishing as that would not match the behavior on other platforms.
On desktop and on iOS, triggering NativeAOT builds only happens with dotnet publish , dotnet build uses "regular" default runtimes/builds -> CoreCLR desktop, Mono iOS

Just for reference, MAUI on Windows uses PublishReadyToRun=true by default on Release builds since .NET 7:

https://github.com/dotnet/maui/blob/5f986ddbaf885eac7117063dfeca5f703004f14d/src/Workload/Microsoft.Maui.Sdk/Sdk/Microsoft.Maui.Sdk.Before.targets#L7

If R2R is what makes the startup performance reasonable, I think we have to use it by default on Android as well? So, dotnet build -c Release on Android would use R2R?

That only applies if you are publishing your app through Visual Studio, or manually through dotnet publish.
dotnet build will not / should not run publish targets when PublishReadyToRun=true.

On a regular console app if you add PublishReadyToRun=true to the project and run dotnet build it will not run R2R AOT compiler. But if you run dotnet publish it will.

So to come back to my previous comment, we should not force _IsPublishing=true for NativeAOT nor R2R just so we would make
dotnet build -p:PublishAOT=true or
dotnet build -p:PublishReadyToRun=true to run/act like dotnet publish, it is against the behavior we have on other platforms (e.g., on macios we solved it with yet another property: https://github.com/dotnet/macios/blob/7cef45367161ca26d3a68e2e096639194ed4df1f/dotnet/targets/Xamarin.Shared.Sdk.props#L39)

The difference is people can actually ship their app with dotnet build -c Release for Android today (and they are). There is currently no requirement to use dotnet publish on Android.

As per our offline discussion, we will address this in the follow-up work of fully enabling R2R builds

jonathanpeppers · 2025-04-15T16:50:13Z

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs

+			var b = CreateApkBuilder ();
+			Assert.IsTrue (b.Build (proj), "Build should have succeeded.");


In this test, is there a way to assert the .dll files inside the .apk has a ReadyToRun image inside? Is Mono.Cecil able to detect them? We already have some tests that use Cecil to assert build output.

We could maybe just inspect whether assemblies from obj/Release/netX.0-android/android-X/R2R/ end up in the .apk? I am saying this because the SPC.dll we ship with CoreCLR already includes R2R image in it - even without enabling R2R builds at app build time.
I will look into it.

It might be enough to compare the MVID of the obj/Release/netX.0-android/android-X/R2R/*.dll file and the one inside the .apk?

Inspecting the .apk it self doesn't seem to be doable. As in CoreCLR builds we use assembly store and all assemblies get merged and converted into a single .so file that ends up in the .apk.
Do we have a way of reversing that process and extracting the .dll it self?

If not, we could potentially add a custom MSBuild target to the test project that simply prints out ResolvedUserAssemblies which go into GeneratePackageManagerJava, if those paths include obj/Release/netX.0-android/android-X/R2R/*.dll assemblies the test passes?

For the test, I think you could use:

android/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/LinkerTests.cs

Lines 281 to 290 in b54ec05

var apk = Path.Combine (Root, b.ProjectDirectory, proj.OutputPath, $"{proj.PackageName}-Signed.apk");

FileAssert.Exists (apk);

var helper = new ArchiveAssemblyHelper (apk, useAssemblyStore);

foreach (string abi in proj.GetRuntimeIdentifiersAsAbis ()) {

Assert.IsTrue (helper.Exists ($"assemblies/{abi}/{assemblyName}.dll"), $"{assemblyName}.dll should exist in apk!");

using var stream = helper.ReadEntry ($"assemblies/{assemblyName}.dll");

stream.Position = 0;

using var assembly = AssemblyDefinition.ReadAssembly (stream);

Then assembly you could either check the MVID or some flag that says there is a R2R image.

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs

jonathanpeppers · 2025-04-15T16:52:34Z

I think I'd rather wait with merging this PR until marshal methods can be used again. Maybe it's a simple matter of ordering? R2R should be called, like Mono AOT, after we're done modifying the assemblies.

@grendello I'm going to look into this, but I don't think we have to block this PR on it. We will need to split up parts of <GenerateJavaStubs/> to run inside the publishing targets, in between the trimmer and either R2R or ILC. NativeAOT already has this same issue.

…sts/BuildTest2.cs PR feedback Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>

dellis1972 · 2025-04-17T07:48:57Z

@ivanpovazan This probably isn't the place but any thoughts on the comments about Ready 2 run here https://docs.monogame.net/articles/getting_started/packaging_games.html?tabs=windows#readytorun-r2r.
It would be interesting to see if that is something that could be improved.

ivanpovazan · 2025-04-17T09:34:17Z

@dellis1972, yes definitely. The recommendations/concerns you linked are related to steady-state performance and we plan on investigating what is the right tuning of R2R, PGO and tiered JIT, to get best of both worlds - fast startup and steady-state performance on mobile apps. For example, current limitation of ReadyToRun is lacking the support of partial AOT mode - precompiling only startup code hot paths, which is something we are looking into.

While this PR enables R2R builds in the SDK and preliminary measurements are showing promising results, I don't think we should make it a default just yet.

…mage in it

ivanpovazan · 2025-04-17T17:37:53Z

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs

+			using var stream = helper.ReadEntry ($"assemblies/{assemblyName}.dll");
+			stream.Position = 0;
+			using var peReader = new System.Reflection.PortableExecutable.PEReader (stream);
+			Assert.IsTrue (peReader.PEHeaders.CorHeader.ManagedNativeHeaderDirectory.Size > 0, 


@jonathanpeppers I used the recommendation from: dotnet/runtime#38440 (comment) to verify that the app main assembly that gets packed into the .apk has R2R image in it.

jonpryor · 2025-04-21T18:48:40Z

Draft commit message:

Context: https://learn.microsoft.com/en-us/dotnet/core/deploying/ready-to-run
Context: https://github.com/dotnet/runtime/blob/c83f92ad7aa7afa152b833a8e8f5ffe36a3287d1/docs/design/coreclr/botr/readytorun-overview.md
Context: https://github.com/dotnet/runtime/blob/main/docs/design/coreclr/botr/readytorun-format.md
Context: https://github.com/xamarin/monodroid/commit/f22cc208374637fea59bef7f849bff29247a3e5a


ReadyToRun (R2R) is a form of ahead-of-time compilation supported by
CoreCLR.It works by adding setting the `ManagedNativeHeader`
CLR header entrypoint to refer to a `READYTORUN_HEADER` section.

R2R is enabled by using `dotnet publish` alongside
`$(PublishReadyToRun)`=true:

	dotnet new console
	dotnet publish -p:PublishReadyToRun=true

You can see some of the changes that R2R produces by using
[`dumpbin /CLRHEADER`][0].  Consider the original assembly:

	> dumpbin /clrheader .\obj\Release\net9.0\win-x64\*.dll
	…
	  clr Header:
	         …
	         6000001 entry point token
	               …
	               0 [       0] RVA [size] of ManagedNativeHeader Directory

vs. the R2R assembly:

	> dumpbin /clrheader .\obj\Release\net9.0\win-x64\R2R\*.dll
	…
	  clr Header:
	         …
	         6000001 entry point token
	               …
	            1548 [      94] RVA [size] of ManagedNativeHeader Directory

The `ManagedNativeHeader` entry point having a non-zero RVA indicates
that the assembly contains native machine code, possibly R2R data.

Enable R2R usage with .NET for Android and CoreCLR, when
`$(UseMonoRuntime)`=false.  Additionally, `$(RuntimeIdentifier)` must
be set, via `-r RID`:

	dotnet new android
	dotnet publish -p:PublishReadyToRun=true -p:UseMonoRuntime=false -r android-arm64

The resulting packaged assemblies will contain R2R data.

## Problem

Android app build runs two assembly post-processing build tasks which
modify assemblies using Mono.Cecil:
`<RewriteMarshalMethods/>` (86260ed36d) and
`<RemoveRegisterAttribute/>` (xamarin/monodroid@f22cc208).
Unfortunately, if this happens *after* R2R image generation, Cecil
will throw a NotSupportedException, as Cecil does not support
modifying assemblies which contain native data:

	System.NotSupportedException: Writing mixed-mode assemblies is not supported
	   at Mono.Cecil.ModuleWriter.Write(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters)
	   at Mono.Cecil.ModuleWriter.WriteModule(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters)
	   at Mono.Cecil.ModuleDefinition.Write(Stream stream, WriterParameters parameters)
	   at Mono.Cecil.ModuleDefinition.Write(WriterParameters parameters)
	   at Mono.Cecil.ModuleDefinition.Write()
	   at Mono.Cecil.AssemblyDefinition.Write()
	   at Xamarin.Android.Tasks.RemoveRegisterAttribute.RunTask() in /Users/runner/work/1/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAttribute.cs:line 37
	   at Microsoft.Android.Build.Tasks.AndroidTask.Execute() in /Users/runner/work/1/s/xamarin-android/external/xamarin-android-tools/src/Microsoft.Android.Build.BaseTasks/AndroidTask.cs:line 25

"Fix" this by *disabling* use of the `<RewriteMarshalMethods/>` 
and `<RemoveRegisterAttribute/>` tasks when using R2R.

## TODO

Investigate running R2R image generation build task after
`<RewriteMarshalMethods/>` and `<RemoveRegisterAttribute/>` so that
R2R builds could also benefit from the performance improvements these
build tasks bring.

Investigate running R2R image generation as part of the
`$(RuntimeIdentifiers)` inner build, so that `-r RID` is not required.

[0]: https://learn.microsoft.com/en-us/cpp/build/reference/clrheader?view=msvc-170

ivanpovazan · 2025-04-22T08:53:37Z

I opened #10062 for tracking the follow-up work

ivanpovazan added 2 commits April 4, 2025 16:15

Enable R2R builds

9a3e300

Add R2R build tests

772de8e

ivanpovazan changed the title ~~WIP: [coreclr] Enable R2R builds~~ [coreclr] Enable R2R builds Apr 7, 2025

ivanpovazan marked this pull request as ready for review April 7, 2025 08:33

ivanpovazan requested review from dellis1972 and jonathanpeppers as code owners April 7, 2025 08:33

ivanpovazan requested a review from grendello April 8, 2025 10:53

jonathanpeppers reviewed Apr 15, 2025

View reviewed changes

Update src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Te…

de98b73

…sts/BuildTest2.cs PR feedback Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>

ivanpovazan changed the title ~~[coreclr] Enable R2R builds~~ [coreclr] Initial support of R2R builds Apr 17, 2025

Use better verification method that app assembly in the apk has R2R i…

619186b

…mage in it

ivanpovazan commented Apr 17, 2025

View reviewed changes

jonathanpeppers approved these changes Apr 18, 2025

View reviewed changes

jonpryor merged commit 30295f8 into main Apr 21, 2025
57 of 59 checks passed

jonpryor deleted the dev/ivanpovazan/enable-r2r branch April 21, 2025 19:44

github-actions bot locked and limited conversation to collaborators May 22, 2025

		var b = CreateApkBuilder ();
		Assert.IsTrue (b.Build (proj), "Build should have succeeded.");

	var apk = Path.Combine (Root, b.ProjectDirectory, proj.OutputPath, $"{proj.PackageName}-Signed.apk");
	FileAssert.Exists (apk);
	var helper = new ArchiveAssemblyHelper (apk, useAssemblyStore);
	foreach (string abi in proj.GetRuntimeIdentifiersAsAbis ()) {
	Assert.IsTrue (helper.Exists ($"assemblies/{abi}/{assemblyName}.dll"), $"{assemblyName}.dll should exist in apk!");

	using var stream = helper.ReadEntry ($"assemblies/{assemblyName}.dll");
	stream.Position = 0;

	using var assembly = AssemblyDefinition.ReadAssembly (stream);

[coreclr] Initial support of R2R builds #10007

[coreclr] Initial support of R2R builds #10007

Uh oh!

Conversation

ivanpovazan commented Apr 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Problem

Changes

Future work

Uh oh!

grendello commented Apr 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jonathanpeppers commented Apr 15, 2025

Uh oh!

dellis1972 commented Apr 17, 2025

Uh oh!

ivanpovazan commented Apr 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jonpryor commented Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ivanpovazan commented Apr 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ivanpovazan commented Apr 4, 2025 •

edited

Loading

jonpryor commented Apr 21, 2025 •

edited

Loading