Repository navigation
[Mono.Android] Load JNI remapping from binary assets - #13023
Closed
simonrozsival wants to merge 6 commits into
Closed
simonrozsival wants to merge 6 commits into
simonrozsival wants to merge 6 commits into
Conversation
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Incremental invalidation, failed-write recovery, asset validation, and cleanup tracking need correction.
Review effort: Balanced
Findings: 1
Open (4)
What changed in this PR
Replaces JNI-remapping native tables with XAJR binary assets, advancing #12940 on top of #12844 while avoiding native relinking when mappings change.
Changes:
- Adds deterministic asset generation, validation, and managed lookup.
- Updates CoreCLR/NativeAOT startup and per-RID packaging; removes remapping-specific LLVM machinery.
- Expands runtime, packaging, and incremental-build tests and documents XA4331.
| File | Description |
|---|---|
| tests/MSBuildDeviceIntegration/Tests/R8RuntimeRemappingTests.cs | Tests asset packaging and updates without relinking. |
| tests/Mono.Android-Tests/Mono.Android-Tests/Java.Interop/JniRemappingLookupTests.cs | Tests binary lookup and pointer lifetime. |
| src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets | Replaces native remapping generation with asset packaging. |
| src/Xamarin.Android.Build.Tasks/Utilities/JniRemappingNativeCodeGenerator.cs | Removes LLVM table emitter. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/GenerateJniRemappingNativeCodeTests.cs | Removes obsolete emitter tests. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingTest.cs | Checks per-RID AAB assets. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs | Checks asset regeneration and cleanup. |
| src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs | Removes remapping source mode. |
| src/Xamarin.Android.Build.Tasks/Tasks/GenerateJniRemappingNativeCode.cs | Removes native generation task. |
| src/Xamarin.Android.Build.Tasks/Resources/proguard_xamarin.cfg | Preserves Java asset reader. |
| src/Xamarin.Android.Build.Tasks/Resources/proguard_trimmable_nativeaot.cfg | Preserves NativeAOT asset reader. |
| src/Xamarin.Android.Build.Tasks/Resources/NativeAotRuntimeProvider.java | Initializes asset access. |
| src/Xamarin.Android.Build.Tasks/Properties/Resources.resx | Adds XA4331 diagnostic. |
| src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs | Exposes XA4331 resource. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Trimmable.targets | Removes relocated remapping import. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Trimmable.CoreCLR.targets | Collects linked assemblies without rescheduling ProGuard. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.R8JniRemapping.targets | Generates and collects binary assets. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.NativeAOT.targets | Removes late linking and exposes publish metadata. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.NativeAOT.R8.targets | Deletes post-R8 relinking machinery. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.NativeAOT.After.targets | Removes obsolete relinking import. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.After.targets | Imports shared remapping targets. |
| src/native/nativeaot/host/jni-remapping-data-stub.cc | Removes empty native descriptor. |
| src/native/nativeaot/host/host.cc | Loads asset during startup. |
| src/native/nativeaot/host/CMakeLists.txt | Removes descriptor stub source. |
| src/native/common/include/runtime-base/jni-remapping-asset.hh | Adds shared JNI asset loader. |
| src/native/common/include/managed-interface.hh | Adds pointer-and-length handoff. |
| src/native/clr/xamarin-app-stub/application_dso_stub.cc | Removes native remapping tables. |
| src/native/clr/include/xamarin-app.hh | Removes obsolete table declarations. |
| src/native/clr/host/host.cc | Loads CoreCLR remapping asset. |
| src/Mono.Android/Mono.Android.csproj | Includes managed asset reader. |
| src/Mono.Android/Microsoft.Android.Runtime/JniRemappingLookup.cs | Uses validated, pinned asset storage. |
| src/Mono.Android/Microsoft.Android.Runtime/JniRemappingAsset.cs | Implements XAJR validation and lookup. |
| src/Mono.Android/Android.Runtime/JNIEnvInit.cs | Initializes lookup and frees temporary bytes. |
| src/Microsoft.Android.Build.Tasks/Utilities/JniRemapping/JniRemappingXmlReader.cs | Reads existing replacement XML. |
| src/Microsoft.Android.Build.Tasks/Utilities/JniRemapping/JniRemappingAssetWriter.cs | Serializes deterministic XAJR data. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/Tasks/GenerateR8JniRemappingTests.cs | Checks R8 mappings through binary lookup. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/Tasks/GenerateJniRemappingAssetTests.cs | Tests serialization, validation, and lifetime. |
| src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/Microsoft.Android.Build.Tasks.Tests.csproj | Updates test dependencies. |
| src/Microsoft.Android.Build.Tasks/Tasks/GenerateJniRemappingAsset.cs | Adds binary generation task. |
| src/Microsoft.Android.Build.Tasks/Microsoft.Android.Build.Tasks.csproj | Shares runtime parser with build tasks. |
| src/java-runtime/java/net/dot/android/JniRemappingAsset.java | Reads per-RID assets. |
| src/java-runtime/java/mono/android/clr/MonoPackageManager.java | Initializes CoreCLR asset access. |
| Documentation/docs-mobile/TOC.yml | Adds XA4331 navigation. |
| Documentation/docs-mobile/messages/xa4331.md | Documents generation failures. |
| Documentation/docs-mobile/messages/index.md | Indexes XA4331. |
| Documentation/docs-mobile/building-apps/build-properties.md | Documents binary remapping transport. |
Files not reviewed (1)
- src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs: Generated file
simonrozsival
force-pushed
the
simonrozsival-binary-jni-remapping-transport
branch
from
October 6, 2026 19:30
8d1678b to
d0dd43c
Compare
simonrozsival
force-pushed
the
simonrozsival-binary-jni-remapping-transport
branch
from
October 7, 2026 09:10
de2160c to
d9f0873
Compare
simonrozsival
force-pushed
the
simonrozsival-binary-jni-remapping-transport
branch
from
October 7, 2026 09:32
d9f0873 to
b02f5c3
Compare
Replace only the final JNI remapping transport with deterministic XAJR assets. Preserve canonical replacement XML, optional descriptors, reverse types, field mappings, and R8 retention policies. Package shared CoreCLR bytes and post-ILC per-RID NativeAOT assets after R8. Validate and pin private managed bytes before exposing stable JNI pointers; release the temporary native handoff. Retire JNI remapping LLVM tables, object inputs, and map-dependent native relinking without changing other code generation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Materialize the actual first-publish native files in outer FileWrites before IncrementalClean. Keep the ILC object used for RID-specific binary filtering and the already-linked library without re-evaluating or relinking after R8. Adapt the migrated descriptor-distinct field integration test to assert real binary lookup behavior. Both full CoreCLR/NativeAOT R8 runtime cases and critical host packaging/incremental/Clean checks pass on the reconstructed parent. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Publish assets only after successful writes, validate startup bytes even with lookups disabled, and invalidate assets when project mapping inputs are removed. Generate final remapping outputs after R8 but before IncrementalClean records FileWrites. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Fix CS8604 when MergeRemapXml is linked into the modern task tests and warnings are treated as errors. Support filename-only outputs without attempting to create an empty directory, and cover that case in the existing merge task tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Validate native asset bytes outside the lookup type, then initialize lookup state only behind the build-time feature switch. Preserve malformed-data rejection and pinned-pointer lifetime without rooting disabled lookup caches. Refresh only the affected Release APK size entries for the intentional binary asset, Java/native loader and validation code, retaining all existing trimming and size thresholds. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Package the validated XAJR payload as a single read-only data symbol in per-ABI libandroid_runtime_blobs.so. Bound its fixed header and declared size against the retained ELF mapping before handing its pointer to managed lookup. Remove Java staging, full-payload copying and startup table scans; retain full producer validation and bounded UTF-8/NUL checks on visited records. Preserve post-R8 packaging, NativeAOT filtering, forwarding signatures, incremental/Clean and opt-off trimming. Include loader, semantic, allocation and atomic-output tests and refresh only the four affected CoreCLR size references without changing thresholds. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
simonrozsival
force-pushed
the
simonrozsival-binary-jni-remapping-transport
branch
from
October 7, 2026 17:02
b02f5c3 to
e89ae0f
Compare
An error occurred while trying to automatically change base from
simonrozsival-move-r8-remapping-tasks
to
main
October 10, 2026 05:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #12940. This is one binary transport layer above #12844, targeting
simonrozsival-move-r8-remapping-tasks, notmain. Native stack #12850 membership/order remains[12846,12847,12848,12692,12844,13023]. Only this session's transport branch/PR changes; lower branches, PR bases and stack metadata are untouched.Replace canonical replacement XML's final transport: XML -> deterministic, versioned, pointer-free XAJR v1 Android asset -> managed lookup, instead of LLVM tables/native objects. Preserve XML merge precedence/diagnostics, type/reverse-type/method/field mappings, optional descriptors, instance-to-static forwarding and opt-in R8 policies/filtering. CoreCLR shares bytes under each RID's logical asset path; NativeAOT serializes separately per RID after the one final R8 pass, filtering from the actual first-publish ILC object.
A small Java reader and pointer-plus-length native handoff supply the asset. The parser validates a private snapshot independently of lookup state. Only the build-time
RuntimeFeature.JniRemappingcall-site guard initializes/pinsJniRemappingLookup, so disabled lookup types/caches are trimmable. Temporary native bytes are freed throughmonodroid_freeinfinally, including failures. No-map applications still read/validate a valid 64-byte empty asset; malformed or missing bytes fail explicitly. The Java byte-array read and two subsequent copies remain; native-asset/memory-mapped alternatives have only been discussed, not implemented here.Remove only remapping-specific emitters,
.ll/.oinputs, native tables and post-R8 relinking/evaluation hooks. Track actual NativeAOT first-publish inputs beforeIncrementalClean; serialize after R8 but before the clean manifest snapshot. Preserve current NDK/linker routing, Java/crypto bootstrap, remaining LLVM consumers, assembly-store/ELF formats, app DSO and unrelated runtime state. No new public switch or native fallback.Current stack snapshot
Base:
6eeb61813986ea4f8ee1ea8c371c6b7595a505dd(#12844), directly based on policy #126925e47c4b825b3e0a4b11ab0c3b41455cdbde81843. Head:b02f5c3ed80ad8e4b2bdc0e90a7ec586a93127dd.The latest parent update adds the explicit-primary-RID CoreCLR review case and main's Java.Interop test migration. All five transport commits replayed without conflict;
git range-diffreports=for every commit. This ancestry-only push used an explicit lease against freshly verified previous transport headd9f0873c72ce1026b2196ddc56044dc01272f9e3. Backups preserve prior owned heads.Earlier current-main conflict resolutions retain reduced logging initialization ABI, retained/scoped typemap and DGML retirement, shared native logging startup and merged assembly-store runtime/compression state. Narrow JNI asset-reader keep rules are wired into both current typemap templates and the legacy template. No retired NativeAOT ProGuard template, compression LLVM emitter or runtime-state table is reintroduced. The parent's runtime-remapping opt-out from typemap no-minification is unchanged. Final assembly collection uses
_ResolveAssemblieswithout late ProGuard generation.All review/CI fixes remain: atomic asset publication with failed-write retry coverage; validation regardless of the lookup switch; project/import invalidation when the last mapping is removed; FileWrites/Clean ordering; nullable merge-output directory handling; and feature-gated lookup initialization. Size-reference conflicts retained newer main values rather than restoring old snapshots; thresholds remain unchanged.
Latest review-case validation
Build succeeded (33 existing generated-code/duplicate-source warnings, 0 errors). Exactly one requested case ran and passed:
MultiRidUsesOneR8Mapping(CoreCLR,true), no skips. It built both RIDs with an explicit primary RID, verified one R8 pass, zero R8/native-link work on no-op, and removal of generated remapping XML/binary files on Clean. The parent owner independently verified the result. Production transport patches are unchanged; earlier source/runtime results below are retained with their actual source attribution.Earlier source/runtime evidence
Own-source SDK/task/runtime/native-host/test outputs were rebuilt; immutable distributions seeded the private SDK/toolchain, not donor/PoC task or runtime outputs. Concurrency was capped at two MSBuild nodes/two Ninja jobs. Device validation used only private
jni-remap-sync3d712-api35/emulator-5584and isolated ADB port 5041. No physical A16 or shared emulator was used. Named helpers were stopped/removed afterward; useful logs/results remain.Commands used the private SDK first on PATH, absolute DOTNET_ROOT pointing to
bin/Release/dotnet,DOTNET_ROLL_FORWARD=Major, andDOTNET_CLI_USE_MSBUILD_SERVER=0. Size checks also used privately installed CI-pinned apkdiff 0.0.17. Device commands additionally usedADB_SERVER_SOCKET=tcp:localhost:5041,ANDROID_ADB_SERVER_PORT=5041,ANDROID_SERIAL=emulator-5584andADB_TARGET='-s emulator-5584'.make all CONFIGURATION=Release MSBUILD_ARGS='-m:2'on preceding 66fb parent plus transportbin/Release/dotnet/dotnet build src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/Microsoft.Android.Build.Tasks.Tests.csproj -c Release -m:2 -v minimal --no-incremental -p:TreatWarningsAsErrors=truebin/Release/dotnet/dotnet test src/Microsoft.Android.Build.Tasks/Tests/Microsoft.Android.Build.Tasks.Tests/Microsoft.Android.Build.Tasks.Tests.csproj -c Release --no-build -v minimal -p:TreatWarningsAsErrors=true --filter 'FullyQualifiedName~GenerateR8JniRemappingTests|FullyQualifiedName~GenerateJniRemappingAssetTests|FullyQualifiedName~CompressionMetadataTests'./dotnet-local.sh test bin/TestRelease/MSBuildDeviceIntegration/net10.0/MSBuildDeviceIntegration.dll --filter 'FullyQualifiedName~DeployToDevice|FullyQualifiedName~MultiRidUsesOneR8Mapping' --logger 'trx;LogFileName=sync66fb-startup-multirid.trx' --results-directory bin/TestRelease/TransportResults -- NUnit.NumberOfTestWorkers=1./dotnet-local.sh test bin/TestRelease/net10.0/Xamarin.Android.Build.Tests.dll --filter 'FullyQualifiedName~BuildReleaseArm64|FullyQualifiedName~JniRemappingAssetSurvivesIncrementalBuild|FullyQualifiedName~NoChangeBuildKeepsDynamicJniRegistrationDisabled|FullyQualifiedName~CheckR8MetadataFilesExist|FullyQualifiedName~MamJsonToXmlTests|FullyQualifiedName~MergeRemapXmlTests|FullyQualifiedName~R8Obfuscation' --logger 'trx;LogFileName=sync66fb-host.trx' --results-directory bin/TestRelease/TransportResults -- NUnit.NumberOfTestWorkers=2./dotnet-local.sh test bin/TestRelease/MSBuildDeviceIntegration/net10.0/MSBuildDeviceIntegration.dll --filter 'FullyQualifiedName~UnchangedProguardRulesDoNotRerunR8|FullyQualifiedName~R8RuntimeRemappingTests' --logger 'trx;LogFileName=sync0057-targeted-parent.trx' --results-directory bin/TestRelease/TransportResults -- NUnit.NumberOfTestWorkers=1on corrected f330/0057 product before the assembly-store-only main mergeReduced pointer-plus-length initialization offsets/alignment were cross-compiled for arm64/arm32/x64/x86 after logging-field retirement (96-/56-byte structs). That ABI is unchanged in subsequent parent updates; these checks are not 32-bit runtime execution proof.
Live user XML/MAM/native malformed-byte/cached-pointer GC MTP tests passed 19/19 on the preceding 3d712 product, not rerun after the parent-only updates. Exact commands:
Historical first publication: full modern suite 377 passed/5 native-tool-fixture skips, NativeAOT Intune/MAM MTP one passed. Those entire suites were not rerun here. Earlier size snapshots passed all four cases before main's size decreases; current size-only inconclusive outcomes are not claimed as passes.
Runtime coverage is API 35 arm64 with 4-KiB pages; 32-bit runtime, 16-KiB devices and Windows runtime execution were not tested. Fresh CI results are not yet verified. The separate Windows directory-move file-lock failure is not changed or claimed fixed; no manual CI retries or issue mutations were performed.
Pull Request
title and
description
should follow the
commit-messages.mdworkflow documentation, and in particular should include: