Repository navigation
Closed
Closed
Copy link
Labels
arch-arm64area-Infrastructure-monoin-prThere is an active PR which will close this issue when it is mergedThere is an active PR which will close this issue when it is mergedos-androidos-iosApple iOSApple iOSruntime-monospecific to the Mono runtimespecific to the Mono runtimesize-reductionIssues impacting final app size primary for size sensitive workloadsIssues impacting final app size primary for size sensitive workloads
Milestone
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Mac OS X 10.18 |
| Queue | IPhone |
| Baseline | 47ebcf3831a4a9b43d9abb19c21e4d66752e1d7c |
| Compare | 68511fd27fe4055ce5203742998ba12019dfcbd4 |
| Diff | Diff |
| Configs | CompilationMode:tiered, HybridGlobalization:true, iOSLlvmBuild:true, iOSStripSymbols:true, RunKind:ios_scenarios, RuntimeType:mono |
Regressions in SOD - iOS HelloWorld Mono .app Size llvm nosymbols
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 13.38 MB | 14.16 MB | 1.06 | 0.00 | True | ||||
| 16.50 KB | 17.00 KB | 1.03 | 0.00 | True | ||||
| 13.00 count | 17.00 count | 1.31 | 0.00 | True | ||||
| 1.19 MB | 1.29 MB | 1.09 | 0.00 | True | ||||
| 1.21 MB | 1.40 MB | 1.16 | 0.00 | True | ||||
| 3.00 count | 5.00 count | 1.67 | 0.00 | True | ||||
| 10.29 MB | 10.71 MB | 1.04 | 0.00 | True | ||||
| 13.00 count | 17.00 count | 1.31 | 0.00 | True | ||||
| 5.14 KB | 5.44 KB | 1.06 | 0.00 | True | ||||
| 10.29 MB | 10.71 MB | 1.04 | 0.00 | True | ||||
| 1.88 MB | 2.04 MB | 1.08 | 0.00 | True | ||||
| 13.38 MB | 14.16 MB | 1.06 | 0.00 | True | ||||
| 4.00 count | 6.00 count | 1.50 | 0.00 | True | ||||
| 1.05 MB | 1.13 MB | 1.07 | 0.00 | True | ||||
| 6.62 KB | 6.91 KB | 1.04 | 0.00 | True | ||||
| 839.63 KB | 847.38 KB | 1.01 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifactsor build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0 - CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0 - Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release - Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SOD - iOS HelloWorld Mono .app Size llvm nosymbols*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'SOD - iOS HelloWorld Mono .app Size llvm nosymbols*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packagesWindows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SOD - iOS HelloWorld Mono .app Size llvm nosymbols*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'SOD - iOS HelloWorld Mono .app Size llvm nosymbols*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packagesDetails
SOD - iOS HelloWorld Mono .app Size llvm nosymbols
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Mac OS X 10.18 |
| Queue | IPhone |
| Baseline | 47ebcf3831a4a9b43d9abb19c21e4d66752e1d7c |
| Compare | 68511fd27fe4055ce5203742998ba12019dfcbd4 |
| Diff | Diff |
| Configs | CompilationMode:tiered, HybridGlobalization:true, iOSLlvmBuild:true, iOSStripSymbols:true, RunKind:ios_scenarios, RuntimeType:mono |
Regressions in SOD - iOS HelloWorld Mono Zip Size llvm nosymbols
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 4.51 MB | 4.77 MB | 1.06 | 0.00 | True | ||||
| 4.51 MB | 4.77 MB | 1.06 | 0.00 | True | ||||
| 4.51 MB | 4.77 MB | 1.06 | 0.00 | True | ||||
| 4.51 MB | 4.77 MB | 1.06 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifactsor build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0 - CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0 - Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release - Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SOD - iOS HelloWorld Mono Zip Size llvm nosymbols*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'SOD - iOS HelloWorld Mono Zip Size llvm nosymbols*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packagesWindows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SOD - iOS HelloWorld Mono Zip Size llvm nosymbols*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'SOD - iOS HelloWorld Mono Zip Size llvm nosymbols*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packagesDetails
SOD - iOS HelloWorld Mono Zip Size llvm nosymbols
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
arch-arm64area-Infrastructure-monoin-prThere is an active PR which will close this issue when it is mergedThere is an active PR which will close this issue when it is mergedos-androidos-iosApple iOSApple iOSruntime-monospecific to the Mono runtimespecific to the Mono runtimesize-reductionIssues impacting final app size primary for size sensitive workloadsIssues impacting final app size primary for size sensitive workloads






























Activity
[-][Perf] Windows/x64: 20 Regressions on 8/9/2024 8:00:01 AM[/-][+][iOS][size][Perf] iOS size regression on 8/9/2024 8:00:01 AM[/+]dotnet-policy-service commentedon Aug 12, 2024
Tagging subscribers to this area: @directhex, @matouskozak
See info in area-owners.md if you want to be subscribed.
dotnet-policy-service commentedon Aug 12, 2024
Tagging subscribers to 'os-ios': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.
dotnet-policy-service commentedon Aug 12, 2024
Tagging subscribers to 'size-reduction': @eerhardt, @SamMonoRT, @marek-safar
See info in area-owners.md if you want to be subscribed.
15 remaining items
noahfalk commentedon Aug 16, 2024
Thanks for investigating!
Can the separate type be an inner class or will DAM.All pull that in too? I'm fine to do it as a top-level class if necessary, but inner class seemed a little nicer if the only point of the class is to get the trimmer behavior we want.
These docs should give a good idea what scenarios you lose out on if you disable Meter:
Metrics are most prevalent in web-services, but they can also be useful for ad-hoc performance investigations in all types of apps. In terms of defaults, my initial thought is that turning on/off EventSource seems like a better lever to use than Meter. If you are planning to turn Meter off by default and leave EventSource on I'd be curious to better understand the rationale.
According to sizoscope it showed me the regression as 250.4 rather than 300, but aside from that yeah, the new dependencies from metrics seemed right.
noahfalk commentedon Aug 16, 2024
Yeah, size optimization for instrumentation has never been a goal we've prioritized. The feedback that I see has been mostly about adding new capabilities, integration, standardization and performance at runtime. In many cases I suspect the perf at runtime has direct tradeoffs against code size as the fastest/lowest alloc implementations tend to be heavier on custom structs and generic instantiations of code to process them. I'm happy to chat more about it any time.
MichalStrehovsky commentedon Aug 19, 2024
Sizoscope doesn't have exact accounting of things - it pretty much always underreports because some bytes of the output are too difficult to attribute but those bytes are generally related to things that are visible in the UI.
Thank you for checking!
MichalStrehovsky commentedon Aug 19, 2024
This would need to be a separate class -
.Allincludes nested types for better or worse (in this case for better, because event source reflects at nested types too).runtime/src/libraries/System.Private.CoreLib/src/System/Diagnostics/Tracing/EventSource.cs
Lines 3141 to 3159 in 0fca85f
noahfalk commentedon Aug 19, 2024
Haha, I didn't even realize inner class reflection was in EventSource's bag of tricks. I'll have to tuck that tid-bit away somewhere. Thanks for the info! I'll put together another change that moves the code into a separate top-level internal class soon, hopefully tomorrow.
Fix trimming for DiagnosticSource
Fix trimming for DiagnosticSource (dotnet#106680)
Fix trimming for DiagnosticSource
[release/9.0] Fix trimming for DiagnosticSource (#106842)
Fix trimming for DiagnosticSource (dotnet#106680)