You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[#inc-517] Add basic crashing test with asan (HW Asan, GWP-ASan)聽#962
This comment acts as a "what-we-already-tried" benchmark to work from, which can guide our next steps in resolving this issue. During the investigation of #974 I had a look into the following 3 ASAN types:
ASan
(Android Docs)
This first Address Sanitizer gave me actual results when trying to build an Android app with native C memory bug detection. For future reference, this might make the most sense to add as a first "basic crashing test".
To run ASan on an emulated Pixel 9 Pro (running Android 14.0 Default Android System Image, so we have Root access) I added the following to the sample app (found here):
In /sample/cmakelists.txt
And these to /sample/build.gradle.kts and sentry-native/ndk/lib/build.gradle.kts
externalNativeBuild {
cmake {
arguments.add(0, "-DANDROID_STL=c++_shared")
arguments.add(0, "-DSENTRY_NATIVE_SRC=$sentryNativeSrc")
arguments.add("-DCMAKE_VERBOSE_MAKEFILE=ON")
abiFilters.addAll(listOf("x86_64", "arm64-v8a"))
abiFilters.forEach { abi ->println("ABI=$abi")
if (abi =="arm64-v8a"|| abi =="x86_64") {
arguments.add(0, "-DANDROID_SANITIZE=address")
}
}
}
}
After which I needed to copy [your ndk folder]\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\9.0.9\lib\linux\libclang_rt.asan-aarch64-android.so to [your app]/src/main/jniLibs/arm64-v8a/ (as instructed here)
When running the following snippet, we get ASan output:
2024-10-22 13:56:47.897 3284-3284 libc io.sentry.ndk.sample A Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3284 (ntry.ndk.sample), pid 3284 (ntry.ndk.sample)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A Cmdline: io.sentry.ndk.sample
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A pid: 3284, tid: 3284, name: ntry.ndk.sample >>> io.sentry.ndk.sample <<<
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A Abort message: '=================================================================
==3284==ERROR: AddressSanitizer: heap-use-after-free on address 0x003a8ad50010 at pc 0x00715b1c8524 bp 0x007fc1c688d0 sp 0x007fc1c688c8
WRITE of size 1 at 0x003a8ad50010 thread T0
#0 0x715b1c8520 (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so+0x3520) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38)
#1 0x71d6546c30 (/apex/com.android.art/lib64/libart.so+0x346c30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#2 0x71d6530480 (/apex/com.android.art/lib64/libart.so+0x330480) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#3 0x71d671a244 (/apex/com.android.art/lib64/libart.so+0x51a244) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#4 0x71d66a3adc (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#5 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#6 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#7 0x71d671ad8c (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#8 0x71d66a3adc (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#9 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#10 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#11 0x71d671ad8c (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#12 0x71d66a4618 (/apex/com.android.art/lib64/libart.so+0x4a4618) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#13 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#14 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#15 0x71d6564ff8 (/apex/com.android.art/lib64/libart.so+0x364ff8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#16 0x71d6546d68 (/apex/com.android.art/lib64/libart.so+0x346d68) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#17 0x71d67d0e30 (/apex/com.android.art/lib64/libart.so+0x5d0e30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#18 0x71d67d0dd4 (/apex/com.android.art/lib64/libart.so+0x5d0dd4) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#19 0x71d6530480 (/apex/com.android.art/lib64/libart.so+0x330480) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#20 0x71d671a244 (/apex/com.android.art/lib64/libart.so+0x51a244) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#21 0x71d66a3adc (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#22 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#23 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#24 0x71d671ad8c (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#25 0x71d66a4618 (/apex/com.android.art/lib64/libart.so+0x4a4618) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#26 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#27 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#28 0x71d671ad8c (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#29 0x71d66a3adc (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#30 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#31 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #32 0x71d671ad8c (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#33 0x71d66a376c (/apex/com.android.art/lib64/libart.so+0x4a376c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#34 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#35 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#36 0x71d671ad8c (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#37 0x71d66a3adc (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#38 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#39 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#40 0x71d671ad8c (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#41 0x71d66a3adc (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#42 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#43 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#44 0x71d6564ff8 (/apex/com.android.art/lib64/libart.so+0x364ff8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#45 0x71d6546d68 (/apex/com.android.art/lib64/libart.so+0x346d68) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#46 0x71d6530480 (/apex/com.android.art/lib64/libart.so+0x330480) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#47 0x71d658ae2c (/apex/com.android.art/lib64/libart.so+0x38ae2c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#48 0x71d658a7e8 (/apex/com.android.art/lib64/libart.so+0x38a7e8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#49 0x71d6546c30 (/apex/com.android.art/lib64/libart.so+0x346c30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#50 0x71d65301a4 (/apex/com.android.art/lib64/libart.so+0x3301a4) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#51 0x71d671a0c0 (/apex/com.android.art/lib64/libart.so+0x51a0c0) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#52 0x71d66a376c (/apex/com.android.art/lib64/libart.so+0x4a376c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#53 0x71d65493d8 (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#54 0x71d6565700 (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#55 0x71d6564ff8 (/apex/com.android.art/lib64/libart.so+0x364ff8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#56 0x71d6546d68 (/apex/com.android.art/lib64/libart.so+0x346d68) (BuildId: d22b3b69a6db691fdd84720465c7a214)
0x003a8ad50010 is located 0 bytes inside of 10-byte region [0x003a8ad50010,0x003a8ad5001a)
freed by thread T0 here:
#0 0x714f16d54c (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so+0xe454c) (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
#1 0x715b1c84cc (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so+0x34cc) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38)
#2 0x71d6546c30 (/apex/com.android.art/lib64/libart.so+0x346c30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#3 0x74a07bfe78 (<unknown module>)
#4 0x7fc1c68f1c ([stack]+0x7f7f1c)
previously allocated by thread T0 here:
#0 0x714f16d668 (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so+0xe4668) (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #1 0x715b1c84c0 (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so+0x34c0) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38)
#2 0x71d6546c30 (/apex/com.android.art/lib64/libart.so+0x346c30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
#3 0x74a07bfe78 (<unknown module>)
#4 0x7fc1c68f1c ([stack]+0x7f7f1c)
SUMMARY: AddressSanitizer: heap-use-after-free (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so+0x3520) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38)
Shadow bytes around the buggy address:
0x003a8ad4fd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x003a8ad4fe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x003a8ad4fe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x003a8ad4ff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x003a8ad4ff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x003a8ad50000: fa fa[fd]fd fa fa fa fa fa fa fa fa fa fa fa fa
0x003a8ad50080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x003a8ad50100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x003a8ad50180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x003a8ad50200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x003a8ad50280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb'
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #01 pc 000000000006c5b4 /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #02 pc 000000000006b034 /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #03 pc 00000000000e88fc /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #04 pc 00000000000eb7fc /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #05 pc 00000000000ec81c /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (__asan_report_store1+52) (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #06 pc 0000000000003520 /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so (Java_io_sentry_ndk_sample_NdkSample_spawnBreadcrumbProcess+468) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #12 pc 00000000000008a8 <anonymous:74a07bf000> (io.sentry.ndk.sample.MainActivity.lambda$onCreate$3+0)
2024-10-22 13:56:48.020 3315-3315 DEBUG crash_dump64 A #17 pc 00000000000006d8 <anonymous:74a07bf000> (io.sentry.ndk.sample.MainActivity$$ExternalSyntheticLambda3.onClick+0)
GWP-ASan
(Android Docs)
This method relies on intercepting randomly chosen subsets of heap allocations to catch memory corruption bugs. This method isn't reliable enough for testing, since you'd need a large enough number of runs to find statistically significant results. It is meant more as a lightweight in-production ASan that, given enough users, will find heap memory safety bugs.
Enabling this for an app can be done by adding this to the AndroidManifest.xml file:
(Android Docs)
Hardware-Assisted ASan can be useful, but needs device support. In my testing (only using emulated devices) this did not work (probably because it needs the actual hardware). In theory it catches a superset of ASan (the same bugs + stack use after return, plus it has lower RAM overhead)
Activity
JoshuaMoelans commentedon Jan 31, 2025
Testing Address Sanitizers
This comment acts as a "what-we-already-tried" benchmark to work from, which can guide our next steps in resolving this issue. During the investigation of #974 I had a look into the following 3 ASAN types:
ASan
(Android Docs)
This first Address Sanitizer gave me actual results when trying to build an Android app with native C memory bug detection. For future reference, this might make the most sense to add as a first "basic crashing test".
To run ASan on an emulated Pixel 9 Pro (running Android 14.0 Default Android System Image, so we have Root access) I added the following to the sample app (found here):
In
/sample/cmakelists.txttarget_compile_options(ndk-sample PRIVATE -fsanitize=address -fno-omit-frame-pointer) target_link_options(ndk-sample PRIVATE -fsanitize=address) set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsanitize=address -fno-omit-frame-pointer") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address -fno-omit-frame-pointer") set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -fsanitize=address")And these to
/sample/build.gradle.ktsandsentry-native/ndk/lib/build.gradle.ktsexternalNativeBuild { cmake { arguments.add(0, "-DANDROID_STL=c++_shared") arguments.add(0, "-DSENTRY_NATIVE_SRC=$sentryNativeSrc") arguments.add("-DCMAKE_VERBOSE_MAKEFILE=ON") abiFilters.addAll(listOf("x86_64", "arm64-v8a")) abiFilters.forEach { abi -> println("ABI=$abi") if (abi == "arm64-v8a" || abi == "x86_64") { arguments.add(0, "-DANDROID_SANITIZE=address") } } } }After which I needed to copy
[your ndk folder]\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\9.0.9\lib\linux\libclang_rt.asan-aarch64-android.soto[your app]/src/main/jniLibs/arm64-v8a/(as instructed here)When running the following snippet, we get ASan output:
ASan output
GWP-ASan
(Android Docs)
This method relies on intercepting randomly chosen subsets of heap allocations to catch memory corruption bugs. This method isn't reliable enough for testing, since you'd need a large enough number of runs to find statistically significant results. It is meant more as a lightweight in-production ASan that, given enough users, will find heap memory safety bugs.
Enabling this for an app can be done by adding this to the
AndroidManifest.xmlfile:HW-ASan
(Android Docs)
Hardware-Assisted ASan can be useful, but needs device support. In my testing (only using emulated devices) this did not work (probably because it needs the actual hardware). In theory it catches a superset of ASan (the same bugs + stack use after return, plus it has lower RAM overhead)