Skip to content

[#inc-517] Add basic crashing test with asan (HW Asan, GWP-ASan)聽#962

Closed

Description

@kahest
### 馃洃 Blocked by 
- [ ] https://github.com/getsentry/sentry-java/pull/3189
- [ ] https://github.com/getsentry/sentry-native/pull/944

Activity

  1. JoshuaMoelans commented on Jan 31, 2025

    @JoshuaMoelans
    Member

    Testing Address Sanitizers

    This comment acts as a "what-we-already-tried" benchmark to work from, which can guide our next steps in resolving this issue. During the investigation of #974 I had a look into the following 3 ASAN types:

    ASan

    (Android Docs)
    This first Address Sanitizer gave me actual results when trying to build an Android app with native C memory bug detection. For future reference, this might make the most sense to add as a first "basic crashing test".

    To run ASan on an emulated Pixel 9 Pro (running Android 14.0 Default Android System Image, so we have Root access) I added the following to the sample app (found here):
    In /sample/cmakelists.txt

    target_compile_options(ndk-sample PRIVATE -fsanitize=address -fno-omit-frame-pointer)
    target_link_options(ndk-sample PRIVATE -fsanitize=address)
    
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsanitize=address -fno-omit-frame-pointer")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address -fno-omit-frame-pointer")
    set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -fsanitize=address")

    And these to /sample/build.gradle.kts and sentry-native/ndk/lib/build.gradle.kts

    externalNativeBuild {
        cmake {
            arguments.add(0, "-DANDROID_STL=c++_shared")
            arguments.add(0, "-DSENTRY_NATIVE_SRC=$sentryNativeSrc")
            arguments.add("-DCMAKE_VERBOSE_MAKEFILE=ON")
            abiFilters.addAll(listOf("x86_64", "arm64-v8a"))
            abiFilters.forEach { abi ->
                println("ABI=$abi")
                if (abi == "arm64-v8a" || abi == "x86_64") {
                    arguments.add(0, "-DANDROID_SANITIZE=address")
                }
            }
        }
    }

    After which I needed to copy [your ndk folder]\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\9.0.9\lib\linux\libclang_rt.asan-aarch64-android.so to [your app]/src/main/jniLibs/arm64-v8a/ (as instructed here)

    When running the following snippet, we get ASan output:

    char *ptr_f = (char *)malloc(10);
    free(ptr_f);
    *ptr_f = 'A';
    ASan output
    2024-10-22 13:56:47.897  3284-3284  libc                    io.sentry.ndk.sample                 A  Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3284 (ntry.ndk.sample), pid 3284 (ntry.ndk.sample)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A  Cmdline: io.sentry.ndk.sample
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A  pid: 3284, tid: 3284, name: ntry.ndk.sample  >>> io.sentry.ndk.sample <<<
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A  Abort message: '=================================================================
                                                                                                        ==3284==ERROR: AddressSanitizer: heap-use-after-free on address 0x003a8ad50010 at pc 0x00715b1c8524 bp 0x007fc1c688d0 sp 0x007fc1c688c8
                                                                                                        WRITE of size 1 at 0x003a8ad50010 thread T0
                                                                                                            #0 0x715b1c8520  (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so+0x3520) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38)
                                                                                                            #1 0x71d6546c30  (/apex/com.android.art/lib64/libart.so+0x346c30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #2 0x71d6530480  (/apex/com.android.art/lib64/libart.so+0x330480) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #3 0x71d671a244  (/apex/com.android.art/lib64/libart.so+0x51a244) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #4 0x71d66a3adc  (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #5 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #6 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #7 0x71d671ad8c  (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #8 0x71d66a3adc  (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #9 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #10 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #11 0x71d671ad8c  (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #12 0x71d66a4618  (/apex/com.android.art/lib64/libart.so+0x4a4618) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #13 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #14 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #15 0x71d6564ff8  (/apex/com.android.art/lib64/libart.so+0x364ff8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #16 0x71d6546d68  (/apex/com.android.art/lib64/libart.so+0x346d68) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #17 0x71d67d0e30  (/apex/com.android.art/lib64/libart.so+0x5d0e30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #18 0x71d67d0dd4  (/apex/com.android.art/lib64/libart.so+0x5d0dd4) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #19 0x71d6530480  (/apex/com.android.art/lib64/libart.so+0x330480) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #20 0x71d671a244  (/apex/com.android.art/lib64/libart.so+0x51a244) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #21 0x71d66a3adc  (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #22 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #23 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #24 0x71d671ad8c  (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #25 0x71d66a4618  (/apex/com.android.art/lib64/libart.so+0x4a4618) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #26 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #27 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #28 0x71d671ad8c  (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #29 0x71d66a3adc  (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #30 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #31 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A      #32 0x71d671ad8c  (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #33 0x71d66a376c  (/apex/com.android.art/lib64/libart.so+0x4a376c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #34 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #35 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #36 0x71d671ad8c  (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #37 0x71d66a3adc  (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #38 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #39 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #40 0x71d671ad8c  (/apex/com.android.art/lib64/libart.so+0x51ad8c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #41 0x71d66a3adc  (/apex/com.android.art/lib64/libart.so+0x4a3adc) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #42 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #43 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #44 0x71d6564ff8  (/apex/com.android.art/lib64/libart.so+0x364ff8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #45 0x71d6546d68  (/apex/com.android.art/lib64/libart.so+0x346d68) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #46 0x71d6530480  (/apex/com.android.art/lib64/libart.so+0x330480) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #47 0x71d658ae2c  (/apex/com.android.art/lib64/libart.so+0x38ae2c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #48 0x71d658a7e8  (/apex/com.android.art/lib64/libart.so+0x38a7e8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #49 0x71d6546c30  (/apex/com.android.art/lib64/libart.so+0x346c30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #50 0x71d65301a4  (/apex/com.android.art/lib64/libart.so+0x3301a4) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #51 0x71d671a0c0  (/apex/com.android.art/lib64/libart.so+0x51a0c0) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #52 0x71d66a376c  (/apex/com.android.art/lib64/libart.so+0x4a376c) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #53 0x71d65493d8  (/apex/com.android.art/lib64/libart.so+0x3493d8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #54 0x71d6565700  (/apex/com.android.art/lib64/libart.so+0x365700) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #55 0x71d6564ff8  (/apex/com.android.art/lib64/libart.so+0x364ff8) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #56 0x71d6546d68  (/apex/com.android.art/lib64/libart.so+0x346d68) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                        
                                                                                                        0x003a8ad50010 is located 0 bytes inside of 10-byte region [0x003a8ad50010,0x003a8ad5001a)
                                                                                                        freed by thread T0 here:
                                                                                                            #0 0x714f16d54c  (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so+0xe454c) (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
                                                                                                            #1 0x715b1c84cc  (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so+0x34cc) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38)
                                                                                                            #2 0x71d6546c30  (/apex/com.android.art/lib64/libart.so+0x346c30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #3 0x74a07bfe78  (<unknown module>)
                                                                                                            #4 0x7fc1c68f1c  ([stack]+0x7f7f1c)
                                                                                                        
                                                                                                        previously allocated by thread T0 here:
                                                                                                            #0 0x714f16d668  (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so+0xe4668) (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A      #1 0x715b1c84c0  (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so+0x34c0) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38)
                                                                                                            #2 0x71d6546c30  (/apex/com.android.art/lib64/libart.so+0x346c30) (BuildId: d22b3b69a6db691fdd84720465c7a214)
                                                                                                            #3 0x74a07bfe78  (<unknown module>)
                                                                                                            #4 0x7fc1c68f1c  ([stack]+0x7f7f1c)
                                                                                                        
                                                                                                        SUMMARY: AddressSanitizer: heap-use-after-free (/data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so+0x3520) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38) 
                                                                                                        Shadow bytes around the buggy address:
                                                                                                          0x003a8ad4fd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                                                                                                          0x003a8ad4fe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                                                                                                          0x003a8ad4fe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                                                                                                          0x003a8ad4ff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                                                                                                          0x003a8ad4ff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                                                                                                        =>0x003a8ad50000: fa fa[fd]fd fa fa fa fa fa fa fa fa fa fa fa fa
                                                                                                          0x003a8ad50080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
                                                                                                          0x003a8ad50100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
                                                                                                          0x003a8ad50180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
                                                                                                          0x003a8ad50200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
                                                                                                          0x003a8ad50280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
                                                                                                        Shadow byte legend (one shadow byte represents 8 application bytes):
                                                                                                          Addressable:           00
                                                                                                          Partially addressable: 01 02 03 04 05 06 07 
                                                                                                          Heap left redzone:       fa
                                                                                                          Freed heap region:       fd
                                                                                                          Stack left redzone:      f1
                                                                                                          Stack mid redzone:       f2
                                                                                                          Stack right redzone:     f3
                                                                                                          Stack after return:      f5
                                                                                                          Stack use after scope:   f8
                                                                                                          Global redzone:          f9
                                                                                                          Global init order:       f6
                                                                                                          Poisoned by user:        f7
                                                                                                          Container overflow:      fc
                                                                                                          Array cookie:            ac
                                                                                                          Intra object redzone:    bb
                                                                                                          ASan internal:           fe
                                                                                                          Left alloca redzone:     ca
                                                                                                          Right alloca redzone:    cb'
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A        #01 pc 000000000006c5b4  /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A        #02 pc 000000000006b034  /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A        #03 pc 00000000000e88fc  /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A        #04 pc 00000000000eb7fc  /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A        #05 pc 00000000000ec81c  /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libclang_rt.asan-aarch64-android.so (__asan_report_store1+52) (BuildId: 2c287b347827d5aec5b5ef36619b30b1df0e20ac)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A        #06 pc 0000000000003520  /data/app/~~aKZshbc8F5M5JpsqBdf-bA==/io.sentry.ndk.sample-tq5NmovDEYAkK0IQ4HblkQ==/lib/arm64/libndk-sample.so (Java_io_sentry_ndk_sample_NdkSample_spawnBreadcrumbProcess+468) (BuildId: 8e7e1f15d92eb5af50726b904e27f3ba40ca8f38)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A        #12 pc 00000000000008a8  <anonymous:74a07bf000> (io.sentry.ndk.sample.MainActivity.lambda$onCreate$3+0)
    2024-10-22 13:56:48.020  3315-3315  DEBUG                   crash_dump64                         A        #17 pc 00000000000006d8  <anonymous:74a07bf000> (io.sentry.ndk.sample.MainActivity$$ExternalSyntheticLambda3.onClick+0)
    

    GWP-ASan

    (Android Docs)
    This method relies on intercepting randomly chosen subsets of heap allocations to catch memory corruption bugs. This method isn't reliable enough for testing, since you'd need a large enough number of runs to find statistically significant results. It is meant more as a lightweight in-production ASan that, given enough users, will find heap memory safety bugs.

    Enabling this for an app can be done by adding this to the AndroidManifest.xml file:

    <application android:gwpAsanMode="always">
      ...
    </application>

    HW-ASan

    (Android Docs)
    Hardware-Assisted ASan can be useful, but needs device support. In my testing (only using emulated devices) this did not work (probably because it needs the actual hardware). In theory it catches a superset of ASan (the same bugs + stack use after return, plus it has lower RAM overhead)

  2. removed their assignment
    on Mar 25, 2025
  3. added
    Tasktasks like repository maintenance, dependencies, e.t.c.
    on Apr 8, 2025
  4. added
    Teststesting, CI
    and removed
    Tasktasks like repository maintenance, dependencies, e.t.c.
    on Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions