Repository navigation
Seg fault in Linq.Expression tests #105706
Description
Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=759698
Build error leg or test failing: System.Linq.Expressions.Tests.WorkItemExecution
Pull request: #105636
Error Message
Fill the error message using step by step known issues guidance.
{
"ErrorMessage": ["SIGSEGV Illegal memory access. Deref invalid pointer, overrunning buffer", "System.Linq.Expressions.Interpreter.FuncCallInstruction"],
"ErrorPattern": "",
"BuildRetry": false,
"ExcludeConsoleLog": false
}Log: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-105636-merge-95523dd4fead49c19e/System.Linq.Expressions.Tests/1/console.673b37a5.log?helixlogtype=result
Dump: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-105636-merge-95523dd4fead49c19e/System.Linq.Expressions.Tests/1/coredump.20.dmp?helixlogtype=result
Relevant portion of crash analysis, some symbols missing cc @hoyosjs
Thread Id: 0x22
Child SP IP Call Site
0x7294dbb1f260 0x72d5778fdef9 0x72d5778fdef9
0x7294dbb1f310 0x72d576e924bb System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.FuncCallInstruction`2[[System.__Canon, System.Private.CoreLib],[System.Int32, System.Private.CoreLib]]..ctor(System.Reflection.MethodInfo)
0x7294dbb1f3a0 0x72d5f4cc3be4 0x72d5f4cc3be4
0x7294dbb1f3c0 0x72d5f4b01155 libcoreclr.so!?? at ??:0:0
0x7294dbb1f400 0x72d5f4bab122 libcoreclr.so!?? at ??:0:0
0x7294dbb1f7a0 0x72d5779304a1 System.Private.CoreLib.dll!System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(System.Object, System.Span`1<System.Object>, System.Reflection.BindingFlags)
0x7294dbb1f820 0x72d57791a176 System.Private.CoreLib.dll!System.Reflection.MethodBaseInvoker.InvokeWithOneArg(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
0x7294dbb1f8c0 0x72d5777fba77 System.Private.CoreLib.dll!System.Reflection.RuntimeConstructorInfo.Invoke(System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
0x7294dbb1f920 0x72d576af0676 System.Private.CoreLib.dll!System.RuntimeType.CreateInstanceImpl(System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
0x7294dbb1f9f0 0x72d576acaac7 System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.CallInstruction.SlowCreate(System.Reflection.MethodInfo, System.Reflection.ParameterInfo[])
0x7294dbb1faf0 0x72d576ac67c2 System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.CallInstruction.FastCreate(System.Reflection.MethodInfo, System.Reflection.ParameterInfo[])
0x7294dbb1fc20 0x72d576ac60a0 System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.CallInstruction.Create(System.Reflection.MethodInfo, System.Reflection.ParameterInfo[])
0x7294dbb1fda0 0x72d576ac5495 System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.InstructionList.EmitCall(System.Reflection.MethodInfo, System.Reflection.ParameterInfo[])
0x7294dbb1fde0 0x72d576abf14d System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.LightCompiler.CompileMethodCallExpression(System.Linq.Expressions.Expression, System.Reflection.MethodInfo, System.Linq.Expressions.IArgumentProvider)
0x7294dbb1ff50 0x72d577210910 System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.LightCompiler.CompileMethodCallExpression(System.Linq.Expressions.Expression)
0x7294dbb1ff90 0x72d576abd924 System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.LightCompiler.CompileNoLabelPush(System.Linq.Expressions.Expression)
0x7294dbb201c0 0x72d576abc619 System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.LightCompiler.Compile(System.Linq.Expressions.Expression)
0x7294dbb201f0 0x72d576abc349 System.Linq.Expressions.dll!System.Linq.Expressions.Interpreter.LightCompiler.CompileTop(System.Linq.Expressions.LambdaExpression)
0x7294dbb20320 0x72d576e86be9 System.Linq.Expressions.dll!System.Linq.Expressions.LambdaExpression.Compile(Boolean)
0x7294dbb20370 0x72d578fef633 System.Linq.Expressions.Tests.dll!System.Linq.Expressions.Tests.CallTests.Call_NoParameters(System.Linq.Expressions.Expression, System.Reflection.MethodInfo, System.Object, Boolean)
0x7294dbb20400 0x72d577aab5b2 System.Private.CoreLib.dll!DynamicClass.InvokeStub_CallTests.Call_NoParameters(System.Object, System.Span`1<System.Object>)
Could be reflection, or codegen. cc @steveharter @AndyAyersMS in case they see anything.
Known issue validation
Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=759698
Error message validated: [SIGSEGV Illegal memory access. Deref invalid pointer, overrunning buffer System.Linq.Expressions.Interpreter.FuncCallInstruction]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/30/2024 4:08:57 PM UTC
Report
Summary
| 24-Hour Hit Count | 7-Day Hit Count | 1-Month Count |
|---|---|---|
| 4 | 20 | 37 |
Activity
dotnet-policy-service commentedon Jul 30, 2024
Tagging subscribers to this area: @cston
See info in area-owners.md if you want to be subscribed.
cston commentedon Aug 2, 2024
See also #105704.
AndyAyersMS commentedon Aug 2, 2024
Let me take a look; this seems to be happing frequently.
AndyAyersMS commentedon Aug 2, 2024
Based on the windows x64 crash in https://dev.azure.com/dnceng-public/public/_build/results?buildId=764230&view=ms.vss-test-web.build-test-results-tab&runId=19436920&resultId=218614&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab
There is a delegate that blows up the process when invoked. The method invoking the delegate is minopts.
Here the methodPtr is an invalid address. The methodPtrAux field is an indirection cell for
Locally (using CI assets) I was able to get 5 crashes in 200 runs, so I may be able to catch this live in the debugger.
Aside from the AV there were also crashes like
Based on the above my guess is that this is an issue in the runtime with stub management?
cc @mangod9
mangod9 commentedon Aug 2, 2024
Don't believe there have been any recent changes I am aware of which might affect this. We can take a look though. Also adding @janvorli @VSadov if it rings a bell?
AndyAyersMS commentedon Aug 2, 2024
Can't quite figure out how to script this under the debugger, because some of the tests intentionally divide by zero.
steveharter commentedon Aug 9, 2024
Adding @AaronRobinsonMSFT ? - some delegate changes were made in #105584 on Jul 27 which was a couple days before the first report on July 29.
mangod9 commentedon Aug 9, 2024
this looks similar to #106072 (comment), and a possible root cause PR has been reverted.
AaronRobinsonMSFT commentedon Aug 10, 2024
I think this was fixed with #106072 (comment)
/cc @jkotas