Repository navigation
macos "Broken pipe" exceptions during tests #162
Description
On macos there is error output running test_bitcoin -t ipc_tests that looks like:
capnp/rpc.c++:3666: error: exception = (unknown):-1: failed: std::exception: (unknown):-1: failed: std::exception: kj/async-io-unix.c++:778: disconnected: ::writev(fd, iov.begin(), iov.size()): Broken pipe; iovTotal = 80; iov.size() = 2followed by lists of stack addresses, repeated may times throughout the test.
It's not really clear what causes these errors, but they seem harmless, and might have to do with capnproto not really having a clean disconnect procedure (according to https://groups.google.com/g/capnproto/c/T7Yjcwi1z2Y). But regardless, libmultiprocess probably should be doing something to catch and handle these errors rather than letting them go to stderr.
These errors happens using capnp 1.1.0 installed from homebrew, and a not very informative stack trace that just shows a write failing looks like:
* thread #2, stop reason = breakpoint 1.1
* frame #0: 0x00007ff80921defa libc++abi.dylib`__cxa_throw
frame #1: 0x00000001025b6f8a libkj.1.1.0.dylib`kj::ExceptionCallback::RootExceptionCallback::onRecoverableException(kj::Exception&&) + 90
frame #2: 0x00000001025b1fa6 libkj.1.1.0.dylib`kj::_::Debug::Fault::~Fault() + 190
frame #3: 0x000000010285e666 libkj-async.1.1.0.dylib`kj::(anonymous namespace)::AsyncStreamFd::writeInternal(kj::ArrayPtr<unsigned char const>, kj::ArrayPtr<kj::ArrayPtr<unsigned char const> const>, kj::ArrayPtr<int const>) + 612
frame #4: 0x000000010285ca62 libkj-async.1.1.0.dylib`kj::(anonymous namespace)::AsyncStreamFd::write(kj::ArrayPtr<kj::ArrayPtr<unsigned char const> const>) + 90
frame #5: 0x000000010285cc10 libkj-async.1.1.0.dylib`non-virtual thunk to kj::(anonymous namespace)::AsyncStreamFd::write(kj::ArrayPtr<kj::ArrayPtr<unsigned char const> const>) + 18
frame #6: 0x00000001026486d6 libcapnp-rpc.1.1.0.dylib`capnp::writeMessages(kj::AsyncOutputStream&, kj::ArrayPtr<kj::ArrayPtr<kj::ArrayPtr<capnp::word const> const> >) + 360
frame #7: 0x0000000102649f8c libcapnp-rpc.1.1.0.dylib`capnp::BufferedMessageStream::writeMessages(kj::ArrayPtr<kj::ArrayPtr<kj::ArrayPtr<capnp::word const> const> >) + 22
frame #8: 0x0000000102648adb libcapnp-rpc.1.1.0.dylib`capnp::MessageStream::writeMessages(kj::ArrayPtr<capnp::MessageAndFds>) + 315
frame #9: 0x000000010269f591 libcapnp-rpc.1.1.0.dylib`capnp::TwoPartyVatNetwork::OutgoingMessageImpl::send()::'lambda'()::operator()() const::'lambda'()::operator()() const + 301
frame #10: 0x000000010269fdbc libcapnp-rpc.1.1.0.dylib`kj::_::TransformPromiseNode<kj::Promise<void>, kj::_::Void, capnp::TwoPartyVatNetwork::OutgoingMessageImpl::send()::'lambda'()::operator()() const::'lambda'(), kj::_::PropagateException>::getImpl(kj::_::ExceptionOrValue&) + 586
frame #11: 0x0000000102815aac libkj-async.1.1.0.dylib`kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&) + 32
frame #12: 0x00000001028165a8 libkj-async.1.1.0.dylib`kj::_::ChainPromiseNode::fire() + 120
frame #13: 0x0000000102816bfa libkj-async.1.1.0.dylib`non-virtual thunk to kj::_::ChainPromiseNode::fire() + 18
frame #14: 0x0000000102813e24 libkj-async.1.1.0.dylib`kj::EventLoop::turn() + 100
frame #15: 0x00000001028145d4 libkj-async.1.1.0.dylib`kj::_::waitImpl(kj::Own<kj::_::PromiseNode, kj::_::PromiseDisposer>&&, kj::_::ExceptionOrValue&, kj::WaitScope&, kj::SourceLocation) + 518
frame #16: 0x00000001011f36d9 test_bitcoin`kj::Promise<unsigned long>::wait(this=<unavailable>, waitScope=<unavailable>, location=(fileName = "ipc/libmultiprocess/src/mp/proxy.cpp", function = "loop", lineNumber = 196, columnNumber = 35)) at async-inl.h:1357:3 [opt]
frame #17: 0x00000001011f30e9 test_bitcoin`mp::EventLoop::loop(this=0x000070000e9a1e20) at proxy.cpp:196:68 [opt]
frame #18: 0x0000000100f9cd01 test_bitcoin`void* std::__1::__thread_proxy[abi:ne190107]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, IpcPipeTest()::$_0> >(void*) [inlined] IpcPipeTest(this=<unavailable>)::$_0::operator()() const at ipc_test.cpp:75:14 [opt]
frame #19: 0x0000000100f9c887 test_bitcoin`void* std::__1::__thread_proxy[abi:ne190107]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, IpcPipeTest()::$_0> >(void*) [inlined] decltype(__f=<unavailable>)::$_0>()()) std::__1::__invoke[abi:ne190107]<IpcPipeTest()::$_0>(IpcPipeTest()::$_0&&) at invoke.h:149:25 [opt]
frame #20: 0x0000000100f9c887 test_bitcoin`void* std::__1::__thread_proxy[abi:ne190107]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, IpcPipeTest()::$_0> >(void*) [inlined] _ZNSt3__116__thread_executeB8ne190107INS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEZ11IpcPipeTestvE3$_0JETpTnmJEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE(__t=<unavailable>, (null)=<unavailable>) at thread.h:192:3 [opt]
frame #21: 0x0000000100f9c887 test_bitcoin`void* std::__1::__thread_proxy[abi:ne190107]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, IpcPipeTest()::$_0> >(__vp=<unavailable>) at thread.h:201:3 [opt]
frame #22: 0x00007ff8092621d3 libsystem_pthread.dylib`_pthread_start + 125
frame #23: 0x00007ff80925dbd3 libsystem_pthread.dylib`thread_start + 15Stack trace was obtained with lldb:
lldb -- src/test/test_bitcoin -t ipc_tests
breakpoint set -E c++
run
bt
Activity
ryanofsky commentedon Aug 23, 2025
This should be resolved now with disconnection exception handling improved in 56fff76 and other disconnection handling improvements and tests in #160