Repository navigation
nerdctl kill isn't working after update to 2.x (unable to cleanup network) #3765
Description
Description
Hey,
with the update to version 2.0, nerdctl kill stopped working for us. Instead of sending the signal to the container, we just receive an error:
unable to cleanup network for container: abc0123456789, "unexpected end of JSON input"
It doesn't matter, if the container is spawned in the host network or if a overlay network is used. Not sure, if I missed something during the 2.0 release, but at least in the changelog I can't find any related breaking change.
The tested configuration is working fine with version 1.7.7 of nerdctl.
Steps to reproduce the issue
- Start a container via containerd
- Send SIGHUP to the container:
sudo nerdctl kill -s HUP abc0123456789
Describe the results you received and expected
received result:
Just an error message - no signal was send:
FATA[0000] 1 errors:
unable to cleanup network for container: bfc5d8b4fa8da, "unexpected end of JSON input"
expected result:
Container receives a SIGHUP.
What version of nerdctl are you using?
Client:
Version: v2.0.2
OS/Arch: linux/amd64
Git commit: 1220ce7ec2701d485a9b1beeea63dae3da134fb5
buildctl:
Version:
Server:
containerd:
Version: v2.0.0
GitCommit: 207ad711eabd375a01713109a8a197d197ff6542
runc:
Version:
Tested also with containerd 2.0.1
config:
debug = false
debug_full = false
address = "unix:///run/containerd/containerd.sock"
namespace = "k8s.io"
cgroup_manager = "systemd"
experimental = true
cni_netconfpath = "/etc/cni/net.d"
Are you using a variant of nerdctl? (e.g., Rancher Desktop)
None
Host information
Client:
Namespace: k8s.io
Debug Mode: false
Server:
Server Version: v2.0.0
Storage Driver: overlayfs
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Log: fluentd journald json-file none syslog
Storage: native overlayfs
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.8.0-41-generic
Architecture: x86_64
Activity
apostasie commentedon Dec 16, 2024
Hey @mathias-ioki
Can you clarify what you mean with
Start a container via containerdor provide the exact command you are using?Things seem to work fine for me with:
The error message and location suggests that there might be something wrong with your network CNI conf.
apostasie commentedon Dec 16, 2024
Could also be that the container was not created by nerdctl, or has otherwise a problem with the label
nerdctl/networks.nerdctl inspect offendingcontainer< could provide some insight.mathias-ioki commentedon Dec 16, 2024
Hey @apostasie, thank you for your fast reply!
We are using a setup with containerd + kubelet in standalone mode. So we are shipping container manifest files to kubelet, which is taking care of starting the container via containerd. So yes, you are right: These containers are neither managed via nerdctl nor have nerdctl related labels attached.
As far as I understood, nerdctl can be always used for debugging, even if it doesn't manage the containers itself. Please let me know, if this isn't the case.
apostasie commentedon Dec 16, 2024
@mathias-ioki agreed with you - nerdctl should at least bail out politely if something is not manageable instead of planting face.
I'll send a patch in a minute.
apostasie commentedon Dec 16, 2024
@mathias-ioki ^
mathias-ioki commentedon Dec 17, 2024
Oh, that was super fast again! Thank you very much @apostasie ! I will give it a try later and let you know, if this fix my issue :)