Skip to content

Native-delay RabbitMQ image crash-loops: root-owned .erlang.cookie (eacces) #4566

Closed

Description

@iancooper

Symptom

The image built from docker/RabbitMQ/Dockerfile, which docker-compose-rmq-native-delay.yaml uses on port 5673 for the RMQ native-delay tests, exits on start:

Kernel pid terminated (application_controller) ("{application_start_failure,rabbitmq_prelaunch,
  ... {failed_to_start_child,auth,{\"Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces\", ...

With no broker on 5673, every native-delay test in RMQ.Sync fails with Connection refused. That is 48 tests: the NativeClassic suite plus the native-exchange, retry-topology and native-delay tests.

The compose file mounts no volume, so this is not the stale rabbitmq_data volume that docker-compose-rmq.yaml warns about. The image itself is the problem.

Seen on: Podman 5.8.2 (applehv) on macOS. I have not checked whether Docker Desktop behaves the same way.

Likely cause

The RUN step runs as root and calls rabbitmq-plugins enable --offline .... This most likely starts Erlang with HOME=/var/lib/rabbitmq and writes a root-owned .erlang.cookie into the image. At runtime the server runs as rabbitmq (uid 999) and cannot read it.

Workaround

Start the image as root. The official entrypoint then fixes ownership under /var/lib/rabbitmq and drops privileges to rabbitmq:

docker run -d --name brighter-rmq-native-delay --user 0 -p 127.0.0.1:5673:5672 \
  -e RABBITMQ_DEFAULT_USER=guest -e RABBITMQ_DEFAULT_PASS=guest -e RABBITMQ_FEATURE_FLAGS=-khepri_db \
  brighter-4388-rabbitmq-native:local

With the broker started this way, RMQ.Sync passes 186/187 (1 skipped) without the mTLS tests.

Suggested fix

Either of these:

  • In the Dockerfile, after rabbitmq-plugins enable --offline, remove the cookie the build created (rm -f /var/lib/rabbitmq/.erlang.cookie) or chown rabbitmq:rabbitmq it.
  • Add user: "0" to the service in docker-compose-rmq-native-delay.yaml.

The Dockerfile fix is cleaner, because then the image works however it is started.

Also noticed

docker-compose ... up --build failed to pull rabbitmq:4.2.6-management with unable to retrieve auth token: invalid username/password when stale Docker Hub credentials were stored, even though an anonymous podman pull of the same image worked. Pre-pulling the base image worked around it. That is local environment, not the repo, but it is worth a line in docker/RabbitMQ/README.md.

Found while verifying #4562.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3 - DoneBugMaintenanceBuild, CI, refactoring, testing infrastructure, and other choresV10.X

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions