problem description
background: there is an application written by golang online, which runs in the docker container, and uses cgo to call the so dynamic library. It will crash because of some circumstances on the weekend, and recover will not catch it. It is suspected that it is the problem of so. I hope to find out the cause of the crash by analyzing the core file.
so now the core question is: how to generate core files for processes with pid 1 in the container?
what methods have you tried
putting aside the container, we know that Go applications need
to generate core files in the system.- the system supports generating core-enter
ulimit-c unlimitedon the currently logged in terminal. - set storage location-
echo"/ tmp/core.%t.%e.%p" | sudo tee / proc/sys/kernel/core_pattern - for golang applications, you need to add the environment variable at startup:
GOTRACEBACK=crash.
for containers, the first two settings are handled:
-
docker runplus parameter-- ulimit core=-1-- security-opt seccomp=unconfined.
The - container inherits the
/ proc/sys/kernel/core_patternconfiguration settings of the host. Here you choose to modify the configuration of the host directly
trigger the way to generate coredump. Here you choose to send a signal to the process. You have tried two ways
-
docker execenter the related container and executekill-SIGQUIT 1 - execute
docker kill-- signal=SIGQUIT < container_id >on the host machine
version of related software
- system: Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-135-generic x86y64)
- docker: client/Server 17.09.0-ce
- docker image: https://hub.docker.com/r/goog...
what result do you expect? What is the error message actually seen?
PS: to avoid container restart caused by pid 1 suspension, a directory of the host is mounted into the / tmp directory of the container.
is that no core file is generated in the / tmp directory.
for comparison, I can generate the core file (actually the core file of the gops process) by Ctrl +\ manually aborting gops trace 1 in the same container I"m running.