Linux Out of Memory? Fix OOM Kills and Docker Exit 137

Your bot disappears, a build prints “Killed”, or a container keeps restarting. By the time you log in, the memory graph looks fine. That is frustrating, but not unusual: terminating the process can release the memory that caused the incident.

Keep the failure time, container name and recent deployment details. If the machine is still struggling, pause nonessential builds or imports through their normal controls. Avoid restarting everything before collecting evidence, and use the HYEHOST panel console if SSH becomes unresponsive.

1. Separate the three common failure types

Docker documents exit 137 as a SIGKILL termination. Treat the number as the start of the investigation, not the diagnosis.

2. Check available memory, not just “free”

free -h
ps -eo pid,comm,rss,%mem --sort=-rss | head -n 16
cat /proc/pressure/memory

In free -h, start with available. Linux uses spare RAM for caches, so a small “free” value is not automatically a problem. Available memory estimates what can be used by new applications without swapping. See the free manual for the column definitions.

The process list sorts by resident memory, with RSS reported in KiB. It is a useful shortlist, not a precise total: shared pages can be counted against multiple processes. The command uses executable names rather than full command arguments to avoid unnecessarily displaying credentials. The ps manual explains the fields.

On kernels with pressure-stall information, /proc/pressure/memory shows time lost waiting on memory. The avg10, avg60 and avg300 values cover different time windows. Persistent pressure alongside slow requests is more useful evidence than one high usage percentage. If the file is absent, skip this check. See the kernel PSI documentation.

3. Confirm what killed the process

sudo journalctl -k -b --since "2 hours ago" --no-pager \
  | grep -Ei 'out of memory|oom-kill|killed process'
sudo journalctl -u systemd-oomd --since "2 hours ago" --no-pager

Adjust the time range to the incident. A kernel entry naming the killed process is much stronger evidence than the shell printing “Killed”. Check surrounding messages too: the victim is not necessarily the only service responsible for exhausting memory.

If the machine rebooted, inspect the previous boot with sudo journalctl -k -b -1 --no-pager. Older records may be unavailable when the journal is not persistent or retention has expired. An empty search is therefore not proof that no OOM event occurred.

Some installations use systemd-oomd, which can terminate a workload in response to pressure before the kernel reaches its own OOM condition. If installed and active, sudo oomctl dump shows the monitored state. It is normal for these tools or records to be absent on systems that do not use the daemon. Read the upstream systemd-oomd documentation; do not disable it just to hide the symptom.

4. Diagnose Docker OOMKilled and exit 137

First list containers, including stopped ones, and check current usage:

docker ps -a --format 'table {{.Names}}\t{{.Status}}'
docker stats --no-stream

Replace my-app below with the affected container name. These commands require Docker access; use sudo if that is how your server is administered.

docker inspect --format \
  'OOMKilled={{.State.OOMKilled}} ExitCode={{.State.ExitCode}} FinishedAt={{.State.FinishedAt}}' \
  my-app
docker inspect --format \
  'Memory={{.HostConfig.Memory}} MemorySwap={{.HostConfig.MemorySwap}}' \
  my-app
docker logs --since 2h --tail 200 my-app

OOMKilled=true supports an OOM diagnosis for that container state. Correlate it with host logs and timestamps. A recreated container is a new object, and a running container’s current state is not a complete incident history. If exit 137 appears without OOM evidence, investigate manual kills, deployments and stop timeouts.

Memory is the configured hard limit in bytes; zero means Docker has not set an explicit container memory limit. MemorySwap has special values and represents a combined RAM-plus-swap allowance when configured positively. Check Docker’s resource-limit documentation before interpreting it.

docker stats is a current snapshot, not peak history. On Linux, its displayed memory usage subtracts cache, so it may differ from other monitoring tools. Use the Docker stats reference when comparing figures. Do not paste application logs publicly without checking for tokens, personal data and connection strings.

5. Check systemd and cgroup limits

For an application run directly as a systemd service, replace my-app.service with its actual unit name:

systemctl show my-app.service \
  -p MemoryCurrent -p MemoryHigh -p MemoryMax -p ControlGroup
sudo journalctl -u my-app.service --since "2 hours ago" --no-pager

On cgroup v2 systems, memory.high applies reclaim pressure and throttling; memory.max is a hard boundary that can lead to an OOM kill. A service can therefore fail while the VPS still has memory available. Parent cgroups can impose limits too.

For deeper inspection, use the ControlGroup path returned above beneath /sys/fs/cgroup and read its memory.events file. An increasing oom_kill counter is evidence of OOM kills associated with that cgroup. Counters are cumulative, so compare before and after a failure. These file names are specific to cgroup v2; consult the kernel memory-controller reference rather than guessing a path.

6. Apply the fix that matches the evidence

For Compose, merge a measured limit into the existing service definition. This example is illustrative, not a recommended size for every application:

services:
  app:
    # Keep the existing image, volumes, networks and environment.
    mem_limit: 1g
    mem_reservation: 768m

mem_limit caps the container; mem_reservation is a softer reservation, not a separate pool of guaranteed RAM. If you also set deploy-level limits, keep them consistent. See the Compose memory settings.

Back up the Compose file and confirm persistent data is on the intended volumes before applying a change. From the project directory, validate with docker compose config -q. During a maintenance window, docker compose up -d --no-deps app applies the change and can recreate that service, interrupting it. Inspect the resulting limit and check application health afterwards. Do not replace a complete Compose file with the fragment above. Docker documents configuration validation and service recreation separately.

7. Decide whether swap will actually help

swapon --show
free -h

Swap can provide breathing room for less-active memory, but it is slower than RAM. A workload that needs its full working set continuously may become unusably slow rather than healthy. Adding swap also does not automatically remove a container’s memory cap or its swap restrictions.

Check the distribution’s guidance and the workload’s latency requirements before changing swap. Avoid disabling existing swap during a memory incident: moving those pages back into RAM can increase pressure. The swapon and swapoff manual covers host swap, while Docker’s swap rules apply separately to containers.

8. Size a VPS from measurements

Record normal usage, busy periods and maintenance peaks. As an illustrative budget, an application using 900 MiB, a database using 500 MiB and 400 MiB for other processes already totals 1,800 MiB before a build or import starts. That would leave little room on a 2 GiB machine. It is a planning example, not a HYEHOST benchmark.

Our Cloud VPS range is a fit for applications that need a full operating system and their own Docker or systemd configuration. Compare the RAM allocation against your measured peaks. VDS adds dedicated CPU resources when sustained compute is also part of the problem; dedicated cores alone do not fix insufficient memory.

If the workload is just a Discord or Telegram bot, managed bot hosting removes the need to administer a complete VPS. It still has a memory allowance: check the runtime, dependencies and peak usage before choosing a plan. Cloud VPS and bot hosting are available in Wolverhampton, Ashburn and Auckland; choose the region for your users, not as a workaround for a memory leak.

9. Verify recovery and catch the next peak

1. Repeat the failing task in a safe test window and check that it completes.
2. Watch memory over time, not just immediately after a restart. Include imports, scheduled jobs and deploys.
3. Check for new kills and restarts. Revisit the journal, container state and application health.
4. Alert on useful signals: sustained low available memory, pressure, container restarts and user-facing failures.
5. Keep recovery separate. A memory upgrade is not a backup. Retain application data and configuration off the affected VM.

Our Beszel monitoring guide is one option for collecting server and container history. If the failure turns out to be storage rather than RAM, use the separate Linux disk-full troubleshooting guide.

Linux out-of-memory FAQs