DNS lookup errors in docker container

Hi,

Using the official docker container, I noticed that caddy logs dns lookup errors constantly and is filling up the container. Here’s a snippet of /var/cells/logs/caddy_errors.log

13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
/var/cells/logs # 

127.0.0.11 is the internal docker dns address, which normally forwards to the host’s dns.

Tried this on a local machine as well as a DigitalOcean droplet (local machine on ubuntu xenial, DO droplet on bionic) and the result is the same.

Thanks

EDIT: Here’s the compose snippet:

---
version: "2"
services:
  pydio-cells:
    image: pydio/cells
    container_name: pydio-cells
    hostname: pydio-cells
    environment:
      - CELLS_BIND=0.0.0.0:8080
      - CELLS_EXTERNAL=192.168.1.25:8080
      - CELLS_NO_TLS=1
    volumes:
      - /home/aptalca/pydio-cells:/var/cells/data
    ports:
      - 8080:8080
    restart: unless-stopped
    mem_limit: 512m
  cellsdb:
    image: linuxserver/mariadb
    container_name: cellsdb
    environment:
      - PUID=1000
      - PGID=1000
      - MYSQL_ROOT_PASSWORD=test
      - TZ=Europe/London
    volumes:
      - /home/aptalca/cellsdb:/config
    restart: unless-stopped

Hello @aptalca,

Could you try with CELLS_BIND also set to a private address, for your case 192.168.1.25:8080,
then check if it still does log this error.

It still logs the error with CELLS_BIND set to docker host IP.

I also noticed that when the container starts, the error is logged for about 25 seconds, and then the error entries stop (I had initially assumed the logging was continuous).

Confirmed through multiple docker starts and it logs the error for 20-25 seconds, multiple times a second, on each container start.

Does your container restart many times?

Does it mean that the container cannot use more than 512 of RAM?
if so it could be what makes the container restart many times and therefore making logs in the caddy_errors

No, it actually runs fine apart from the dns errors listed above. I had to set the limit because otherwise my Digitalocean droplet runs out of memory (1gb total).

On my local machine I run without the memory limit and still get the errors.

Hello,

Same issue here . Did you manage to solve it ?

I’ve got the same problem as well working through a Kubernetes install. It works fine until the container is restarted.

Hello,

I’ll reproduce a setup and ask the devs if they could analyze the behaviour.

@aptalca try to use your external IP as you bind address and see if that works. Also try without the no_tls option