DNS lookup errors in docker container

aptalca · December 13, 2019, 12:28am

Hi,

Using the official docker container, I noticed that caddy logs dns lookup errors constantly and is filling up the container. Here’s a snippet of /var/cells/logs/caddy_errors.log

13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:51 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:52 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
13/Dec/2019:00:21:53 +0000 [ERROR 502 /auth/dex/.well-known/openid-configuration] dial tcp: lookup PENDING on 127.0.0.11:53: no such host
/var/cells/logs #

127.0.0.11 is the internal docker dns address, which normally forwards to the host’s dns.

Tried this on a local machine as well as a DigitalOcean droplet (local machine on ubuntu xenial, DO droplet on bionic) and the result is the same.

Thanks

EDIT: Here’s the compose snippet:

---
version: "2"
services:
  pydio-cells:
    image: pydio/cells
    container_name: pydio-cells
    hostname: pydio-cells
    environment:
      - CELLS_BIND=0.0.0.0:8080
      - CELLS_EXTERNAL=192.168.1.25:8080
      - CELLS_NO_TLS=1
    volumes:
      - /home/aptalca/pydio-cells:/var/cells/data
    ports:
      - 8080:8080
    restart: unless-stopped
    mem_limit: 512m
  cellsdb:
    image: linuxserver/mariadb
    container_name: cellsdb
    environment:
      - PUID=1000
      - PGID=1000
      - MYSQL_ROOT_PASSWORD=test
      - TZ=Europe/London
    volumes:
      - /home/aptalca/cellsdb:/config
    restart: unless-stopped

zayn · December 18, 2019, 2:03pm

Hello @aptalca,

Could you try with CELLS_BIND also set to a private address, for your case 192.168.1.25:8080,
then check if it still does log this error.

aptalca · December 18, 2019, 8:57pm

It still logs the error with CELLS_BIND set to docker host IP.

I also noticed that when the container starts, the error is logged for about 25 seconds, and then the error entries stop (I had initially assumed the logging was continuous).

Confirmed through multiple docker starts and it logs the error for 20-25 seconds, multiple times a second, on each container start.

zayn · December 19, 2019, 8:36am

Does your container restart many times?

Does it mean that the container cannot use more than 512 of RAM?
if so it could be what makes the container restart many times and therefore making logs in the caddy_errors

aptalca · December 19, 2019, 1:04pm

No, it actually runs fine apart from the dns errors listed above. I had to set the limit because otherwise my Digitalocean droplet runs out of memory (1gb total).

On my local machine I run without the memory limit and still get the errors.

kmy · December 22, 2019, 9:14am

Hello,

Same issue here . Did you manage to solve it ?

Jamie_Chapman-Brown · December 30, 2019, 3:21am

I’ve got the same problem as well working through a Kubernetes install. It works fine until the container is restarted.

zayn · January 2, 2020, 8:51am

Hello,

I’ll reproduce a setup and ask the devs if they could analyze the behaviour.

kmy · January 6, 2020, 3:37pm

@aptalca try to use your external IP as you bind address and see if that works. Also try without the no_tls option

drzraf · June 22, 2020, 6:35pm

Same error with 2.0.5 and 2.0.7 (native, no docker).
No success with CELLS_BIND.

(reboot fixed it this error, but the login still failed because the /auth/dex/ returns 502)

It returns 502, because caddy is always proxy /auth/dex PENDING {...} which, I believe, is
related to Failed to initialize OIDC server: server: no connectors specified.

At some point I even get
failed to initialize storage: failed to open migrations: creating dex_migration table: Error 1040: Too many connections

Topic		Replies	Views
Pydio Cells With caddy V2 in two different containers. Cannot find authority Pydio Cells install , docker	2	761	September 6, 2021
ERR_CONNECTION_CLOSED after Cells installed with Docker and Caddy reversed proxy Pydio Cells	7	2340	October 18, 2018
[SOLVED] Docker-compose apache reverse proxy - no route to host Pydio Cells	12	2754	July 22, 2019
Several issues with Docker container pydio/cells Pydio Cells	6	1690	May 16, 2023
Pydio fails to start Pydio Cells install , docker	2	1689	January 14, 2021

DNS lookup errors in docker container

Related Topics