Cells 2.0 - Error while applying modifiers to registry

alpha23 · December 6, 2019, 11:32pm

Why am I receiving the errors below when bringing up a new container with an existing install? On a side note, cells has been the most frustrating install/configuration using docker of the numerous service applications using docker that I have installed. I do not know if it is because of the design or the testing but I’ve encountered many configuration issues.

The following is the relevant part of the compose file:
services:
cells:
container_name: pydio
image: pydio/cells:2.0.0
restart: on-failure:5
volumes:
- ./data:/var/cells/data
- ./cellsdir:/var/cells
- /etc/timezone:/etc/timezone:ro
- ./ssl/server.pem:/root/ssl/ssl.cert
- ./ssl/server.key:/root/ssl/ssl.key
networks:
- back
environment:
- CELLS_BIND=${URL}:${PORT}
- CELLS_EXTERNAL=${URL}:${PORT}
- CELLS_NO_SSL=0
- CELLS_SSL_CERT_FILE=/root/ssl/ssl.cert
- CELLS_SSL_KEY_FILE=/root/ssl/ssl.key
ports:
- “${PORT}:${PORT}”

The following are the errors:

pydio | 2019-12-06T23:18:06.864Z INFO pydio.gateway.proxy Restart done
pydio | 2019-12-06T23:18:09.646Z ERROR Error while applying modifiers to registry! {“error”: “{"id":"go.micro.client","code":500,"detail":"none available","status":"Internal Server Error"}”}
pydio | 2019-12-06T23:18:09.749Z ERROR Frontend Service Not Detected
pydio | 2019-12-06T23:18:14.127Z ERROR Error while applying modifiers to registry! {“error”: “{"id":"go.micro.client","code":500,"detail":"none available","status":"Internal Server Error"}”}
pydio | 2019-12-06T23:18:14.231Z ERROR Frontend Service Not Detected
pydio | 2019-12-06T23:18:16.710Z ERROR Error while applying modifiers to registry! {“error”: “{"id":"go.micro.client","code":500,"detail":"none available","status":"Internal Server Error"}”}
pydio | 2019-12-06T23:18:16.813Z ERROR Frontend Service Not Detected

zayn · December 19, 2019, 10:51am

Hello @alpha23,

alpha23:

pydio | 2019-12-06T23:18:06.864Z INFO pydio.gateway.proxy Restart done
pydio | 2019-12-06T23:18:09.646Z ERROR Error while applying modifiers to registry! {“error”: “{“id”:“go.micro.client”,“code”:500,“detail”:“none available”,“status”:“Internal Server Error”}”}
pydio | 2019-12-06T23:18:09.749Z ERROR Frontend Service Not Detected
pydio | 2019-12-06T23:18:14.127Z ERROR Error while applying modifiers to registry! {“error”: “{“id”:“go.micro.client”,“code”:500,“detail”:“none available”,“status”:“Internal Server Error”}”}
pydio | 2019-12-06T23:18:14.231Z ERROR Frontend Service Not Detected
pydio | 2019-12-06T23:18:16.710Z ERROR Error while applying modifiers to registry! {“error”: “{“id”:“go.micro.client”,“code”:500,“detail”:“none available”,“status”:“Internal Server Error”}”}
pydio | 2019-12-06T23:18:16.813Z ERROR Frontend Service Not Detected

The frontend error should be normal behaviour at the beginning (it is starting all the services).

Do you have the errors still appear after?

alpha23 · December 20, 2019, 3:07am

The services do not come up. The errors above are exhibited. Has the container been tested under the following conditions?

Existing install, existing data, new database
New install, existing data, existing database
Existing install, existing data, existing database

For existing installs, the application should not reinstall the database, if it exists or overwrite data. For a new install with existing data and database, the application should only reinstall the require app files.

Please advise.

alpha23 · January 2, 2020, 12:26am

Please advise. If the container has not been tested under at least the above conditions, users could face data loss and/or incur substantial difficulty getting the application to a known good state.

zayn · January 2, 2020, 8:29am

Hello @alpha23,

The container install checks if you already have a pydio.json file (which contains the configuration of the db, datasource, …)

github.com

pydio/cells/blob/master/tools/docker/images/cells/docker-entrypoint.sh

#!/bin/sh
set -e

# first arg is `-f` or `--some-option`
if [ "${1#-}" != "$1" ]; then
	set -- cells "$@"
fi

if [ "$1" == "cells" ]; then
	# This file acts as a flag to check if we can start Cells or if we want to perform the non-interactive install.
	FILE="/$CELLS_WORKING_DIR/pydio.json"
	if [ ! -f "$FILE" ] ; then 
		
		if [ -f "$CELLS_INSTALL_YAML" ] ; then 
			# Non interactive install based on a yaml config file   	
			cells install --yaml $CELLS_INSTALL_YAML "$@"

		elif [ -f "$CELLS_INSTALL_JSON" ] ; then 
			# Non interactive install based on a json config file   	
			cells install --json $CELLS_INSTALL_JSON "$@"

This file has been truncated. show original

Therefore it is best if you have a volume for the config and data, then when you run a new container it should use the same configuration file (pydio.json) and connect to the database where your files are installed.

(The container should act as just a simple Cells binary) and everything else should be a volume.

alpha23 · January 8, 2020, 10:01pm

Per my initial post, I do have a volume for config and data. See my compose config above which includes:

volumes:

./data:/var/cells/data
./cellsdir:/var/cells

While the code may appear check the conditions you mentioned, has the following been tested?

Existing install, existing data, new database
New install, existing data, existing database
Existing install, existing data, existing database

Please advise. The same error in the original post is exhibited when instantiated new services with existing volumes.

Feydib · November 5, 2020, 5:36pm

@alpha23, did you finaly manage to sort it out ?
I’m facing exactly the same issue when I want to recompose my docker and use the same volumes. I tried 1, 2 and 3 options and in each situation I don’t manage to start Cells.
The only way for me is to delete all data, all working dir and the db, then run docker-compose up -d again and restart all configuration from scratch. It could not be a problem but if this is the process, I’d like to understand how I can backup and restore my data if I need to rebuild one of my containers…

bsinou · November 6, 2020, 7:56am

Hello,

FYI, the “configuration” of the app is stored both in the config fies and in the database.
So, if you restore an existing installation with a fresh image, you have to insure:

you mount the existing configuration volume (CELLS_WORKING_DIR, that is by default at /var/cells ) in the container
you have a DB running “at the same location” (that is connection and database info have not changed) => this info is stored in the /var/cells/pydio.json and used by Cells to find the DB at starting time.

Thus said, about your use cases:

Existing install, existing data, new database => wont work, cf above, you will get an error at launch time, the system thinks then that it has lost the DB
New install, existing data, existing database => this can be recovered but it is not straight forward and cant be easily automated in your docker compose. TL;DR it is not the idea (cf above)
Existing install, existing data, existing database => this works and has been tested many times. You might give more details about your config and the error you get so that someone might help you on this specific situation.

Hope this helps.

Feydib · November 6, 2020, 11:08am

Hi @bsinou,
Thanks for your answer. I agree with the 2 first use case so lets concentrate on the last one (Existing install, existing data, existing database).
Let me explain my steps:
1/ Fresh install from docker-compose with named volumes
I have 3 containers : 1 for cell, 1 for mariaDB, 1 for PHPmyadmin
2/ First access to cells (https://localhost) with initial configuration.
Here, everything works. I can stop and restart all my dockers and they all restart correctly.

Now I try to recover from my data so next steps are:
3/ I do a docker-compose down which removes all containers but not my named volume. I verified that all data are still there.
4/ do a docker-compose up -d (I didn’t change anything neither in the docker-compose file, nor in the folders)
This restarts and rebuild all containers but I can’t access Pydio Cells. docker logs pydio_cells_1 it shows the error (subject of this thread). And I never managed to correct that.

However, I manage to recover my containers by following the backup/restore procedure which needs some more steps. I descibe them if some people have the same issue, it could help:
Before step 3, do a docker-compose stop then do a backup of each named volume (ie : cp -p * ./backup/)
Afterward, you can do steps 3 and 4, but you need 1 more step, you need to restore volumes at initial place replacing the once that have been created with the new install… (mv ./backup/ * .)

Then restart your containers (docker-compose restart) and this time, I manage to get everything working…

bsinou · November 6, 2020, 11:45am

I had a look at your first docker compose,
First question, could you try to use the latest stable version:

instead of:

image: pydio/cells:2.0.0

use:

image: pydio/cells:2.1.6

the 2.0.0 is exactly one year old today (happy birthday ) but also much too old : we have solved many issues this year and I really cannot help you if you decide to stay there.

Feydib · November 6, 2020, 12:21pm

Sorry, here is my docker compose:

version: ‘3.3’

services:

cells:
image: pydio/cells:latest
restart: always
ports: [“443:443”,“8080:8080”]
environment:
- CELLS_BIND=$INT_WEB_URL
- CELLS_EXTERNAL=$EXT_WEB_URL
volumes:
- “./cells-work:/var/cells”
- “./cells-data:/var/cells/data”
networks:
- cells-backend

MariaDB image with a default database cells and a dedicated user pydio

cells-db:
image: mariadb:latest
restart: always
expose: [“3306”]
environment:
MYSQL_ROOT_PASSWORD: $MYSQL_ROOT_PASSWORD
MYSQL_DATABASE: $CELLS_DB
MYSQL_USER: $MYSQL_USER
MYSQL_PASSWORD: $MYSQL_PASSWD
DEFAULT_CHARACTER_SET: utf
CHARACTER_SET_SERVER: utf
COLLATION_SERVER: utf8_general_ci
INNODB_LARGE_PREFIX: ‘ON’
volumes:
- “./mysql-data:/var/lib/mysql”
networks:
- cells-backend

PHP Myadmin container to manage MariaDB database

myadmin:
image: phpmyadmin:latest
networks:
- cells-backend
ports: [“127.0.0.1:8181:80”]
links:
- “cells-db:db”

networks:
cells-backend:

As you can see, I use latest version (current is 2.1.6)

bsinou · November 6, 2020, 12:24pm

OK and could you precise the MariaDB version, please ?

Feydib · November 6, 2020, 12:42pm

Server version: 10.1.19-MariaDB-1~jessie

bsinou · November 6, 2020, 1:14pm

OK we might have a suspect here: see https://pydio.com/en/docs/cells/v2/cells-installation

We support Maria DB 10.3 and above. Can you update this image or not ?

or use mysql 5.7, with something like that:

 mysql:
    image: mysql:5.7
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: P@ssw0rd
      MYSQL_DATABASE: cells
      MYSQL_USER: pydio
      MYSQL_PASSWORD: pydio
    volumes:
      - "./mysqldir:/var/lib/mysql"
    command:
      [
        mysqld,
        --character-set-server=utf8mb4,
        --collation-server=utf8mb4_unicode_ci,
      ]

Feydib · November 6, 2020, 11:00pm

Good catch !

I was working on an old MariaDB docker image (reminder for myself : don’t use latest tag !)
So I change my docker-compose to use MariaDB:10.5.7 (latest at the time writing), run a docker-compose up -d which rebuild my DB. I tested that Cells was still working: YES.
Then I deleted my containers with docker-compose down and then did a last docker-compose up -d (note : previously, this steps was failing…) and this time everything works !

Thank you very much @bsinou.

system · May 16, 2023, 10:30am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Cells 2.0 docker compose install errors Pydio Cells install	2	1157	November 20, 2019
[Help Wanted] Deploying Pydio Cells to Kubernetes Pydio Cells	1	870	September 24, 2023
SSL self-signed config does not work Pydio Cells	4	1326	September 11, 2019
Cells install on Windows server 2019 Pydio Cells install	7	724	August 12, 2021
[Solved] Ssl Cells install on Ubuntu 18. Fails to parse private key Pydio Cells	2	707	March 28, 2019

Cells 2.0 - Error while applying modifiers to registry

MariaDB image with a default database cells and a dedicated user pydio

PHP Myadmin container to manage MariaDB database

Related topics