Database Errors on docker cell upgraded to V5

Pydio Cells
1

For first post, please answer the questions below!

Describe your issue in detail

Many errors in logs, probably following an unnoticed partial or failed upgrade.

image
image1617×582 59.9 KB

What version of Cells are you using?

V5

image
image915×251 29.4 KB

What is the server OS? Database name/version? Browser name or mobile device description (if issue appears client-side)?

OS : bullseye/sid

DB : dockerized mariadb:latest

What steps have you taken to resolve this issue already?

Restarted the docker. But migration still does not happen some table do not get created. When restarting cells, among some crashes with stack traces I can see this kind of errors :
2026-05-27T12:17:19.413Z INFO pydio.grpc.install [SQL] Error 1005 (HY000): Can’t create table cells.idm_usr_meta_values_rel (errno: 150 “Foreign key constraint is incorrectly formed”) [57.574ms] [rows:0] CREATE TABLE idm_usr_meta_values_rel (e_value_uuid varchar(255) NOT NULL,meta_uuid varchar(191),PRIMARY KEY (e_value_uuid,meta_uuid),CONSTRAINT fk_idm_usr_meta_values_rel_entity_values FOREIGN KEY (e_value_uuid) REFERENCES idm_usr_meta_entity_values(uuid),CONSTRAINT fk_idm_usr_meta_values_rel_meta FOREIGN KEY (meta_uuid) REFERENCES idm_usr_meta(uuid)) {“file”: “github.com/pydio/cells/v5/common/storage/sql/logger.go:82”, “layer”: “sql”, “tag”: “idm”}

2

Hi, thanks for reporting. Can I ask you what version you were upgrading from ? Also, can you run this sql command on your database please to give me more information on the table ?

Thanks very much,

Greg

3

Hello,

Here is the error from the db server :

MariaDB [cells]> CREATE TABLE idm_usr_meta_values_rel (e_value_uuid varchar(255) NOT NULL,meta_uuid varchar(191),PRIMARY KEY (e_value_uuid,meta_uuid),CONSTRAINT fk_idm_usr_meta_values_rel_entity_values FOREIGN KEY (e_value_uuid) REFERENCES idm_usr_meta_entity_values(uuid),CONSTRAINT fk_idm_usr_meta_values_rel_meta FOREIGN KEY (meta_uuid) REFERENCES idm_usr_meta(uuid));
ERROR 1005 (HY000): Can't create table `cells`.`idm_usr_meta_values_rel` (errno: 150 "Foreign key constraint is incorrectly formed")
MariaDB [cells]>

Regarding the version, I would say the last prior to the release of v5, as I have a daily cron updating the docker image.

Regards,

4

Oh so sorry, just realised I hadn’t pasted the command I wanted you to run :slight_smile: :

show create table idm_usr_meta;

If could run this on your db that would be great.

Thanks,

Greg

5

Here you go :

MariaDB [cells]> show create table idm_usr_meta;
+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table        | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| idm_usr_meta | CREATE TABLE `idm_usr_meta` (
  `uuid` varchar(255) NOT NULL,
  `node_uuid` varchar(255) NOT NULL,
  `namespace` varchar(255) NOT NULL,
  `owner` varchar(255) DEFAULT NULL,
  `timestamp` int(11) DEFAULT NULL,
  `format` varchar(50) DEFAULT NULL,
  `data` longblob DEFAULT NULL,
  PRIMARY KEY (`uuid`),
  UNIQUE KEY `namespace` (`namespace`,`node_uuid`,`owner`),
  UNIQUE KEY `idx_idm_usr_meta_u1` (`node_uuid`,`namespace`,`owner`),
  KEY `idm_usr_meta_node_idx` (`node_uuid`),
  KEY `idx_idm_usr_meta_no` (`node_uuid`),
  KEY `idx_idm_usr_meta_ns` (`namespace`),
  KEY `idx_idm_usr_meta_ow` (`owner`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci |
+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Regards,
6

Is this a collation issue? there seem to be a mix in our database :

| idm_workspaces                                 | InnoDB |      10 | Dynamic    |   528 |            186 |       98304 |               0 |       163840 |         0 |           NULL | 2026-05-27 08:55:24 | 2026-05-28 08:59:00 | NULL       | latin1_swedish_ci     |     NULL |                |         |                0 | N         |
| ladon_migrations                               | InnoDB |      10 | Dynamic    |     4 |           4096 |       16384 |               0 |            0 |         0 |           NULL | 2025-06-12 12:46:36 | NULL                | NULL       | utf8mb3_general_ci    |     NULL |                |         |                0 | N         |
| ladon_policy_permission                        | InnoDB |      10 | Dynamic    |     0 |              0 |       16384 |               0 |        16384 |         0 |              1 | 2025-06-12 12:46:36 | NULL                | NULL       | latin1_swedish_ci     |     NULL |                |         |                0 | N         |
| oauth2_access                                  | InnoDB |      10 | Dynamic    |     1 |          16384 |       16384 |               0 |            0 |         0 |           NULL | 2026-05-27 08:55:30 | 2026-05-28 11:32:42 | NULL       | utf8mb4_uca1400_ai_ci |     NULL |                |         |                0 | N         |
| oauth2_code                                    | InnoDB |      10 | Dynamic    |    26 |           1890 |       49152 |               0 |            0 |         0 |           NULL | 2026-05-27 08:55:30 | 2026-05-28 11:32:42 | NULL       | utf8mb4_uca1400_ai_ci |     NULL |                |         |                0 | N         |
| oauth2_flow                                    | InnoDB |      10 | Dynamic    |     1 |          16384 |       16384 |               0 |            0 |         0 |           NULL | 2026-05-27 08:55:29 | 2026-05-28 11:32:42 | NULL       | utf8mb4_uca1400_ai_ci |     NULL |                |         |                0 | N         |
| oauth2_jwk                                     | InnoDB |      10 | Dynamic    |     0 |              0 |       16384 |               0 |            0 |         0 |           NULL | 2026-05-27 08:55:29 | NULL                | NULL       | utf8mb4_uca1400_ai_ci |     NULL |                |         |                0 | N         |
| oauth2_oidc                                    | InnoDB |      10 | Dynamic    |    26 |           1890 |       49152 |               0 |            0 |         0 |           NULL | 2026-05-27 08:55:30 | 2026-05-28 11:32:42 | NULL       | utf8mb4_uca1400_ai_ci |     NULL |                |         |                0 | N         |
| oauth2_pkce                                    | InnoDB |      10 | Dynamic    |     0 |              0 |       16384 |               0 |            0 |         0 |           NULL | 2026-05-27 08:55:30 | 2026-05-28 11:32:42 | NULL       | utf8mb4_uca1400_ai_ci |     NULL |                |         |                0 | N         |
| oauth2_refresh                                 | InnoDB |      10 | Dynamic    |    26 |           1890 |       49152 |               0 |            0 |         0 |           NULL | 2026-05-27 08:55:30 | 2026-05-28 11:32:42 | NULL       | utf8mb4_uca1400_ai_ci |     NULL |                |         |                0 | N         |
| schema_migration                               | InnoDB |      10 | Dynamic    |    55 |            297 |       16384 |               0 |        16384 |         0 |           NULL | 2025-06-12 12:46:37 | NULL                | NULL       | latin1_swedish_ci     |     NULL |                |         |                0 | N         |
+------------------------------------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------------+----------+----------------+---------+------------------+-----------+
68 rows in set (0.014 sec)

MariaDB [cells]> SELECT @@character_set_database, @@collation_database;
+--------------------------+-----------------------+
| @@character_set_database | @@collation_database  |
+--------------------------+-----------------------+
| utf8mb4                  | utf8mb4_uca1400_ai_ci |
+--------------------------+-----------------------+
1 row in set (0.001 sec)

MariaDB [cells]>
7

Hi,

Indeed it looks like a collation issue. Can you run this command on your db (after stopping the docker container) :

ALTER TABLE idm_usr_meta
CONVERT TO CHARACTER SET utf8mb4
COLLATE utf8mb4_uca1400_ai_ci;

It should fix the migration issue and unblock you. We’ll add a fix in-app for the next upgrade.

Thanks,

Greg

8

Thnak you very much.

I do not have all the errors I had before. However, all does not seem perfect. The system seems extremely sluggish for some operation like listing users or listing logs.

When I go to listing it hangs for about 20s before either not showing anything, or logging me off.

I see many timeout errors in the logs :

2026-05-28T15:38:51.657Z        ERROR   pydio.rest.jobs got a token but failed to load claims   {"error": "rpc error: code = DeadlineExceeded desc = context deadline exceeded\ntimeout",

Here is an example that ended with a logoff :

image
image3774×829 206 KB

9

There still must be something with the startup then,

What you’re describing looks like some db issues so I’ll try to look for slow queries, locks or missing indexes. It could also be related to CPU so we’ll look for both.

For db issues :

can you please run the following on mariadb ?

SHOW FULL PROCESSLIST;
SHOW ENGINE INNODB STATUS;

Please also capture the full docker logs for both Cells and MariaDB after a restart ?

If possible, it would also be useful to enable/check slow queries temporarily, because the users/logs listing may be triggering a specific slow query rather than a hard lock:

SET GLOBAL slow_query_log = 'ON';
SET GLOBAL long_query_time = 2;

Then reproduce the issue and send the slow query log if available.

For cpu issues :

can you please run stats on the running containers and send me the output after reproducing the issue please ?

docker ps
docker stats --no-stream <cells_container_name> <mariadb_container_name>

Thanks very much,

Greg

10

Hello,

I have run the logs, and they are here (I have not been able to get the slow log to show though):slight_smile: InTest Document Server

Logs1.txt : mariadb status & process list before logging in, after logging in, after showing cells console, during waiting for log, after log have shown nothing

Logs2.txt : same, but after restarting cells

Logs3.txt : Docker Stats during restart

Logs_Cell : docker logs for cell restart was ~11:57 UTC

Logs_DB : docker logs for mariadb

11

Thanks very much for that. Can you please run the following on the mariadb please ?

EXPLAIN ANALYZE
  SELECT
    u.uuid,
    u.name,
    COUNT(DISTINCT ur.role) AS role_count,
    COUNT(DISTINCT ua.name) AS attr_count,
    COUNT(DISTINCT rp.resource) AS policy_count
  FROM (
    SELECT uuid, name
    FROM idm_user_idx_tree
    WHERE LOWER(name) = LOWER('<username>')
    LIMIT 1
  ) AS u
  LEFT JOIN idm_user_roles ur
    ON ur.uuid = u.uuid
  LEFT JOIN idm_user_attributes ua
    ON ua.uuid = u.uuid
  LEFT JOIN idm_resource_policies rp
    ON rp.resource = u.uuid
  GROUP BY u.uuid, u.name;

Thanks,

Greg

12

It looks like database is missing at least one table :

MariaDB [cells]> EXPLAIN ANALYZE
    ->   SELECT
    ->     u.uuid,
    ->     u.name,
    ->     COUNT(DISTINCT ur.role) AS role_count,
    ->     COUNT(DISTINCT ua.name) AS attr_count,
    ->     COUNT(DISTINCT rp.resource) AS policy_count
    ->   FROM (
    ->     SELECT uuid, name
    ->     FROM idm_user_idx_tree
    ->     WHERE LOWER(name) = LOWER('<username>')
    ->     LIMIT 1
    ->   ) AS u
    ->   LEFT JOIN idm_user_roles ur
    ->     ON ur.uuid = u.uuid
    ->   LEFT JOIN idm_user_attributes ua
    ->     ON ua.uuid = u.uuid
    ->   LEFT JOIN idm_resource_policies rp
    ->     ON rp.resource = u.uuid
    ->   GROUP BY u.uuid, u.name;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ANALYZE
  SELECT
    u.uuid,
    u.name,
    COUNT(DISTINCT ur.role) AS role_...' at line 1
MariaDB [cells]> EXPLAIN  SELECT     u.uuid,     u.name,     COUNT(DISTINCT ur.role) AS role_count,     COUNT(DISTINCT ua.name) AS attr_count,     COUNT(DISTINCT rp.resource) AS policy_count   FROM (     SELECT uuid, name     FROM idm_user_idx_tree     WHERE LOWER(name) = LOWER('<username>')     LIMIT 1   ) AS u   LEFT JOIN idm_user_roles ur     ON ur.uuid = u.uuid   LEFT JOIN idm_user_attributes ua     ON ua.uuid = u.uuid   LEFT JOIN idm_resource_policies rp     ON rp.resource = u.uuid   GROUP BY u.uuid, u.name;
ERROR 1146 (42S02): Table 'cells.idm_resource_policies' doesn't exist
MariaDB [cells]>
13

sorry my bad it should be idm_user_policies :

EXPLAIN ANALYZE
  SELECT
    u.uuid,
    u.name,
    COUNT(DISTINCT ur.role) AS role_count,
    COUNT(DISTINCT ua.name) AS attr_count,
    COUNT(DISTINCT rp.resource) AS policy_count
  FROM (
    SELECT uuid, name
    FROM idm_user_idx_tree
    WHERE LOWER(name) = LOWER('<username>')
    LIMIT 1
  ) AS u
  LEFT JOIN idm_user_roles ur
    ON ur.uuid = u.uuid
  LEFT JOIN idm_user_attributes ua
    ON ua.uuid = u.uuid
  LEFT JOIN idm_user_policies rp
    ON rp.resource = u.uuid
  GROUP BY u.uuid, u.name;

Please make sure you change the by an existing username so that it matches something in the database,

Thanks,

Greg

14

Here is the explain return (somehow my db does not like the analyze keyword):

MariaDB [cells]> EXPLAIN ANALYZE   SELECT     u.uuid,     u.name,     COUNT(DISTINCT ur.role) AS role_count,     COUNT(DISTINCT ua.name) AS attr_count,     COUNT(DISTINCT rp.resource) AS policy_count   FROM (     SELECT uuid, name     FROM idm_user_idx_tree     WHERE LOWER(name) = LOWER('user')     LIMIT 1   ) AS
 u   LEFT JOIN idm_user_roles ur     ON ur.uuid = u.uuid   LEFT JOIN idm_user_attributes ua     ON ua.uuid = u.uuid   LEFT JOIN idm_user_policies rp     ON rp.resource = u.uuid   GROUP BY u.uuid, u.name;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ANALYZE   SELECT     u.uuid,     u.name,     COUNT(DISTINCT ur.role) AS role_...' at line 1
MariaDB [cells]> EXPLAIN  SELECT     u.uuid,     u.name,     COUNT(DISTINCT ur.role) AS role_count,     COUNT(DISTINCT ua.name) AS attr_count,     COUNT(DISTINCT rp.resource) AS policy_count   FROM (     SELECT uuid, name     FROM idm_user_idx_tree     WHERE LOWER(name) = LOWER('docs')     LIMIT 1   ) AS u   LEFT JOIN idm_user_roles ur     ON ur.uuid = u.uuid   LEFT JOIN idm_user_attributes ua     ON ua.uuid = u.uuid   LEFT JOIN idm_user_policies rp     ON rp.resource = u.uuid   GROUP BY u.uuid, u.name;
+------+-------------+-------------------+------+----------------------------------------------------------+-----------------------+---------+--------+------+--------------------------+
| id   | select_type | table             | type | possible_keys                                            | key                   | key_len | ref    | rows | Extra                    |
+------+-------------+-------------------+------+----------------------------------------------------------+-----------------------+---------+--------+------+--------------------------+
|    1 | PRIMARY     | <derived2>        | ALL  | NULL                                                     | NULL                  | NULL    | NULL   | 2    | Using filesort           |
|    1 | PRIMARY     | ur                | ref  | PRIMARY,idm_usr_roles_uuid_idx,idx_idm_user_roles_ui     | PRIMARY               | 130     | u.uuid | 1    |                          |
|    1 | PRIMARY     | ua                | ref  | PRIMARY,idm_usr_attr_uuid_idx,idx_idm_user_attributes_ui | idm_usr_attr_uuid_idx | 130     | u.uuid | 3    | Using index              |
|    1 | PRIMARY     | rp                | ref  | resource,idx_idm_user_policies_res                       | resource              | 258     | func   | 5    | Using where; Using index |
|    2 | DERIVED     | idm_user_idx_tree | ALL  | NULL                                                     | NULL                  | NULL    | NULL   | 628  | Using where              |
+------+-------------+-------------------+------+----------------------------------------------------------+-----------------------+---------+--------+------+--------------------------+
5 rows in set (0.023 sec)

MariaDB [cells]>
15

Hi,

Sorry for the late answer, I was doing some investigation.

We have done some changes to the way the connection pool is handled in the v5. By default for the home edition, we set up 10 connections. It could be that your connection pool is too low - could you try changing it to see if that resolve the app appearing slow ? In the pydio.json, you can add a parameter maxConnection to the database connection url to increase the number of connections ?

"dsn": "mysql://root:secret@tcp(127.0.0.1:3308)/cells?parseTime=true&prefix={{.Meta.prefix}}&policies={{.Meta.policies}}&singular={{.Meta.singular}}&maxConnections=40"

You can check what makes sense for your setup but I’d recommend at least 40 for your setup (by default mariadb authorizes ~150 connections)

Can you let me know after you’ ve tried that change if the speed is better on the requests ?

Thanks,

Greg

16

Hello,

I have done the change, but did not see any visible effect on the performances. Right now, the “client” side of cells operates properly, while the “admin” side still has issues, and does not return the logs.

Below is a sar log :

Linux 5.4.0-216-generic (srv-intest02.actilis.net)      08/06/2026      _x86_64_        (2 CPU)


16:04:53      dev8-16     98,60    444,00    429,60      0,00      8,86      0,31      4,27     30,40
16:04:58      dev8-16     43,80    459,20     45,60      0,00     11,53      0,16      5,15     18,72
16:05:03      dev8-16     42,60    428,00     49,60      0,00     11,21      0,10      3,51     16,96
16:05:08      dev8-16     73,80    261,60     35,20      0,00      4,02      0,05      1,35     22,08
16:05:13      dev8-16    219,80   2756,80     60,40      0,00     12,82      0,47      3,04     46,00
16:05:18      dev8-16     98,60   1175,20    116,20      0,00     13,10      0,73      8,88     28,00
16:05:23      dev8-16    153,00   1066,40    124,80      0,00      7,79      0,50      4,66     59,52
16:05:28      dev8-16     65,60    287,20     88,00      0,00      5,72      0,39      6,75     18,32
16:05:33      dev8-16     36,00    288,00     36,00      0,00      9,00      0,09      3,42     14,24
16:05:38      dev8-16     30,40    146,40    183,20      0,00     10,84      0,13      5,97     17,92
16:05:43      dev8-16    335,40  41028,00     37,60      0,00    122,44      2,23      8,31     82,88
16:05:48      dev8-16    450,40  56109,60     42,40      0,00    124,67      2,73      7,69     99,04
16:05:53      dev8-16    456,20  55044,00    193,60      0,00    121,08      3,21      8,65     98,80
16:05:58      dev8-16    503,80  60437,60    105,60      0,00    120,17      2,50      6,43     99,04
16:06:03      dev8-16    466,60  53050,40    136,00      0,00    113,99      2,67      7,25     99,12
16:06:08      dev8-16    432,60  48108,80    125,60      0,00    111,50      2,69      7,83     99,52
16:06:13      dev8-16    364,00  37483,20    114,40      0,00    103,29      3,07     10,11     99,52
16:06:18      dev8-16    256,20  26140,80     36,00      0,00    102,17      1,87      9,13     99,60
16:06:23      dev8-16     37,20   2268,80     32,00      0,00     61,85      0,30      9,70     13,84
16:06:28      dev8-16     27,00    265,60     52,80      0,00     11,79      0,25     10,78     16,48
16:06:33      dev8-16     33,60    389,60     52,80      0,00     13,17      0,23      8,26     16,64
16:06:38      dev8-16     19,40    243,20     24,80      0,00     13,81      0,09      6,61     13,36
16:06:43      dev8-16    405,20  38406,40     59,20      0,00     94,93      2,48      7,71     82,96
16:06:48      dev8-16    476,20  58874,40     25,60      0,00    123,69      2,53      6,87     99,52
16:06:53      dev8-16    517,80  57491,20    115,20      0,00    111,25      2,60      6,54     99,68
16:06:58      dev8-16    476,60  54798,40     71,20      0,00    115,13      2,48      6,76     99,28
16:07:03      dev8-16    479,40  51499,20     86,40      0,00    107,60      2,91      7,65     99,28
16:07:08      dev8-16    451,20  47811,20     76,80      0,00    106,13      2,93      8,11     98,88
16:07:13      dev8-16    407,00  41457,60     87,20      0,00    102,08      2,81      8,54     99,60
16:07:18      dev8-16    272,60  26128,80    124,00      0,00     96,31      2,38     10,58     99,44
16:07:23      dev8-16    256,60  13842,40    120,90      0,00     54,42      1,36      6,87     62,96
16:07:28      dev8-16     32,60    524,00     47,20      0,00     17,52      0,15      6,04     14,32
16:07:33      dev8-16     37,00    320,00     52,80      0,00     10,08      0,13      4,88     17,44
16:07:38      dev8-16     34,60    330,40     44,00      0,00     10,82      0,14      5,48     15,52




Average:      dev8-16    237,98  22922,54     89,20      0,00     96,70      1,40      7,44     58,79

and here are the actions performed on cell from my browser :

  • 16:05:03 login
  • 16:05:33 logged in see the content of our main base folder
  • 16:06:18 click on “cell console” screen is instantly visible with white zone where the logs will show up
  • 16:06:38 alarms in 1st page are visible, clicked on logs
  • 16:07:23 logged out, back to login screen

Regards,

17

Thanks for the explanation. The sar logs seem to show the disk i/o getting saturated. And seeing that you’re saying that the logs page is triggering it, it might be related to the logs size.

Have you setup the document db to use mongodb or have you kept the standard installation ? If you’re not using mongodb, can you check the size of the bleve ?


export CELLS_WORKING_DIR=/var/cells

find $CELLS_WORKING_DIR/services/pydio.grpc.log -type f -exec ls -lh {} \;

Thanks,

Greg

18

Hello,

There are indeed quite a few logs. 14G of them in 1131 files.

There are 66 bleve folders each storing around 200MB.

Regarding the document db, I guess I am running on the default setting for that.

  • How can I purge these logs?
  • Is there a setting to automatically purge theses logs?
  • How can I check if I am on MongoDB?
  • Should I switch to MongoDB?

Regards,