Upload restart for big files

Pydio Cells
1

I use last version of Pydio Cells in docker with an nginx as reverse proxy.

The S3 endpoint is provider by wasabi.com (Amsterdam Istance)

I have an issue with the upload of big file : several Giga.

The upload restart several times when it reach 15 %.
Enable / disable buffers in nginx doesn’t change the situation.

I set ulimits in the container of Pydio Cells.

I try to tune the upload size too.

docker-compose.xml for testing

version: ‘3.4’

services:

web:
image: “nginx:1.15”
ports:
- “80:80”
command: “/bin/sh -c ‘while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g “daemon off;”’”
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf

# Cells image with two named volumes for the static and for the data

cells:
image: pydio/cells:latest
restart: always
volumes: [“static:/root/.config/pydio/cells/static/pydio”, “data:/root/.config/pydio/cells/data”]
environment:
- CELLS_BIND=cells:8080
- CELLS_EXTERNAL=##.##.##.##:80 (public IP)
- CELLS_NO_SSL=1
ulimits:
nofile:
soft: 80000
hard: 80000

# MySQL image with a default database cells and a dedicated user pydio

mysql:
image: mysql:5.7
restart: always
environment:
MYSQL_ROOT_PASSWORD: ####
MYSQL_DATABASE: cells
MYSQL_USER: pydio
MYSQL_PASSWORD: ####
command: [mysqld, --character-set-server=utf8mb4, --collation-server=utf8mb4_unicode_ci]
ports: [“3306:3306”]

volumes:
static: {}
data: {}

2

Hi @SebastienID, do you see any reproducible behaviour linked to upload time (not percentage), like it always breaks after 10mn or something similar?

3

Hello @SebastienID,

Could you tell me which Pydio Version are you on, you can find it in the web ui under the Upgrade menu.

Also could you share with me your Nginx config, maybe you are getting timed out by nginx.

4

Hi Charles,

Below is the The nginx conf :

events {
worker_connections 1024;
multi_accept on;
}

http {
include       /etc/nginx/mime.types;
default_type  application/octet-stream;
server_tokens off;




server {
    listen 80;
    client_max_body_size 200M;
   
    client_body_timeout 30;
	client_header_timeout 10;
	keepalive_timeout 30;
	send_timeout 60;
	keepalive_requests 100;

    location / {
           proxy_buffering on;

            proxy_pass http://cells:8080$request_uri;
            #proxy_pass_request_headers on;
            #proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
    }
   
   
    location /io {
           proxy_buffering on;
		   proxy_buffer_size 1k;
		   proxy_buffers 12 4k;

            proxy_pass http://cells:8080$request_uri;
            #proxy_pass_request_headers on;
            #proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
    }

    location /ws {
            proxy_buffering off;
            proxy_pass [http://cells:8080](http://cells:8080);
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_read_timeout 86400;
    }

}

}

Best regards

Sébastien

Le lun. 9 sept. 2019 à 18:16, Charles via Pydio Forums pydio@discoursemail.com a écrit :