Can´t upload large files (>2GB), Timeout after 60 seconds


#1

Pydio 8 Community
PHP 7.1
Apache 2.4
Ubuntu 18.04

I can upload files up to 1,75GB to my Pydio. That takes around 10-15min.
If I try to upload anything larger however, the upload stops everytime after 60 Seconds.

What I tried:
post_max_size 11G
upload_max_filesize 10G
memory_limit 12G
max_execution_time 0 (unlimited, also tried 15000)
max_input_time -1 (unlimited)
session.gc_maxlifetime 15000

After searching half a day I am out of Ideas. Can you give me a hint? Are there any other error logs, than the apache-php-log? (var/log/error.log)

Sincerly N1gh1ow1


#2

Hi

I’m currently struggling with a 50GB file.

We use an Nginx frontend with an Apache backend. On Apache we have the directive: Timeout 1200 (you probably don’t need that much).

On Nginx we have:
keepalive_timeout 1200;
client_body_timeout: 1200;

I would like to know the process of uploading a file. My 50GB file fails and I think it is a matter of where the file is temporarily stored while uploading.

Edited: I should add we can upload 5GB no problem.


#3

Hi cyberfarer,

how did you configure the Apache timeout?

Does your upload also stops after 1 minute with an error?

Sincerly N1gh1ow1

PS: Found it (/etc/apache2/apache2.conf), but it is allready set to 300, so that can’t be the cause to my problem)


#4

Update: Yay, progress (kinda)! Problem: I don´t know how.

I installed a 2nd pydio to another folder for testing and got the issue, that Pydio wont import the upload_size from the php.ini. (always states: file to big!)
So I changed permissions of php.ini to 775, changed the values of max_execution etc. back to a high value (was unlimited before), and deleted the *ser-Files in /data/cache.

E voilá, I could upload my 2GB file.

Unfortunately the limit is still there, bigger files (2,5 GB and larger) will still fail, but this time after ~2min.

Edit: now its like before, I am so frustrated right now


#5

For Apache:

<Directory /var/www/html>
	Options Indexes FollowSymLinks MultiViews
	DirectoryIndex index.php index.html
	AllowOverride all
	Require all granted
</Directory>

   Timeout 600
   LogLevel info

#6

BTW, did you set your tmp location?


#7

Hi cyberfarer,

thanks for your reply.

Yes I set the /tmp location. Created a directory at pydio-root. How can I test, that Pydio is using the directory?

EDIT: I set the /tmp in the bootstrap.conf and the Pydio-GUI

Sincerly
N1gh1ow1


#8

I gave up on the issue and tried a different approach. Used another uploader (pluploader) and could upload a 4 GB file just fine.

For more info see my other post: Enable PLupload

One thing though: I think the issue could have been related to chunking. Pluploader allows setting the chunk size. If you left this field empty, it will crash just like before, because it will just use the biggest chunksize possible. If you set it to 2mb however, it works like a charm. So maybe if one could enable chunking with the standard uploader, the issue might be resolved.

Sincerly N1gh1ow1


#9

Hi,

Make sure your system is 64bit (32 bit system can handle files < 2G)
and disk free (/tmp) should larger than file size