Login times out before upload completes


#1

I’m trying to upload a large (930MB) file, and am hitting multiple problems. I keep getting timeouts, but am willing to accept that perhaps it could be my server, or bandwidth, so under Settings > Uploaders > Multipart Uploads I reduced the “Queue Size” to 1, and the Part Size from 50MB down to 15MB.

This seemed to succeed a bit better,with 8 or 9 parts completing, but at that point I left it running and when I came back the system had timed out my login. When I logged back in, the in-progress upload wasn’t listed as running.

As a side effect, this left a file listed in the folder I was uploading to, with a size of “NaN B”. When I try to delete this, nothing happens.

Am I missing something obvious?


#2

Hi,
did you modify the ulimit to ulimit -n 8192 and also i would like to know how much time does the upload take?


#3

Thanks for the quick response - where do I do that modification?


#4

just before starting cells, you must put this command in the cli .
If it’s a docker container then it could be another issue.


#5

It’s definitely in a docker container, but I’m happy to try the CLI - is it just a standard parameter?


#6

Hi,
lets, confirm some settings and understand your case (so that i can get all the elements to reproduce if needed), first do you know the cells-version, second when you upload and have the error could you take a look at the logs, (you can docker logs -f <cells-container>).


#7

While I’m in setup mode I’m currently using the docker tag “latest”. Digging to find the version in the web interface, weirdly this doesn’t show up in the “About” screen, and I eventually found it in the “Upgrade” screen. Any chance it could be added to the About screen?
Anyway, answering your question, I’m on 1.2.5.

For the logs: I don’t currently have any Pydio logs showing, but I’m planning on adding PYDIO_LOGS_LEVEL env variable when I restart next.

I do see logs via Portainer, which I think are the same as what you’re asking for, right? Here’s what I see:

2019-01-30T14:55:25.522Z	ERROR	pydio.gateway.data	PutObjectPart has failed	{"error": "Put http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=4&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6: Connection closed by foreign host http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=4&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6. Retry again."}
2019-01-30T14:55:26.030Z	ERROR	pydio.gateway.data	PutObjectPart has failed	{"error": "Put http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=5&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6: context canceled"}
2019-01-30T14:55:26.252Z	ERROR	pydio.gateway.data	PutObjectPart has failed	{"error": "Put http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=6&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6: context canceled"}
2019-01-30T14:56:07.560Z	ERROR	cannot init oidc provider	{"error": "Get https://files.example.com/auth/dex/.well-known/openid-configuration: dial tcp 71.136.153.141:443: connect: connection timed out"}
2019-01-30T14:56:07.560Z	ERROR	pydio.gateway.websocket	invalid jwt received from websocket connection
2019-01-30T14:57:09.000Z	ERROR	cannot init oidc provider	{"error": "Get https://files.example.com/auth/dex/.well-known/openid-configuration: dial tcp 71.136.153.141:443: connect: connection timed out"}
2019-01-30T14:57:09.000Z	ERROR	pydio.gateway.websocket	invalid jwt received from websocket connection
2019-01-30T14:57:14.180Z	ERROR	cannot init oidc provider	{"error": "oidc: issuer did not match the issuer returned by provider, expected \"https://files.example.com/auth/dex\" got \"http://cells:8080/auth/dex\""}
2019-01-30T14:57:14.180Z	ERROR	pydio.gateway.websocket	invalid jwt received from websocket connection
2019-01-30T14:57:25.768Z	ERROR	pydio.gateway.data	PutObjectPart has failed	{"error": "Put http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=4&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6: Connection closed by foreign host http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=4&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6. Retry again."}
2019-01-30T14:57:26.274Z	ERROR	pydio.gateway.data	PutObjectPart has failed	{"error": "Put http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=5&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6: Connection closed by foreign host http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=5&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6. Retry again."}
2019-01-30T14:57:26.472Z	ERROR	pydio.gateway.data	PutObjectPart has failed	{"error": "Put http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=6&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6: Connection closed by foreign host http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=6&uploadId=6257d1a1-dc52-4aca-a0ef-ab2b59893ee6. Retry again."}

Thanks for any help! I have to say, Cells is looking really great, very light and modular. If I can solve my few remaining issues, I’ll be really happy.


#8

And, after the login timeout (sorry, forgot to include these):

|2019-01-30T15:19:25.321Z|INFO|pydio.web.auth|Login|{"identity": {"UserID":"d5d64569-09f3-4918-8f3f-2748f274a5f2","Username":"geoff","Email":"","EmailVerified":true,"Groups":[],"AuthSource":"pydioapi","DisplayName":"","Profile":"admin","Roles":["ROOT_GROUP","ADMINS","ADMINS","d5d64569-09f3-4918-8f3f-2748f274a5f2"],"GroupPath":"/","ConnectorData":null}}|
|---|---|---|---|---|
|2019-01-30T15:19:35.119Z|ERROR|pydio.gateway.data|PutObjectPart has failed|{"error": "Put http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=3&uploadId=dbf2e366-6925-49df-9c2a-5287214f5a83: context canceled"}|
|2019-01-30T15:19:35.120Z|ERROR|pydio.gateway.data|PutObjectPart has failed|{"error": "Put http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=1&uploadId=dbf2e366-6925-49df-9c2a-5287214f5a83: Connection closed by foreign host http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=1&uploadId=dbf2e366-6925-49df-9c2a-5287214f5a83. Retry again."}|
|2019-01-30T15:19:35.120Z|ERROR|pydio.gateway.data|PutObjectPart has failed|{"error": "Put http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=2&uploadId=dbf2e366-6925-49df-9c2a-5287214f5a83: Connection closed by foreign host http://10.0.1.8:34241/pydiods1/Test.File%20For%20Uppload%20To%20Server.mp4?partNumber=2&uploadId=dbf2e366-6925-49df-9c2a-5287214f5a83. Retry again."}|
|2019-01-30T15:19:35.133Z|INFO|pydio.gateway.data|Received MultipartAbort - Clean temporary node:|{"Node": "Uuid:\"926e7cd2-6c9f-442c-b270-49b6cea7a49a\" Path:\"pydiods1/Test.File For Uppload To Server.mp4\" Type:LEAF MTime:1548861092 Etag:\"temporary\" MetaStore:<key:\"name\" value:\"\\\"Test.File For Uppload To Server.mp4\\\"\" > MetaStore:<key:\"pydio:meta-data-source-name\" value:\"\\\"pydiods1\\\"\" > MetaStore:<key:\"pydio:meta-data-source-path\" value:\"\\\"Test.File For Uppload To Server.mp4\\\"\" > "}|

Let me know if it would help you to have a login to the Cells/Portainer site.


#9

Would you mind telling me if you are using an public address to access your cells?
also do you use some kind of proxy?

could you check if the container ip matches the ip of your datasource,
you can find it here,


#10

Yes, I’m using a public address, with Traefik doing a reverse Proxy. If it helps I can PM you a login to Portainer to view the logs, and a login to Cells?

Everything else seems to work fine except for the two issues I raised here and here, could one of those be related?


#11

Hi,
did you count the time elapsing until it timeouts, is it 10 mins?

edit: i fixed a typo thats why my sentence was not making sense


#12

Hi Zayn - sorry, I din’t count it carefully, I can do this again and see if I can measure it correctly.