AWS S3 Driver issue - can't Create Folder, or upload small files

#1

I have installed a new version of Pydio, then added a Workplace using the S3 AWS Driver, to point to an AWS S3 Bucket. I used an IAM role with full EC2Admin permissions.

The S3 Driver has an issue when I try:

  • New, New Folder - Access Denied error
  • New, New empty file - Works
  • Upload, Pick Folder, small folder or file (eg. 1kb) - Access Denied error
  • Upload Pick Folder, folder with large files (eg. 30mb) - Works

Here is the error that pops up:
StreamWrapper.php error l.746 message=Error executing “PutObject” on “https://pydio-test.s3.southeast-2.amazonaws.com/Test-Data/src/”; AWS HTTP error: Client error response [url] https://pydio-test.s3.southeast-2.amazonaws.com/Test-Data/src/ [status code] 403 [reason phrase] Forbidden AccessDenied (client): Access Denied - <?xml version="1.0" encoding="UTF-8"?> AccessDeniedAccess Denied08B29A4CA12ECUHHDavg9JkH9n1c0GVy6fSFCpnNiFI9wCwP864KtxbuaaprcW+rLhEwDynVFKwg/GdvI=

Ubuntu 18.04
Pydio 8.2.3 2019-03-12
Apache 2.4.29
Php 7.2

#2

Hello @martinw,

Could you tell me which fields you filled(in the workspace settings) and with which information (make sure to hide your Secret and Key).

#3

Hi Zayn,

I just put in the minimum information, and left the rest as default.

Here is the information requested:
WorkSpace settings
General Options

  • Label: prd-pydio-test
  • Alias: prd-pydio-test
    Driver: S3 (Amazon Web Service)
  • Key: sldjfdskl
  • Secret Key: slkdjfsdklj
  • Region: South-East (Sydney)
  • Bucket: prd-pydio-test
  • Inner Path: /
  • The Rest: defaults
#4

Hello,

I am having the same issue. I am running pydio version 8.2.1.

I used an Access key that has permissions for listing, reading and writing. The workspace will read the contents of the bucket but when I try to create a folder or upload a file I receive an error with the PutObject…

image
image.png709×227 13.9 KB

I am able to use the same access key with a different application and it is able to read and write to the same bucket.

#5

Hello @rmeske,

This issue is hard to pinpoint because it seems to happen on random cases,

Could you try by putting your aws storage url (something of the sort), in the Storage URL field?

#6

Hello @zayn,

When I add the storage url I get a different error.

image
image.png719×114 8.27 KB

#7

Hello @zayn,

I went through the code and see that if the file being uploaded is >5MB it uses a multipart upload otherwise it uses PutObject. So I did some testing.

  • New folder - fails
  • New Empty file - fails
  • Upload a file smaller than 5MB - gives the same error but does upload the file successfully.
  • Upload a file larger than 5MB - Success
  • Upload an empty folder - fails
  • Upload a folder with less than 5MB of files - gives the same error but does create the folder and uploads the files successfully.
  • Upload a folder with more than 5MB of files - gives the same error but does create the folder and uploads the files successfully.

So there seems to be two different cases:

  1. Using Pydio to create a folder or empty file gives the error “Length Required” and does not create the folder or file.
  2. Uploading files smaller than 5MB or uploading a folder containing files will give the error “Length Required” but the file(s) are actually uploaded and folders are actually created.

I know Pydio is nearing end of life, but our use case for Pydio is as a front end to a EDMS that provides access to specific folders for external users. Pydio access the EDMS using the ftp protocol which does not look like it is supported in Pydio Cells.

So it would be great if this could be updated so we can use it for other uses cases.

Best regards,
Ron

#8

Any update on this issue?