Pydio Cells Storage on another Server

I am trying to create a new install of Pydio Cells version in order to replace Pydio 8 installation. I have built a new server (FreeBSD bhyve VM with Debian 9.4) since I was previously running on FreeBSD and created an install. I am trying to setup storage but can’t get anywhere. I can create new storage data store to a folder on the system and then new workspaces within it without trouble. However I am trying to mount storage from a cifs mounted file system that links to my windows workstation where the vast majority of my data actually resides.
I can create the storage datastore, but then when I go to add a workspace to the storage it just starts churning and eventually times out looking at the logs it says it was unable to locate the the data store. and there is a timeout connecting to the rest frontend.

Ts : 1527020866
Level : error
Logger : pydio.rest.frontend
Msg : message=[404] Error connecting to the API (https://pydio.dweimer.net/a/tree/admin/list)
UserName : admin
UserUuid : d26d5c32-5a0f-11e8-b9ac-589cfc02bb17
GroupPath : /
RemoteAddress : 10.20.130.5
UserAgent : Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0
HttpProtocol : HTTP/1.1
SpanUuid : 969884e1-5dfe-11e8-a8ee-589cfc02bb17
SpanParentUuid : 27c4ed44-79b3-71e0-6dbe-ef524b36778d
SpanRootUuid : 27c4ed44-79b3-71e0-6dbe-ef524b36778d
HasRoot : true

Has anyone else tried this? Without the ability to connect to my windows workstation, or my FreeBSD system (haven’t tried this one yet, as that data isn’t currently shared) where my web pages are stored. I can access the data on the Linux system itself just fine as the cells user account.

Hi,
sorry in advance if i didn’t understand it, but you did try to mount your storage to the server that is handling cells, and then when you create a datasource you browse the local files looking for this mounted storage folder and then you should be able to store into it.

@dweimer I’m having the exact same issue. Did you ever find a solution? Thanks

Nope, I haven’t been able to get past it, rebuilt entire server, reinstalled debian linux and everythign from ground up. still have same problem. System can browse files, but I am unable to access it through Pydio. I can browse to the folder on the storage section but when I try to add the workspace it returns an error. Even though its browse able in the drop down list, the backend api doesn’t seem to have access to it.

2018-06-14T09:12:16.857-0500 ERROR Could not find DataSource workstation {“currentSources”: [“cells”, “personal”, “pydiods1”]}
2018-06-14T09:12:16.880-0500 ERROR pydio.rest.frontend message=[404] Error connecting to the API (https://pydio.dweimer.net:9443/a/tree/admin/list) {“FrontIp”: “10.20.130.5”, “UserId”: “dweimer”, “WorkspaceId”: “settings”, “Source”: “ApiClient.php l.287”, “Nodes”: []}


Think I found something, thought maybe a restart of the server after adding the workspace would help. But I did see this error, its refusing to cross the mount point.

time=“2018-06-14T09:28:45-05:00” level=fatal msg=“Invalid command line arguments server=‘:9001’, args=[/smbfs]” cause=“Cross-device mounts detected on path (/smbfs) at following locations [/smbfs/workstation/Downloads]. Export path should not have any sub-mounts, refusing to start.” source="[server-main.go:96:github.com/pydio/cells/vendor/github.com/pydio/minio-srv/cmd.serverHandleCmdArgs()]"

Hi,
on your previous screenshot i see that downloads in highlighted in red and that’s because you cannot use Caps, try to renamed it to /downloads and you should be able to select it from the interface.

I did rename it at somepoint after seeing a different error, it currently all lowercase and still fails to start showing the mountpoint error.

2018-06-18T07:51:20.099-0500 INFO pydio.grpc.data.objects.local2 Starting objects service local2
time=“2018-06-18T07:51:20-05:00” level=fatal msg=“Invalid command line arguments server=‘:9001’, args=[/smbfs]” cause=“Cross-device mounts detected on path (/smbfs) at following locations [/smbfs/workstation/downloads]. Export path should not have any sub-mounts, refusing to start.” source="[server-main.go:96:github.com/pydio/cells/vendor/github.com/pydio/minio-srv/cmd.serverHandleCmdArgs()]"

Hi Dean,
this is currently a limitation of Cells - We are starting a minio server to serve the file using http protocol, and minio is limiting this on mounted folders. We are working on that for direct smb/cifs mount. Or you can try using a docker volume with tool : http://netshare.containx.io/ and mount your smb share directly inside docker. I tested that recently and it was working (minio sees it as an standard folder inside the docker).
Charles

Are minio datasources yet working with cells? I have an external-to-cells minio server but cannot seem to figure out the correct parameters to input for S3 in the “Datasource” screen. In reading the official documentation I also cannot figure it out. When I input what I think should be the correct parameters for my minio server, the result is: “Objects service down”. However, when I go to Settings->Services, I only see green lights for both objects and for the minio Storage I created.

1 Like

hi @v60fan
Could you maybe post the config you use for this datasource? Switching to S3 datasource type with the correct endpoint url and api key/secret should work with an external minio server, indeed.
Charles

Sure. Is this what you mean?

DataSource identifier: minio (I think this is just a label, so doesn’t matter too much)
Enabled: switched to on
Internal Port: 9001 (left as default)

Storage Type: Remote Object Storage (S3)
Bucket Name: cells (I created this bucket in the top level of my minio server)
S3 Api Key: [the same one I use in my config.json for minio] (I’m certain this is correct)
S3 Api Secret Key: [the same one I use in my config.json for minio] (I’m certain this is correct)
Internal path: [left blank] (I’m not sure if I’m supposed to fill anything here or not)
Custom endpoint: https://s3.mydomain.com (set to a real URL, and it’s the base URL for the minio service)

1 Like

Humm, params seem ok.
Can you have a close look at the log when you start cells ? Do you see errors regarding this datasource ?
Charles

Hi @charles so that I could follow the log, I have restarted the cells container (I’m running in docker and I’m also using a traefik reverse proxy for SSL termination).

I do see a couple types of error:

First is:
level=fatal msg="Unable to initialize gateway layer" cause="Unable to initialize event notification. Get https://s3.mydomain.com/: x509: failed to load system roots and no roots provided" source="[gateway-main.go:357:github.com/pydio/cells/vendor/github.com/pydio/minio-srv/cmd.gatewayMain()]"

And then I also see:
ERROR pydio.grpc.data.sync.(.+) Could not run {"error": "objects not reachable"}

1 Like

is the external minio served on self-signed ?

No, it’s valid LetsEncrypt SSL

@charles are there any news on this limitation?
Without the possibility to add nfs/smb remote storage pydio is pretty much useless to me.

Until it is possible, could you please mention this in the documentation?

1 Like

@v60fan at the end of the day, did you get it?

I was trying to configure Minio with Pydio

But what I get is objects down

No, I wasn’t able to overcome the issue I had above. I ended up giving up.

I have a similar issue of the “cross-devices mount” detailed here: Error while listing on creation or resync of a DataSource

I succeed to get past it but was block on the next step, that is building the index (the storage exist, the workspace as well, but I cannot list anything inside for some obscure reason.)
I’m waiting for support as well, it seems pretty basic use case so I hope we’ll find a solution (it was working as is in pydio server from version 5 to 8).

wait&see