WebDAV issue which is difficult to replicate

Hello all,

When I first installed Cells, I’ve used the ‘standard’ configuration using HTTP (no SSL) and port 8080 and accessing it via the server’s IP address (no docker). WebDAV worked flawlessly, as expected. So I know that ‘it works’, at least with one specific set of configurations.

Since the first installation, a lot has been changed. Access is now made through a proper fully qualified domain name (FQDN), for which my customer has a valid SSL certificate (allegedly signed directly by GlobalSign, although I cannot confirm that), so I moved to full HTTPS support, as well as using HTTP/2 (supported by the embedded Caddy server), and switched to port 8443.

Cells certainly works on a web browser

I’m using a Mac to connect to Cells, but when trying to access https://my.fqdn.tld:8443/dav/ I get on the logs [slightly truncated, sorry about that]:

2019-05-23T23:51:27.118+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace ._. is not accessible\",\"status\":\"F
2019-05-23T23:51:27.213+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace ._. is not accessible\",\"status\":\"F
2019-05-23T23:51:27.262+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace .Spotlight-V100 is not accessible\",\"
2019-05-23T23:51:27.307+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace .metadata_never_index is not accessibl
2019-05-23T23:51:27.351+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace .metadata_never_index_unless_rootfs is
2019-05-23T23:51:27.395+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace .localized is not accessible\",\"statu
2019-05-23T23:51:27.440+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace .DS_Store is not accessible\",\"status
2019-05-23T23:51:27.493+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace .hidden is not accessible\",\"status\"
2019-05-23T23:51:27.597+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace .DS_Store is not accessible\",\"status
2019-05-23T23:51:27.692+0100    ERROR    pydio.gateway.dav        ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace .DS_Store is not accessible\",\"status

If you’re a Mac user, these items ought to be familiar to you — they are automatically generated by macOS and are hidden (it’s what macOS uses to keep track of the visual elements on the desktop or Finder). But no such items exist on the workspace — I have double-checked them, running locate to see if I hadn’t missed any, even resorting to resource-intensive find… these items are simply not on the remote server, although I’m pretty sure they will be on all my local folders. The only explanation that crossed my mind is that somehow, when using WebDAV, Cells is trying to sync both ways, and when looking at the local mount point (/Volume/dav/ on a Mac), it somehow finds these files and gets ‘confused’ by them. I’m pretty sure none of that ever happens on Pydio 8! (Also, as said, this did work exactly as expected at least once…)

I’ve used both the Finder and Cyberduck — both behave exactly in the same way (same backend errors, too), even though I suspect that Cyberduck is removing the last / from the connection string (but I cannot confirm this).

So I’ve created a brand new folder called Gatos under my personal files and tried to access https://my.fqdn.tld:8443/dav/ws-personal-files/Gatos/ to see what happened:

|- DAV END      {"method": "HEAD", "path": "/dav/ws-personal-files/Gatos/", "error": "file does not exist"}
ReadNode Error  {"error": "{\"id\":\"workspace.not.accessible\",\"code\":403,\"detail\":\"Workspace ws-personal-files is not
|- DAV END      {"method": "PROPFIND", "path": "/dav/ws-personal-files/Gatos/", "error": "file does not exist"}

Ok, this is even stranger, since this was freshly created from the Cells interface. It works flawlessly, no permission problems when uploading/moving them around/deleting them (all done in the browser interface, of course), I can share these files or place them in a cell, and so forth.

I’ve also noticed that Cells is running MinIO — I originally thought that this would only be enabled if one connected to a remote S3 bucket. But apparently Cells ‘bucketizes’ everything locally, i.e. everything inside Cells ‘becomes’ a S3 bucket (or several ones), which, in turn, allows Cells to export a S3-compatible API to the external world — that’s kind of neat! However, as far as I can understand what is happening beneath the surface, this requires Cells/MinIO to add a lot of entries on a separate structure stored under .minio.sys, and although I haven’t been very thorough, I think that some directories have more entries under .minio.sys than under the ‘real’ folder, or, in other words, there might be a synchronization issue here (or maybe I have no idea about what I’m saying!).

Does any of you have any clue about what’s wrong? Clearly I messed up the configuration at some point — but where? (Since Cells continues to work exactly as before inside the Web browser). Unfortunately, I cannot remember precisely what changes I’ve made — my only hope is that this is something incredibly obvious (‘did you uncheck the checkbox on page XXX by mistake?’) which eluded me…

Any hints or ideas are welcome! Thanks in advance…