SFTP for everyone?

Roadmap / Requests
1

Hello Pydio Team,

Would you agree to make SFTP functionality available in the community version of the Cells? Let me try to persuade you it’s a good idea :slight_smile:

I need a programmable way of interacting with the files stored on my cells server. I tried WebDAV at first, but I have directories with a lot of files, and at some point I started hitting timeout when trying to list the directory, as it doesn’t scale very well in WebDAV. I though I’d be able to use SFTP instead, but realized that it’s only available in Cells Enterprise.

At some point I discovered SFTPGo, and realized that with some customization I’ll be able to leverage cells’ REST and S3 APIs and make a SFTP “shim” using SFTPGo. Because SFTPGo is so customizable I can relatively easy use it to provide transparent support for the SFTP in cells with all the features (authentication, versioning, etc.)

This is what got me thinking - If I spend some time making this shim work and release it under AGPL (like SFTPGo), it will basically provide the Cells Enterprise feature “for free” for everyone, albeit with some installation/configuration burden added. Instead, I’m asking you to make the SFTP feature public, saving everyone some time and effort. Besides, unlike Flows, Audit, LDAP integration SFTP doesn’t seem like it’s strictly an Enterprise feature to me.

Please let me know what you think about it.

Sergey

2

Hi Sergey,
We will consider this request :slight_smile:
That said, have you considered using the cells-client ?
Charles

3

Hi Charles,
Yes, I was looking into cells-client and cells-sync, but I need to perform connections from Android and iOS, and I couldn’t get the mobile apps to do what I wanted (Android one plain didn’t work, and iOS one doesn’t have the sync feature). Besides, I already have a solution that can sync files using SFTP/WebDAV, but, as I’ve said before, WebDAV has issues with large dirs.
Please let me know what the decision is when it’s made.
Sergey

4

Frankly speaking, open source business is pretty hard, and we must keep some differentiators between home and enterprise versions. I’d prefer to understand why the android app is not working for you, and eventually add sync in the iOS one.
Regarding Android, we are currently deeply refactoring the app, and it could be interesting to have you included in the beta-testers if you are interested.
As for your “sftp shim”, i’m afraid it will add a big overhead to write the whole layer on top of rest APIs. And I’m not sure the “issues with large dirs” you see with WebDav would not happen in the same way.
Last but not least : no way to have your company pay for the enterprise version ? :wink:
-c

5

Frankly speaking, open source business is pretty hard, and we must keep some differentiators between home and enterprise versions.

I understand, and I totally support that. That’s why I’m trying to be open and frank with you here. I would also understand if you ask me to keep my SFTP shim private, just out of respect. I mean, I’m using the fruits of your hard work for free. But having said that, I just believe SFTP is kinda taking over the world as the cross-platform file sharing protocol, outpacing CIFS, WebDAV, FTP, etc. so I just think it’s a “must have” nowadays, and not an enterprise feature, like LDAP, quotas, which are essential for the enterprises, and mostly useless for anyone else.

Still, just to make it crystal clear. It’s your project, your business, and I’m in no position to tell you what to do. Whatever your decision will be - I’ll accept it.

I’d prefer to understand why the android app is not working for you

Sure, let me report the issue separately.

Regarding Android, we are currently deeply refactoring the app, and it could be interesting to have you included in the beta-testers if you are interested.

Oh yes I am, how do I apply?

As for your “sftp shim”, i’m afraid it will add a big overhead to write the whole layer on top of rest APIs.

This is very much a possibility, but my load is close to nonexistent, and WebDAV just doesn’t work for me, so I’ll be happy with whatever I can achieve.

I’m not sure the “issues with large dirs” you see with WebDav would not happen in the same way.

S3 interface of minio allows me to list dirs, so I was able to test it with SFTPGo, and it works just fine. The problem currently is with creation, especially empty dirs, which is the known minio issue/feature, depending on how you ask. I suspect fixing this problem was one of the reasons for creating the additional Cells REST API

Last but not least : no way to have your company pay for the enterprise version ?

Oh I would push for that, but I’m not a company, but merely an individual :slight_smile: . I use Cells for my own “data junkyard”, offloading photos/docs/etc. from my family’s devices into it. I’m trying to contribute o the project in other ways, like I’ve added FreeBSD support, and I’m looking into other ways of improving the codebase and paying with my free time. I understand that’s not exactly what pays your bills, but it is what it is :slight_smile:

Talking about FreeBSD - if any of your enterprise customers are interested in running Cells on FreeBSD, but you have trouble packaging it - let me know, I’d be glad to help.