I can't redirect my local IP to my Cloudflare DDNS

[Father]

For first post, please answer the questions below!

Describe your issue in detail

I have not been able to redirect my Pydio cell, installed in Docker, through a tunnel created with Cloudflare, I get error 502.

What version of Cells are you using?

lastest available

What is the server OS? Database name/version? Browser name or mobile device description (if issue appears client-side)?

Docker, mysql, chrome

What steps have you taken to resolve this issue already?

Help me pls!

[bsinou]

502 (bad gateway) errors are usually network configuration problems, - and I must admit usually nightmare to debug and fix in complex setups…

That said, we would need much more details and error log to be able to help you.

• How is Cells configured (cells configure sites)?
• Can you access Cells directly?
• How is cloudflare configured? Do you have any log on this side?

And a final note, like with e.g. Azure WAF, using Cells behind Cloudflare introduces issues when uploading very large files that are not trivial to solve - keep this in mind while you are designing and testing your solution before you go live.

Happy File Sharing !

[GwynethLlewelyn]

Hi @Father, and welcome to the world of Pydio Cells

To add to all the above, it would help if you could clarify exactly what you mean with some of your terminology.

• Cloudflare DDNS: Usually, “DDNS” means Dynamic DNS, a way for people without a permanent IP address to have a fast DNS resolver swiftly change the hostname (www.example.com) to an IPv4 and/or IPv6 address.
  This requires some software support on your side to inform the remote DNS provider that your IP address has just changed.
  If you’re using Cells at a home/office environment where the IP address is prone to change, then, aye, it’s likely you need to use such a service. And it’s also possible that your brand of router (often installed by the utility service themselves — cable, fibre, etc.) already has support for a few popular DDNS providers. Usually, Cloudflare is not one of those, so you will need to run something on your server instead.
  Among its trillions of free services, Cloudflare also provides Dynamic DNS. They even have a webpage describing what to configure and how. They recommend running ddclient, a well-known, decades-old, but strong and robust solution.
  If that’s what you’re using, the first step is to check what IP address is currently being reported by your service (using some free web services such as https://whatismyipaddress.com/) and make sure it matches exactly what Cloudflare currently thinks that your IP address is (you can also use a remote service for that; I’m fond of DNS Lookup Tool - DNS Tools - MxToolbox but just search for “remote DNS tool” on Google, you’ll find tons).
  If the two match exactly, well, then at least that step is complete
• Cloudflare Tunnel: Now this is a completely different approach, and one that has nothing to do with Dynamic DNS. Instead, if you’re using this method, you’re very likely using Cloudflare’s Zero Trust service. This means that your server is running an instance of cloudflared, a small daemon which establishes an encrypted ‘reverse tunnel’ from Cloudflare’s edge servers to your own machine. Again, if that’s what you’re using, Cloudflare also has a great web page with all the resources you need to set things up. The main advantage of Cloudflare Tunnel is that it essentially encrypts all traffic inbound to your server, and you choose exactly what you allow to go through. It works pretty much like a VPN, with the difference that clients connecting through your tunnel do not need any additional software or configuration: they just point their browsers (or Cells Clients) to the Cloudflare address that you give them (with optional extra authentication) and everything else is handled between Cloudflare’s edge servers automatically.
• You really need to specify your server OS. Every system requires different configurations, but the major differences are for Windows vs. Unix (and, among those, macOS is the notoriously most annoying variant). While you can safely pack Cells inside a Docker container, the way you communicate with Cloudflare is quite different!
• Does everything work locally? Before attempting any complex multi-re-directional, multi-layer setup, which is always complex, messy, and quite hard to debug, please let us know if you can connect from inside your network successfully. Are you using only private IP addresses (i.e., such as 192.168.0.1, which is not routable over the Internet) when doing those tests? Keep in mind that Docker also introduces another layer of redirection, i.e., the IP address to which Cells connects inside Docker will not be the same that your other computers on the internal network will see. If that step is ruled out, and everything works well internally, that step can be considered done
• How does your outbound router forward packets to the ‘outside world’? Again, I’m assuming a home/office scenario, where your internal network is all on private IP addresses, and the router’s job is also to provide NAT services and correct port translations between outside connections (to the real IP address currently assigned to the router) and the internal network. And here the mess starts to become seriously complex. Now we need to have an idea of what brand and model of router you’ve got, and how port translation has been configured. Each brand, of course, has a different interface and/or programming language, and what works on one system may not work on another — there might be differences even for models from the same brand!
  In some cases, your Internet Service Provider might not even allow you to redirect/forward ports from the outside to the inside. Or you might be in an office where the network administrator only allows outbound HTTPS connections, through a (transparent) Web proxy — all in the name of security, of course. Not to mention that the cheapest routers — and service providers are fond of those — will not allow anyone to tinker with NAT or port forwarding, and such options may be locked out of the router, or (deliberately!) not even implemented.
  And, last but not least, some (large) providers may not even assign your router a real IP address at all. Instead, you just get a private network address — from your provider. It’s up to them to do NAT on their routers, and, if that’s the case, you’re out of luck, they won’t allow you any access to their routers at all. Everything will still work flawlessly for outbound HTTPS calls, of course (and, these days, almost everything can be piped through the web’s protocols — even voice and video), but that’s pretty much what you can do: you’re not supposed to do anything else on a home/office setup, after all.
  In other words: although you might be able to use a Cloudflare Tunnel to circumvent some of these limitations — namely, the router and whatever firewalls are set up there, outside your control — there still must be a way for traffic to flow inwards. And that, indeed, is the crux of the problem.
  Without understanding better how your network topology looks like, it might be next-to-impossible to help you out; unfortunately, there is no one-size-fits-all solution.

Whew. That’s a start, I’d say. Since Cloudflare does show a “502” error page, and being an eternal optimist, I would imagine that most of your issues are already fixed and that, at least, Cloudflare can communicate with some IP address. Let’s assume the best of scenarios: Cloudflare is able to reach the machine where Pydio Cells is running, inside its Docker container. That machine is actually running some sort of Web server (it might be Apache, Nginx, Caddy, or something even simpler). That machine, in turn, needs to contact Cells inside the Docker container — but, due to some configuration issue, it fails to do so, and returns a Gateway Error.

In that case, things would be much more narrowed down, and all IP translation issues, as well as port forwarding, would only be limited to your machine and the Docker container. That should be relatively simple to figure out.

Let’s assume the following scenario: all machines in your network can point to the machine running Docker + Cells, using its IP address (or internal name, if you are running something like a small DNS service inside your network), and all of them work fine. That means that at least part of the configuration must be correct: at the very least, the internal machines are contacting whatever is running on the Cells machine which is providing reverse-proxy services to the Docker container. This, in turn, would mean that the problem would be either on the cloudflared setup, or at the router level, which is not translating correctly IP addresses from the outside to internal ones.

Scenario #2: All your internal network gets exactly the same error as those coming from the outside via Cloudflare. Then we have the opposite issue: everything is correctly configured until it reaches your machine running Cells, and the most likely scenario (that doesn’t mean it’s the only possible one!) is a misconfiguration of the reverse-proxy and the Docker container.

I can imagine a few more exotic scenarios — I’m quite creative in coming out with them, because I’ve seen so many different possibilities… — but my guess is that it will be one of the two above.

What would help is to know what kind of reverse-proxy mechanism is installed on your machine to provide access to the applications running inside the Docker container. If it’s a ‘real’ web server, that means you will have a way of serving a static HTML page from your machine. If so, that’s the first test you should do: can you see that HTML page from inside your network? What about from the outside? If nobody can see the HTML page, then it’s the reverse-proxy that is badly configured. If everybody can see the HTML page (even from the outside!), then it’s the Docker setup that has issues. If your internal network can see both the HTML page and the Cells installation, then, good news, your machine is correctly configured; if, additionally, everybody from the outside can also see the HTML page but not the Cells installation, then, good news as well, your connection routing is correct, and the problem is only with how Pydio Cells is configured (very likely, it’s just figuring out what the correct addresses are for internal IP vs. external domain name).

Also — remember, dealing with HTTP/S for Pydio Cells is just half the story; Cells also needs gRPC to be properly configured, or else ‘nothing works’. And if you thought that this would be easy — fix the HTTP pipeline, and everything will work! — I’m sorry to disappoint you: it’s a bit more complicated than that.

The best of news is that it’s highly likely that there is a solution for your case. It can just take a wee bit longer to figure out. My own ‘production’ scenario ‘only’ involves three layers: Cloudflare (of course!), a reverse-web proxy (Nginx, in my case), and Pydio Cells, running outside Docker (one level less of redirections to worry about). At each of those three levels, there is a firewall involved — on the server, on the network, and on Cloudflare. Believe me, getting ‘only‘ three layers to work together was enough of a nightmare for me — and you’re adding at least two more! I don’t envy your job…