Extremely High Data Usage

Hey all,

Back again with another interesting predicament. Long story short, ever since installing and getting Cells Sync working with my remote system, I have had little issue… until I receive an email from my ISP that I had exceeded my monthly data limit. Sure enough, I looked at my network usage, and Cells Sync had been responsible for a total of 268.83GB of data, which struck me as odd seeing as the current size of my synchronized files amounts to roughly 20GB

I have also observed Cells Sync appears to restart itself quite often, judging by the number of times I have found my “Hidden Icons” tray overflowing with dead instances of Cells Sync (that promptly vanish when hovered over.) Could there maybe be some sort of re-sync being constantly performed that would cause Cells Sync to amass so much data usage? I know I have observed Cells Sync often “analyzing” nodes, but no indication of it actually updating or redownloading/uploading data…

I’m running Windows 10 Enterprise with the latest version of Cells Sync 0.9.2. My server is running on Ubuntu 20.04 with cells 2.2.6

Anyone have any advice or ideas as to what may be going on here?

Hi Marc,
Interesting indeed. Unless that data is constantly resynced, I don’t imagine the “metadata” transferred back and forth eating so much network data. Can you have a look at cells-sync log files ? If data is uploaded/download, that should appear here.
Charles

Hi Charles

Yeah, I thought so too! The metadata shouldn’t take much data space at all.

Looks like my cells-sync log files are exceptionally beefy. (about 50mb) But I was able to find this from a scan.

2021-05-22T22:45:58.456-0700	[34mINFO[0m	[35msync-task[0m	Diff Stats	{"s": {"EndpointLeft":"https://example.net/source","EndpointRight":"snapshot://left","conflicts":0,"missingLeft":0,"missingRight":0}}
2021-05-22T22:45:58.456-0700	[34mINFO[0m	[35msync-task[0m	Finished analyzing nodes	{"i": 3909}
2021-05-22T22:45:58.456-0700	[34mINFO[0m	[35msync-task[0m	Sending unidirectional patch	{"patch": {"Source":"https://example.net/source","Target":"snapshot://left","Type":"TreePatch"}}
2021-05-22T22:45:59.970-0700	[31mERROR[0m	[32mhttp-server[0m	Cannot read message x_+ԄퟹûĄݰ憇溊2021-05-22T22:45:59.970-0700	[34mINFO[0m	[36msystray[0m	2021-05-22T22:45:59.970-0700	[34mINFO[0m	[36msystray[0m	Got Error: websocket: close 1002 (protocol error): unexpected reserved bits 0x60
2021-05-22T22:45:59.970-0700	[34mINFO[0m	[36msystray[0m	2021-05-22T22:45:59.970-0700	[34mINFO[0m	[36msystray[0m	Closing bindConn listener
2021-05-22T22:45:59.972-0700	[31mERROR[0m	[36msystray[0m	panic: concurrent write to websocket connection

Lots of fun characters in there… from what I can see it looks like it is in fact finding there’s no difference between local and remote. (Based on: Diff Stats {"s": {"EndpointLeft":"https://example.net/source","EndpointRight":"snapshot://left","conflicts":0,"missingLeft":0,"missingRight":0}} )

And yet there’s clearly some kind of error being fired. Could this be resulting in a full-sync every time?

Cheers

Hey folks, looking for a bump on this one… I’ve been really nervous to use CellsSync due to this issue for fear of it demolishing my available bandwidth.

Some additional observations I’ve made: I wonder if it isn’t resyncing, so much as the scanning for changes every minute is adding up to a lot of bandwidth.

These are far from exact metrics, but I observed that, when performing a check, it takes CellSync roughly 3 or 4 seconds to scan for changes, and during that time Task Manager reported an average of 4mbps network usage coming from CellSync. I know Task Manager is FAR from the most reliable for network data, but if the “check” task is transferring even 4mb of data, and running once per minute, and assuming the computer is running 16 hours a day (my use case), that’s still almost 4gb of transfer in a day just to check if a sync needs to be run.

Could this possibly be the source of my high data usage?

Cheers