I had a major stability problem a couple of weeks ago and this MIGHT be related, but I’m not sure so I’m posting it as a new issue…
I have run into a sync problem with the Common Files workspace, in particular with the pydiods1 datasource.
If someone creates a new folder in the common files workspace, no one can see that new folder (including the person who created it) until I go into the admin console and re-sync the pydiods1 datasource. New folders DO appear in /volume/cells/data/pydiods1/ folder on the server itself. Just not in cells.
This is only for the local datastores. For anything in minio, if files are loaded through pydio, they appear almost immediately. But if the files are loaded to minio directly (I have a shared folder on my pc that syncs to minio periodically), they don’t appear in pydio until I run a re-sync.
So I guess what I’m looking for is this, is there a sync task that needs to run periodically (and maybe isn’t) to re-sync all data sources? or do I just have to go in and do it myself all the time?
For the record, directly accessing and modifying the underlying file system without going through Pydio Layers is not seen by the index until you trigger a resync. This is how it works and expected.
If you want to impact the FS from automated scripts or equivalent, you should rather have a look at the Cells Client or the REST APIs.
Blockquote
For the record, directly accessing and modifying the underlying file system without going through Pydio Layers is not seen by the index until you trigger a resync . This is how it works and expected.
I don’t think you understand. The folders are being created through pydio. I only checked the underlying file system to see if the folder was actually being created on bare metal as there is no apparent error inside pydio. This is the same for minio in that files uploaded through pydio to a minio datasource disappear instantly and do not re-appear until I re-sync.
So am I to understand that something new needs to be created externally to trigger scheduled resync commands? Can I do that with a shell script on the parent server?
For this issue it could be just the UI not refreshing and displaying the content,
could you try to reproduce this and just reload the workspace. (it might be the websocket not notifying the ui that a new folder has been created and therefore refresh the listing to display it).
If the issue still appears could you give a description of your entire setup (server, reverse proxy…)
Indeed if you create data directly on the storage (and not through cells) you will have to resync,
you could either run it manually (on the webUI or with the cli) each time you put data or have a cron job running it for you,
for instance with the cells binary to resync a datasource you can run
I tried hitting f5 on the workspace several times, and I also logged out of pydio and back in a few times as well with no change. Was this what you meant?
For this one, I spoke too quickly. MOST of the time, this is the case, but occasionally (and I did not see this when testing) it acts the same as the local stores.
I think this is exactly what I need. I’m going to set a chron job to run a script sync the most used datasources once a day (like at midnight) and see if that helps. if my wife (or the inlaws) complain, I might add a few entries in crontab to run it more often but I think (based on how long each sync takes), I don’t want to run this more than once every few hours at the worst case scenario.