External changes to pydio workspaces not being detected

Pydio 8
1

I’m using Pydio Server 8 (latest version 8.2.2) on Docker.
So far so good, sync is working and seems everything is fine.

However, I have created a workspace where I have some files stored on the server, and expected that the “observer changes” feature would detect them and sync them over with the sync client.
However, it does not, and actually I can browse through the files and folder in pydio of the new workspace, but syncing shows as empty.
Also, I can’t index the workspace, using the cli, it says that I don’t have access (there’s only one user and it’s the admin account).

Could anybody give a hand? thanks in advance.

2

Hi,
to index the workspace you need to be logged as the apache user, but since it’s not really possible you have to run the command as a root user php cmd.php -u=admin -p=password -r=workspaceSlug -a=index.

As for the sync client, you must set the frequency of synchronization on the client itself (it will sync whenever there’s an event meaning a file upload, renaming, etc…) by default it should be automatic.

33
33892×1184 91.3 KB

You should disable this setting, it is only useful if you for instance add a file directly on the filesystem.

As long as you work on the synced folder of your sync client, it will do the process automatically.

regards.

3

imagen
imagen710×121 3.37 KB

Regarding the observer, I have activated it since yes, the changes are done outside pydio, at filesystem level (done by other tool in the machine). I have even tested at creating a file manually in the FS (touch test) and it isn’t being synced, but under pydio website I can see the file just fine.

4

only the admin user or a user with admin profile (with access to the workspace) are able to use the command to index.

For the sync client, on my end it works just fine even if you create a file with the cli,
now to tackle the issue, first i would like to know your OS and the sync client version.

5

It’s the only user, and it’s the admin user.

OS is Ubuntu 18.04.1 LTS (4.15.0-43-generic)
Docker version 18.06.0-ce, build 0ffa825 image linuxserver/pydio:latest

Is there maybe an official docker image for pydio 8?
I have pydio cells docker installed too but I’m waiting for the sync client to be released before trying it.
This is a home setup, not for any enterprise just to have a private cloud for me.

Sync client is latest stable
PydioSync v1.2.9 - Build 770d097f7c0d1e9e2026b97ee5f064484f90c41c (develop) -

6

Would you mind trying with this version,
https://download.pydio.com/pub/pydio-sync/develop/packages/

7

I have setup the new client.
It’s weird, it is syncing only the first tree of folders and files (and not the subfolders and files inside those.)

8

Ok,
could you confirm that it works now?
assuming that you will say yes,
when you say only the top 3 files & folders you mean amongst for instance 40 files and folders it will only pick 3 of each?
Also for the subfolders do you mean on your local system or for the workspace?

9

I say TREE. not three XD
I mean
root / folder1/subfolder1/file1

It will only get the folder1, not the subfolder1 or file1 (which are inside folder1)
All of them are synced, but only like that.

10

my bad my eyes are playing tricks on me today,
this is weird indeed when it syncs the folder it should also sync all of it’s content.

11

Could you take a look at the sync client logs,
i think you can find them like this (i have the mac client but i believe it’s the same).

18
Screenshot 2019-01-04 at 17.12.18.png934×1904 222 KB

if you cannot find them i will ask the dev about that.

12

What should I need to look at? I don’t see any issue here.

2019-01-04 17:16:32 INFO    5452  MainThread logging : 1627 | info | Product Version Number 2.0.0 and Version Date September 10 2018
2019-01-04 17:16:32 INFO    5452  MainThread logging : 1627 | info | -----------------------------------------------
2019-01-04 17:16:32 INFO    5452  MainThread logging : 1627 | info | Starting agent locally on http://localhost:5556/
2019-01-04 17:16:32 INFO    5452  MainThread logging : 1627 | info | ------------------------------------------------
2019-01-04 17:16:32 INFO    5452  MainThread logging : 1627 | info | Structure and Integrity of SQLite file C:\Users\XXXXX\AppData\Roaming\Pydio\server-music\changes.sqlite is intact 
2019-01-04 17:16:32 INFO    5452  MainThread logging : 1627 | info | Structure and Integrity of SQLite file C:\Users\XXXXX\AppData\Roaming\Pydio\server-music\pydio.sqlite is intact 
2019-01-04 17:16:32 INFO    7120  Thread-1 logging : 1627 | info | Scanning for changes since last application launch
2019-01-04 17:16:33 INFO    8628  Thread-2 logging : 1627 | info | Starting permanent monitor
2019-01-04 17:16:33 INFO    7120  Thread-1 logging : 1627 | info | Loading remote changes with sequence 99486 for job id server-music
2019-01-04 17:16:34 INFO    7120  Thread-1 logging : 1627 | info | Loading local changes with sequence 70 for job id server-music
2019-01-04 17:16:34 INFO    7120  Thread-1 logging : 1627 | info | No changes detected in server-music
2019-01-04 17:16:37 INFO    8184  Dummy-9  logging : 1627 | info | LANGUAGE guessed: ['es_ES', '', 'en_US']
2019-01-04 17:16:40 INFO    8184  Dummy-9  logging : 1627 | info | Requiring job server-music
2019-01-04 17:16:43 INFO    8184  Dummy-9  logging : 1627 | info | Job stopping
2019-01-04 17:16:43 INFO    8184  Dummy-9  logging : 1627 | info | R E S Y N C server-music
2019-01-04 17:16:43 INFO    8184  Dummy-9  logging : 1627 | info | Job Started
2019-01-04 17:16:43 INFO    2948  Thread-10 logging : 1627 | info | Scanning for changes since last application launch
2019-01-04 17:16:44 INFO    6788  Thread-11 logging : 1627 | info | Starting permanent monitor
2019-01-04 17:16:45 INFO    2948  Thread-10 logging : 1627 | info | Loading remote changes with sequence 0 for job id server-music
2019-01-04 17:16:45 INFO    2948  Thread-10 logging : 1627 | info | Loading local changes with sequence 0 for job id server-music
2019-01-04 17:16:45 INFO    2948  Thread-10 logging : 1627 | info | Reducing changes for server-music
2019-01-04 17:16:46 INFO    2948  Thread-10 logging : 1627 | info | No changes detected for server-music
2019-01-04 17:16:56 INFO    2948  Thread-10 logging : 1627 | info | Loading remote changes with sequence 68209 for job id server-music
2019-01-04 17:16:56 INFO    2948  Thread-10 logging : 1627 | info | Loading local changes with sequence 70 for job id server-music
2019-01-04 17:16:56 INFO    2948  Thread-10 logging : 1627 | info | No changes detected in server-music
13

Hi,
is that the log just after you tried to add a new file in your tree?
i you may, could you try to create the same tree and to see if there is an event triggering that will sync that.

14

A new file where?
The issues are when creating a new file via the FS and having it to sync via sync client.
If I create it via pydio (or the windows client) I don’t have any issues.

As a “related” issue, I have to constantly add the permissions to the user to read/write to this workspace, it gets deleted/unchecked from time to time ( a couple of hours usually)

15

sorry i meant on your fs directly,
what i want is that you reproduce the sync case with the tree to see what happens on the Sync Client log (if there is no event then we’ll know that the client is not detecting the new files/folders).

is it related to everytime the sync client, is syncing?
i want to know if there is any correlation.

16

Ok, I have created a new set of folder and they are not getting synced (nothing at all)
Regarding the permissions, it doesn’t seem to have any correlation, just seems to happen randomly

2019-01-07 12:03:00 INFO    8136  Thread-26 logging : 1627 | info | Loading remote changes with sequence 99492 for job id pydio.zriel.ga-music
2019-01-07 12:03:01 INFO    8136  Thread-26 logging : 1627 | info | Loading local changes with sequence 73 for job id pydio.zriel.ga-music
2019-01-07 12:03:01 INFO    8136  Thread-26 logging : 1627 | info | waiting db writing to end before retrieving local changes...
2019-01-07 12:03:01 INFO    8136  Thread-26 logging : 1627 | info | waiting db writing to end before retrieving local changes...
2019-01-07 12:03:02 INFO    8136  Thread-26 logging : 1627 | info | No changes detected in pydio.zriel.ga-music