File deleted automatically, but I don't want to?

Pydio Cells
1

Hey there,

I did some modification to folder and file :

I have a folder nammed “Hélène” that contains videos, and also a folder that contains remastered videos inside the folder Hélène.

I deleted all the videos inside the root folder ( Hélène ), then moved all the remastered videos to the root folder, and deleted the remastered folder.

Sadly I lose all the videos except one…
I’m totally sure that I didn’t deleted the videos.

Here is a screen from the log :

Sans titre-1
Sans titre-11704×771 86.9 KB

It appears that the deletion coincide with the data sync that a crontab run all 30min.
There is nothing inside the trash.

Why the files have been deleted and is there a way to get them back ?

Thanks

2

Hello,

just to be sure:

  • it is a structured datasource or a flat one ?
  • did you wait until move jobs are done before deleting the remastered folder ?
3

Hello,

I think its a structured datasource ( its the same path in linux and pydio cells )

Yes I waited until the popup in left bottom with the task list was ok.

Thanks,

4

hmm,

So I assume you already had a look:

  • in the recycle of this workspace
  • via linux on the FS

and could not find your files back, right?

This might be a bug:

  • Which version are you in?
  • can you reproduce?
  • Could you have a look at the real log files (usually /var/cells/logs/pydio.log and /var/cells/logs/task.log if you follow our recommended best practices) and see if there is more error that the one you show above?
5

Hey,

Yes and yes, i can’t find the files. If I do the find command I have result like this : but this is not file, this is folder that contains a json.

root@cloud:/home/pydio# find / -name "*(2022).mp4"

/disk1/pydio/.minio.sys/buckets/pydiods1/Partage Plex/Videos/Hélène/1 à 2 ans/dessin craie (2022).mp4
/disk1/pydio/.minio.sys/buckets/pydiods1/Partage Plex/Videos/Hélène/1 à 2 ans/discussion avant dodo (2022).mp4
/disk1/pydio/.minio.sys/buckets/pydiods1/Partage Plex/Videos/Hélène/1 à 2 ans/gouter (2022).mp4

I’m in pydio cells home 4.0.1
I send you the log by private message, from the times I moved the first file to the time it delete.

6

OK I’ll try to have a look at it.

There is one important question you did not answer: did you try and succeed to reproduce, or have you seen that only once ?

7

Hello, I tryed to reproduce and didn’t achieve to reproduce the problem.
In fact, I Think I deleted the files while it was moving, because it was very long, file are 2gb, it take a really long time before it finish, and popup disapear.
This time, I waited a very long time to be sure.

Maybe you can try to add a protection on this ?

8

OK, thanks for the feedback.

We already have some locks for such use cases that should prevent this to happen, but we might have missed a corner case.

We will try to have a look with big files and your user story and see if we can further improve our fail safes.

9

Oh no.

I restored my videos from my backup, and I did the exact same process again, being very careful.
The optimised videos were created by another app and I had to resync the datasources to see them. But once synced, all was okay with the files.
Everything went fine, until I moved the optimized videos to the parent folder.

The pop up with the list of move tasks appeared for a few seconds, then disappeared and all my files just disappeared, again!

I have more error log than before :

{"level":"error","ts":"2022-11-03T15:05:26+01:00","logger":"pydio.grpc.tasks","msg":"Error during copy/move","error":"{\"Id\":\"object.not.found\",\"Code\":404,\"Detail\":\"object was not found, this is not normal: Partage Plex/Videos/Hélène/1 à 2 ans/Plex Versions/Personnalise _ Universal TV 2085/Dessin a la craie (2022).mp4\",\"Status\":\"Not Found\"}","OperationUuid":"copy-move-608f0ee1-c4a4-4929-9f5f-8dd05fa51ea2-88393a96","SchedulerJobUuid":"copy-move-608f0ee1-c4a4-4929-9f5f-8dd05fa51ea2","SchedulerTaskUuid":"88393a96-6f55-4a4f-abfb-dcb76bcc9c06","SchedulerTaskActionPath":"ROOT/actions.tree.copymove$0","UserName":"####","UserUuid":"a42936da-3ed2-4a34-87b6-e3ab4c662ef5","GroupPath":"/","Profile":"admin","Roles":",,,ROOT_GROUP,ADMINS,a42936da-3ed2-4a34-87b6-e3ab4c662ef5"}
{"level":"info","ts":"2022-11-03T15:05:26+01:00","logger":"pydio.grpc.tasks","msg":"{\"level\":\"error\",\"ts\":\"2022-11-03T15:05:26+01:00\",\"logger\":\"pydio.grpc.tasks\",\"msg\":\"Error during copy/move\",\"error\":\"{\\\"Id\\\":\\\"object.not.found\\\",\\\"Code\\\":404,\\\"Detail\\\":\\\"object was not found, this is not normal: Partage Plex/Videos/Hélène/1 à 2 ans/Plex Versions/Personnalise _ Universal TV 2085/Dessin a la craie (2022).mp4\\\",\\\"Status\\\":\\\"Not Found\\\"}\",\"OperationUuid\":\"copy-move-608f0ee1-c4a4-4929-9f5f-8dd05fa51ea2-88393a96\",\"SchedulerJobUuid\":\"copy-move-608f0ee1-c4a4-4929-9f5f-8dd05fa51ea2\",\"SchedulerTaskUuid\":\"88393a96-6f55-4a4f-abfb-dcb76bcc9c06\",\"SchedulerTaskActionPath\":\"ROOT/actions.tree.copymove$0\",\"UserName\":\"####\",\"UserUuid\":\"a42936da-3ed2-4a34-87b6-e3ab4c662ef5\",\"GroupPath\":\"/\",\"Profile\":\"admin\",\"Roles\":\",,,ROOT_GROUP,ADMINS,a42936da-3ed2-4a34-87b6-e3ab4c662ef5\"}"}
{"level":"info","ts":"2022-11-03T15:05:29+01:00","logger":"pydio.grpc.tasks","msg":"Forcing close session move-b2b8d3a7-81fd-4728-953c-c50bf196d9f5 and unlock","OperationUuid":"copy-move-4b10ee6c-abdc-44d1-a93b-56b4a83e7989-4d16ca31","SchedulerJobUuid":"copy-move-4b10ee6c-abdc-44d1-a93b-56b4a83e7989","SchedulerTaskUuid":"4d16ca31-00ce-46fe-b491-3e154022ba60","SchedulerTaskActionPath":"ROOT/actions.tree.copymove$0","UserName":"####","UserUuid":"a42936da-3ed2-4a34-87b6-e3ab4c662ef5","GroupPath":"/","Profile":"admin","Roles":",,,ROOT_GROUP,ADMINS,a42936da-3ed2-4a34-87b6-e3ab4c662ef5"}
{"level":"info","ts":"2022-11-03T15:05:29+01:00","logger":"pydio.grpc.tasks","msg":"{\"level\":\"info\",\"ts\":\"2022-11-03T15:05:29+01:00\",\"logger\":\"pydio.grpc.tasks\",\"msg\":\"Forcing close session move-b2b8d3a7-81fd-4728-953c-c50bf196d9f5 and unlock\",\"OperationUuid\":\"copy-move-4b10ee6c-abdc-44d1-a93b-56b4a83e7989-4d16ca31\",\"SchedulerJobUuid\":\"copy-move-4b10ee6c-abdc-44d1-a93b-56b4a83e7989\",\"SchedulerTaskUuid\":\"4d16ca31-00ce-46fe-b491-3e154022ba60\",\"SchedulerTaskActionPath\":\"ROOT/actions.tree.copymove$0\",\"UserName\":\"####\",\"UserUuid\":\"a42936da-3ed2-4a34-87b6-e3ab4c662ef5\",\"GroupPath\":\"/\",\"Profile\":\"admin\",\"Roles\":\",,,ROOT_GROUP,ADMINS,a42936da-3ed2-4a34-87b6-e3ab4c662ef5\"}"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"API: GetObject(bucket=io, object=common-files/Partage Plex/Videos/Hélène/1 à 2 ans/Plex Versions/Personnalise _ Universal TV 2086/Discussion avant dodo (2022).mp4)"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"Time: 15:05:30 CET 11/03/2022"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"DeploymentID: b6c8c441-6e07-4c8a-8863-ee66a069be15"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"RequestID: 17241829F850EF8C"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"RemoteHost: 192.168.1.1"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"Host: ####"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"Error: Unable to write all the data to client Object not found: io/common-files/Partage Plex/Videos/Hélène/1 à 2 ans/Plex Versions/Personnalise _ Universal TV 2086/Discussion avant dodo (2022).mp4 (*fmt.wrapError)"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"       2: github.com/minio/minio@v0.0.0-20220720015624-ce8397f7d944/cmd/object-handlers.go:515:cmd.objectAPIHandlers.GetObjectHandler()"}
{"level":"error","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.gateway.data","msg":"       1: net/http/server.go:2109:http.HandlerFunc.ServeHTTP()"}
{"level":"info","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"[s3://127.0.0.1:35495/pydiods1] Analyzed 500 nodes","SpanRootUuid":"67a66001-3ebe-4879-8329-20c97da0e3b3","SpanParentUuid":"67a66001-3ebe-4879-8329-20c97da0e3b3","SpanUuid":"e957251d-c184-406f-83fb-9b32e23cfada","OperationUuid":"resync-ds-pydiods1-516ca999","SchedulerJobUuid":"resync-ds-pydiods1","SchedulerTaskUuid":"516ca999-e309-4fe3-88fd-39f12e46c43d","SchedulerTaskActionPath":"ROOT/actions.cmd.resync$0"}
{"level":"info","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-03T15:05:30+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"[s3://127.0.0.1:35495/pydiods1] Analyzed 500 nodes\",\"SpanRootUuid\":\"67a66001-3ebe-4879-8329-20c97da0e3b3\",\"SpanParentUuid\":\"67a66001-3ebe-4879-8329-20c97da0e3b3\",\"SpanUuid\":\"e957251d-c184-406f-83fb-9b32e23cfada\",\"OperationUuid\":\"resync-ds-pydiods1-516ca999\",\"SchedulerJobUuid\":\"resync-ds-pydiods1\",\"SchedulerTaskUuid\":\"516ca999-e309-4fe3-88fd-39f12e46c43d\",\"SchedulerTaskActionPath\":\"ROOT/actions.cmd.resync$0\"}"}
{"level":"warn","ts":"2022-11-03T15:05:30+01:00","msg":"[ClientsPool] cannot find datasource, retrying in 10s...","ds":"versions","retries":4}
{"level":"info","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.grpc.tasks","msg":"{\"level\":\"warn\",\"ts\":\"2022-11-03T15:05:30+01:00\",\"msg\":\"[ClientsPool] cannot find datasource, retrying in 10s...\",\"ds\":\"versions\",\"retries\":4}"}
{"level":"warn","ts":"2022-11-03T15:05:30+01:00","msg":"[ClientsPool] cannot find datasource, retrying in 10s...","ds":"versions","retries":4}
{"level":"info","ts":"2022-11-03T15:05:30+01:00","logger":"pydio.grpc.tasks","msg":"{\"level\":\"warn\",\"ts\":\"2022-11-03T15:05:30+01:00\",\"msg\":\"[ClientsPool] cannot find datasource, retrying in 10s...\",\"ds\":\"versions\",\"retries\":4}"}
{"level":"warn","ts":"2022-11-03T15:05:31+01:00","msg":"[ClientsPool] cannot find datasource, retrying in 10s...","ds":"versions","retries":4}
{"level":"info","ts":"2022-11-03T15:05:31+01:00","logger":"pydio.grpc.tasks","msg":"{\"level\":\"warn\",\"ts\":\"2022-11-03T15:05:31+01:00\",\"msg\":\"[ClientsPool] cannot find datasource, retrying in 10s...\",\"ds\":\"versions\",\"retries\":4}"}
{"level":"warn","ts":"2022-11-03T15:05:31+01:00","msg":"[ClientsPool] cannot find datasource, retrying in 10s...","ds":"versions","retries":4}
{"level":"info","ts":"2022-11-03T15:05:31+01:00","logger":"pydio.grpc.tasks","msg":"{\"level\":\"warn\",\"ts\":\"2022-11-03T15:05:31+01:00\",\"msg\":\"[ClientsPool] cannot find datasource, retrying in 10s...\",\"ds\":\"versions\",\"retries\":4}"}
{"level":"warn","ts":"2022-11-03T15:05:31+01:00","msg":"[ClientsPool] cannot find datasource, retrying in 10s...","ds":"versions","retries":4}
{"level":"info","ts":"2022-11-03T15:05:31+01:00","logger":"pydio.grpc.tasks","msg":"{\"level\":\"warn\",\"ts\":\"2022-11-03T15:05:31+01:00\",\"msg\":\"[ClientsPool] cannot find datasource, retrying in 10s...\",\"ds\":\"versions\",\"retries\":4}"}
{"level":"info","ts":"2022-11-03T15:05:31+01:00","msg":"Got errors on datasource, should resync now branch: /common-files/Partage Plex/Videos/Hélène/1 à 2 ans"}
{"level":"info","ts":"2022-11-03T15:05:31+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-03T15:05:31+01:00\",\"msg\":\"Got errors on datasource, should resync now branch: /common-files/Partage Plex/Videos/Hélène/1 à 2 ans\"}"}
10

Bad news, indeed.

Could you please precise:

  • do you use structured or flat datasources?
  • do you use the Cells Sync Desktop client?
11

just to make sure, after “all your files disappeared”, did you try to resync? did they really disappear from FS ? I would suspect the datasource is in a wrong state after the first re-indexation, and triggering the move operation just “shows” hidden errors.
So if you can re-play the same scenario again, can you make sure that after the first ds resync, you see all files correctly and you are able to read them correctly as well? all of them? With/without accented characters?

12

Yep, I can reproduce more or less your case.
You should verify that if you make a final resync, moved files do re-appear at the correct location (where they were moved to).
Probably linked to the changes on eTag we introduced. We’ll dig that to fix the issue.

13

I think I nailed it, could you kindly give a try with this binary Index of /pub/cells/dev/linux-amd64 ?

Thx !

1 Like
14

Hey there,

Sorry for the delay and thanks for the help.
I tryed with the new update : 4.0.4 ( in the changelog it says there is a fix ? New files detection on structured datasource (local FS) was partially broken)

Exactly same problem. Moved all the file, when resync, they all disapeared.

Here is the log :

{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:07+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"S3 Walk Operation + Stats took\",\"d\":23.931269012}"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Diff Stats","s":{"EndpointLeft":"s3://127.0.0.1:35495/pydiods1","EndpointRight":"index://pydiods1","conflicts":0,"missingLeft":7,"missingRight":0},"SpanRootUuid":"56ec142b-ef2b-441e-9156-8f2392b302cc","SpanParentUuid":"56ec142b-ef2b-441e-9156-8f2392b302cc","SpanUuid":"a8b085ca-d999-43f6-b58c-dcc854774117","OperationUuid":"resync-ds-pydiods1-88f69879","SchedulerJobUuid":"resync-ds-pydiods1","SchedulerTaskUuid":"88f69879-a458-4fe4-8183-65f71070a00b","SchedulerTaskActionPath":"ROOT/actions.cmd.resync$0"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Finished analyze : 31991 nodes (15993 on s3://127.0.0.1:35495/pydiods1, 15998 on index://pydiods1)","i":31991,"SpanRootUuid":"56ec142b-ef2b-441e-9156-8f2392b302cc","SpanParentUuid":"56ec142b-ef2b-441e-9156-8f2392b302cc","SpanUuid":"a8b085ca-d999-43f6-b58c-dcc854774117","OperationUuid":"resync-ds-pydiods1-88f69879","SchedulerJobUuid":"resync-ds-pydiods1","SchedulerTaskUuid":"88f69879-a458-4fe4-8183-65f71070a00b","SchedulerTaskActionPath":"ROOT/actions.cmd.resync$0"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:07+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Diff Stats\",\"s\":{\"EndpointLeft\":\"s3://127.0.0.1:35495/pydiods1\",\"EndpointRight\":\"index://pydiods1\",\"conflicts\":0,\"missingLeft\":7,\"missingRight\":0},\"SpanRootUuid\":\"56ec142b-ef2b-441e-9156-8f2392b302cc\",\"SpanParentUuid\":\"56ec142b-ef2b-441e-9156-8f2392b302cc\",\"SpanUuid\":\"a8b085ca-d999-43f6-b58c-dcc854774117\",\"OperationUuid\":\"resync-ds-pydiods1-88f69879\",\"SchedulerJobUuid\":\"resync-ds-pydiods1\",\"SchedulerTaskUuid\":\"88f69879-a458-4fe4-8183-65f71070a00b\",\"SchedulerTaskActionPath\":\"ROOT/actions.cmd.resync$0\"}"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:07+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Finished analyze : 31991 nodes (15993 on s3://127.0.0.1:35495/pydiods1, 15998 on index://pydiods1)\",\"i\":31991,\"SpanRootUuid\":\"56ec142b-ef2b-441e-9156-8f2392b302cc\",\"SpanParentUuid\":\"56ec142b-ef2b-441e-9156-8f2392b302cc\",\"SpanUuid\":\"a8b085ca-d999-43f6-b58c-dcc854774117\",\"OperationUuid\":\"resync-ds-pydiods1-88f69879\",\"SchedulerJobUuid\":\"resync-ds-pydiods1\",\"SchedulerTaskUuid\":\"88f69879-a458-4fe4-8183-65f71070a00b\",\"SchedulerTaskActionPath\":\"ROOT/actions.cmd.resync$0\"}"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Sending unidirectional patch","patch":{"Source":"s3://127.0.0.1:35495/pydiods1","Target":"index://pydiods1","Type":"TreePatch"},"SpanRootUuid":"56ec142b-ef2b-441e-9156-8f2392b302cc","SpanParentUuid":"56ec142b-ef2b-441e-9156-8f2392b302cc","SpanUuid":"a8b085ca-d999-43f6-b58c-dcc854774117","OperationUuid":"resync-ds-pydiods1-88f69879","SchedulerJobUuid":"resync-ds-pydiods1","SchedulerTaskUuid":"88f69879-a458-4fe4-8183-65f71070a00b","SchedulerTaskActionPath":"ROOT/actions.cmd.resync$0"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:07+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Sending unidirectional patch\",\"patch\":{\"Source\":\"s3://127.0.0.1:35495/pydiods1\",\"Target\":\"index://pydiods1\",\"Type\":\"TreePatch\"},\"SpanRootUuid\":\"56ec142b-ef2b-441e-9156-8f2392b302cc\",\"SpanParentUuid\":\"56ec142b-ef2b-441e-9156-8f2392b302cc\",\"SpanUuid\":\"a8b085ca-d999-43f6-b58c-dcc854774117\",\"OperationUuid\":\"resync-ds-pydiods1-88f69879\",\"SchedulerJobUuid\":\"resync-ds-pydiods1\",\"SchedulerTaskUuid\":\"88f69879-a458-4fe4-8183-65f71070a00b\",\"SchedulerTaskActionPath\":\"ROOT/actions.cmd.resync$0\"}"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Filtering TreePatch took 8.896µs","duration":0.000008896,"source":"s3://127.0.0.1:35495/pydiods1","target":"index://pydiods1"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:07+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Filtering TreePatch took 8.896µs\",\"duration\":0.000008896,\"source\":\"s3://127.0.0.1:35495/pydiods1\",\"target\":\"index://pydiods1\"}"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.index.pydiods1","msg":"Opening Indexation Session de59624d-7003-4e19-91c0-91dc9e893ae1","SpanRootUuid":"56ec142b-ef2b-441e-9156-8f2392b302cc","SpanParentUuid":"56ec142b-ef2b-441e-9156-8f2392b302cc","SpanUuid":"e0149ea4-f2af-4c66-b2c0-288ebdfd311c","OperationUuid":"resync-ds-pydiods1-88f69879","SchedulerJobUuid":"resync-ds-pydiods1","SchedulerTaskUuid":"88f69879-a458-4fe4-8183-65f71070a00b","SchedulerTaskActionPath":"ROOT/actions.cmd.resync$0"}
{"level":"info","ts":"2022-11-14T17:38:07+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:07+01:00\",\"logger\":\"pydio.grpc.data.index.pydiods1\",\"msg\":\"Opening Indexation Session de59624d-7003-4e19-91c0-91dc9e893ae1\",\"SpanRootUuid\":\"56ec142b-ef2b-441e-9156-8f2392b302cc\",\"SpanParentUuid\":\"56ec142b-ef2b-441e-9156-8f2392b302cc\",\"SpanUuid\":\"e0149ea4-f2af-4c66-b2c0-288ebdfd311c\",\"OperationUuid\":\"resync-ds-pydiods1-88f69879\",\"SchedulerJobUuid\":\"resync-ds-pydiods1\",\"SchedulerTaskUuid\":\"88f69879-a458-4fe4-8183-65f71070a00b\",\"SchedulerTaskActionPath\":\"ROOT/actions.cmd.resync$0\"}"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Finished resync 370.597187ms"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:08+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Deleted file\",\"NodePath\":\"Partage Plex/Videos/Hélène/1 à 2 ans/Dessin a la craie (2022).mp4\",\"target\":\"index://pydiods1\"}"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Deleted file","NodePath":"Partage Plex/Videos/Hélène/1 à 2 ans/Dessin a la craie (2022).mp4","target":"index://pydiods1"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Deleted file","NodePath":"Partage Plex/Videos/Hélène/1 à 2 ans/Discussion avant dodo (2022).mp4","target":"index://pydiods1"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:08+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Deleted file\",\"NodePath\":\"Partage Plex/Videos/Hélène/1 à 2 ans/Discussion avant dodo (2022).mp4\",\"target\":\"index://pydiods1\"}"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Deleted file","NodePath":"Partage Plex/Videos/Hélène/1 à 2 ans/Helene 2 ans bougies (2022).mp4","target":"index://pydiods1"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:08+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Deleted file\",\"NodePath\":\"Partage Plex/Videos/Hélène/1 à 2 ans/Helene 2 ans bougies (2022).mp4\",\"target\":\"index://pydiods1\"}"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Deleted file","NodePath":"Partage Plex/Videos/Hélène/1 à 2 ans/Le gouter (2022).mp4","target":"index://pydiods1"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:08+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Deleted file\",\"NodePath\":\"Partage Plex/Videos/Hélène/1 à 2 ans/Le gouter (2022).mp4\",\"target\":\"index://pydiods1\"}"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:08+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Deleted file\",\"NodePath\":\"Partage Plex/Videos/Hélène/1 à 2 ans/Nettoyage de la terrasse (2022)-1.mp4\",\"target\":\"index://pydiods1\"}"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Deleted file","NodePath":"Partage Plex/Videos/Hélène/1 à 2 ans/Nettoyage de la terrasse (2022)-1.mp4","target":"index://pydiods1"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Deleted file","NodePath":"Partage Plex/Videos/Hélène/1 à 2 ans/Salak (2022).mp4","target":"index://pydiods1"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"{\"level\":\"info\",\"ts\":\"2022-11-14T17:38:08+01:00\",\"logger\":\"pydio.grpc.data.sync.pydiods1\",\"msg\":\"Deleted file\",\"NodePath\":\"Partage Plex/Videos/Hélène/1 à 2 ans/Salak (2022).mp4\",\"target\":\"index://pydiods1\"}"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Deleted file","NodePath":"Partage Plex/Videos/Hélène/1 à 2 ans/Tete dans la chambre (2022).mp4","target":"index://pydiods1"}
{"level":"info","ts":"2022-11-14T17:38:08+01:00","logger":"pydio.grpc.data.sync.pydiods1","msg":"Finishing patch session"}

I don’t understand why it says : ““logger”:“pydio.grpc.data.sync.pydiods1”,“msg”:“Deleted file””
I never asked to delete them, I just moved them.

The files are not in FS, they really disappear.

Thanks

15

Hum.
Ok, can you try to describe a simple scenario to reproduce this issue from scratch? Make sure to provide all the successive steps starting from an empty structured DS.
We’ll have it!
thx

16

Hello,

Here is the exact step :

  • copy an uncompressed video in the cells web interface from my personal data source to a shared data source
  • compress the video in the shared datasource with an external software ( Plex )
  • Plex creates a new folder ( called Plex Version ) in the shared folder, then in this folder it creates the compressed video
  • I have to synchronize my datasource to see the new folder and file created by Plex in cells interface
  • I move the compressed video to the root of the folder
  • I delete the uncompressed video in the root of the shared folder
  • everything works ! the compressed video is here, in the good folder.
  • resync shared datasource
  • the compressed video is deleted from cells and FS :frowning: :frowning: :frowning:
17

Can you please precise how big are the video files (compressed and uncompressed) ?

18

For the last test, uncompressed videos was 1.2gb and compressed around 400mb

19

Is it also OK on the FS ?

The sync (launched on the DS from the admin console) cannot delete a file on the FS:
it is a unidirectional sync reading the FS and updating the db (index), it does not modify the FS.
So we miss something here.

We tried to reproduce but everythings work on our test instance with two 1GB files that have the same name.

A few more question to follow up:

  • Do you observe the same issue with smaller files?
  • Are you sure there is no Cells Sync Desktop client that might be connected on this instance
  • We had to switch the step delete and move from your process or the 2 files cannot have the same name
  • We assume that the plex app runs with the correct user (it creates a folder and a file…), right ?
  • Do you permanently delete or move to the recycle bin?
  • In your process the file also transits trough your Personal File DS. Is it structured or flat?
20

Hello, thanks for the answer and sorry for the delay,

I did the exactly same process, instead of move I tryed copy.

It works… better as expected ?! New file is copied in the correct folder… AND the source file used to copy… disappear. It disappear from web interface and FS.
Excatly what the move function is supposed to do ? But I used copy.

I’m lost :crazy_face:

I tryed with a 110mo as source and 17mo as compressed : yes.

Yes totally sure

Yes, there is a plex user dedicated to plex app.

It’s a permanently delete

It’s a structured

Thanks!