Unable to delete user groups (1.2.5 / dev HEAD)

Pydio Cells
1

Just wanted to clean up the test installations for 1.2.5 and dev HEAD. While removing users worked, removing groups (no matter if empty or not) did not.

2019-02-05T21:09:24.947+0100	DEBUG	pydio.rest.user	Received User.Delete API request (LOGIN)	{"login": "group-users", "request": "/user/group-users/"}
2019-02-05T21:09:24.963+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'group-users') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Cghncm91cC1idA=="}]}
2019-02-05T21:09:24.966+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'group-users') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Cghncm91cC1idA=="}]}
2019-02-05T21:09:24.967+0100	DEBUG	pydio.grpc.user	SHOULD DELETE THESE	{"usersGroups": null}
2019-02-05T21:09:24.967+0100	DEBUG	Delete	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'group-users') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC"}

Line 4 in the log lines above (… SHOULD DELETE THESE …) should contain the details of the group to be deleted.

Below the same for a user deleted successfully:

2019-02-05T21:11:20.562+0100	DEBUG	pydio.rest.user	Received User.Delete API request (LOGIN)	{"login": "test-user", "request": "/user/test-user"}
2019-02-05T21:11:20.571+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'test-user') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Cgl0ZXN0LXVzZXI="}]}
2019-02-05T21:11:20.587+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'test-user') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Cgl0ZXN0LXVzZXI="}]}
2019-02-05T21:11:20.590+0100	DEBUG	pydio.grpc.user	SHOULD DELETE THESE	{"usersGroups": [{"Uuid":"18305106-0787-41bf-9b62-9e0000e72b95","GroupPath":"/","Attributes":{"profile":"standard","pydio:labelLike":"test-user"},"Roles":[{"Uuid":"ROOT_GROUP","GroupRole":true},{"Uuid":"18305106-0787-41bf-9b62-9e0000e72b95","Label":"test-user","UserRole":true}],"Login":"test-user","Password":"sha256:1000:b09qTVNZbURPTWpUclFaeHlzQVVMRUxVSXVRbWp1bWU=:kPaneuVCjoq6tahICUMFK9mJ1X6dsltB"}]}
2019-02-05T21:11:20.591+0100	DEBUG	Delete	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'test-user') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC"}
2019-02-05T21:11:20.621+0100	DEBUG	pydio.grpc.workspace	DELETING POLICIES ON EVENT	{"event": "Type:DELETE User:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" GroupPath:\"/\" Attributes:<key:\"profile\" value:\"standard\" > Attributes:<key:\"pydio:labelLike\" value:\"test-user\" > Roles:<Uuid:\"ROOT_GROUP\" GroupRole:true > Roles:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" Label:\"test-user\" UserRole:true > Login:\"test-user\" Password:\"sha256:1000:b09qTVNZbURPTWpUclFaeHlzQVVMRUxVSXVRbWp1bWU=:kPaneuVCjoq6tahICUMFK9mJ1X6dsltB\" > ", "subject": "user:test-user"}
2019-02-05T21:11:20.644+0100	DEBUG	pydio.grpc.workspace	DELETING POLICIES ON EVENT	{"event": "Type:DELETE Role:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" Label:\"User test-user\" UserRole:true LastUpdated:1549397467 AutoApplies:\"\" Policies:<id:219 Resource:\"18305106-0787-41bf-9b62-9e0000e72b95\" Action:READ Subject:\"profile:standard\" Effect:allow > Policies:<id:220 Resource:\"18305106-0787-41bf-9b62-9e0000e72b95\" Action:WRITE Subject:\"user:test-user\" Effect:allow > Policies:<id:221 Resource:\"18305106-0787-41bf-9b62-9e0000e72b95\" Action:WRITE Subject:\"profile:admin\" Effect:allow > > ", "subject": "role:18305106-0787-41bf-9b62-9e0000e72b95"}
2019-02-05T21:11:20.642+0100	DEBUG	DELETING POLICIES ON EVENT	{"event": "Type:DELETE User:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" GroupPath:\"/\" Attributes:<key:\"profile\" value:\"standard\" > Attributes:<key:\"pydio:labelLike\" value:\"test-user\" > Roles:<Uuid:\"ROOT_GROUP\" GroupRole:true > Roles:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" Label:\"test-user\" UserRole:true > Login:\"test-user\" Password:\"sha256:1000:b09qTVNZbURPTWpUclFaeHlzQVVMRUxVSXVRbWp1bWU=:kPaneuVCjoq6tahICUMFK9mJ1X6dsltB\" > ", "subject": "user:test-user"}
2

Hi,
are there any user inside the group while you were deleting it?

3

Hi zayn,

no, the groups are empty. I actually tried both (empty and not empty) with the same result.

4

@zayn did you reproduce that?

5

In my case, on the latest pull as of today (8 feb, 10PM) deleting groups does work.

if you want to take a look at the logs falk.john here’s a peek,

2019-02-08T10:25:25.178+0100	DEBUG	pydio.rest.user	Got Claims	{"claims": {"aud":"cells-front","iss":"http://192.168.0.172:8080/auth/dex","sub":"CiRjNTJiNjlhYS1jYzRhLTQzYmEtOTdhMi1mNWY5MDU0OGIxZmMSBXB5ZGlv","nonce":"c07f725e-2e1f-4f77-8756-dfc0cdc88dde","name":"admin","email":"","profile":"admin","email_verified":true,"roles":"ROOT_GROUP,ADMINS,c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","expiry":"0001-01-01T00:00:00Z","authSource":"pydioapi","displayName":"","groupPath":"/"}}
2019-02-08T10:25:25.187+0100	INFO	pydio.rest.user	Received User.Delete API request (GROUP)	{"login": "grouptestname", "crtGroup": "/", "request": "/user/grouptestname%2F"}
2019-02-08T10:25:25.194+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (mpath1 LIKE \"1.3.%\") and t.level >= 3) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Gg1ncm91cHRlc3RuYW1lIAE="}]}
2019-02-08T10:25:25.202+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND ((mpath1 LIKE \"1.3.%\") and t.level >= 3 OR (mpath1 LIKE \"1.3\"))) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Gg1ncm91cHRlc3RuYW1lIAE="}]}
2019-02-08T10:25:25.207+0100	DEBUG	pydio.grpc.user	SHOULD DELETE THESE	{"usersGroups": [{"Uuid":"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93","GroupPath":"/","Attributes":{"displayName":"grouptestlabel","pydio:labelLike":"grouptestname"},"IsGroup":true,"GroupLabel":"grouptestname"}]}
2019-02-08T10:25:25.212+0100	DEBUG	Delete	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND ((mpath1 LIKE \"1.3.%\") and t.level >= 3 OR (mpath1 LIKE \"1.3\"))) ORDER BY `n`.`name` ASC"}
2019-02-08T10:25:25.242+0100	DEBUG	pydio.grpc.workspace	DELETING POLICIES ON EVENT	{"event": "Type:DELETE User:<Uuid:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" GroupPath:\"/\" Attributes:<key:\"displayName\" value:\"grouptestlabel\" > Attributes:<key:\"pydio:labelLike\" value:\"grouptestname\" > IsGroup:true GroupLabel:\"grouptestname\" > ", "subject": "role:f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93"}
2019-02-08T10:25:25.258+0100	DEBUG	pydio.rest.user	JWTHttpHandler: Checking JWT
2019-02-08T10:25:25.258+0100	DEBUG	pydio.rest.user	JWTHttpHandler: Checking JWT
2019-02-08T10:25:25.259+0100	DEBUG	DELETING POLICIES ON EVENT	{"event": "Type:DELETE User:<Uuid:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" GroupPath:\"/\" Attributes:<key:\"displayName\" value:\"grouptestlabel\" > Attributes:<key:\"pydio:labelLike\" value:\"grouptestname\" > IsGroup:true GroupLabel:\"grouptestname\" > ", "subject": "role:f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93"}
2019-02-08T10:25:25.262+0100	DEBUG	pydio.grpc.workspace	DELETING POLICIES ON EVENT	{"event": "Type:DELETE Role:<Uuid:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" Label:\"Group grouptestname\" GroupRole:true LastUpdated:1549617915 AutoApplies:\"\" Policies:<id:23 Resource:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" Action:READ Subject:\"profile:standard\" Effect:allow > Policies:<id:24 Resource:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" Action:WRITE Subject:\"profile:admin\" Effect:allow > > ", "subject": "role:f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93"}
2019-02-08T10:25:25.267+0100	DEBUG	pydio.rest.user	Got Claims	{"claims": {"aud":"cells-front","iss":"http://192.168.0.172:8080/auth/dex","sub":"CiRjNTJiNjlhYS1jYzRhLTQzYmEtOTdhMi1mNWY5MDU0OGIxZmMSBXB5ZGlv","nonce":"c07f725e-2e1f-4f77-8756-dfc0cdc88dde","name":"admin","email":"","profile":"admin","email_verified":true,"roles":"ROOT_GROUP,ADMINS,c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","expiry":"0001-01-01T00:00:00Z","authSource":"pydioapi","displayName":"","groupPath":"/"}}
2019-02-08T10:25:25.270+0100	DEBUG	pydio.rest.user	Got Claims	{"claims": {"aud":"cells-front","iss":"http://192.168.0.172:8080/auth/dex","sub":"CiRjNTJiNjlhYS1jYzRhLTQzYmEtOTdhMi1mNWY5MDU0OGIxZmMSBXB5ZGlv","nonce":"c07f725e-2e1f-4f77-8756-dfc0cdc88dde","name":"admin","email":"","profile":"admin","email_verified":true,"roles":"ROOT_GROUP,ADMINS,c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","expiry":"0001-01-01T00:00:00Z","authSource":"pydioapi","displayName":"","groupPath":"/"}}
2019-02-08T10:25:25.277+0100	DEBUG	pydio.rest.user	Received User.Get API request	{"q": {"Queries":[{"GroupPath":"/","NodeType":2}],"Limit":1000,"Operation":1}}
2019-02-08T10:25:25.289+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE (EXISTS (SELECT 1 FROM `idm_user_policies` WHERE (((`idm_user_policies`.`subject` = '*') OR (`idm_user_policies`.`subject` = 'user:admin') OR (`idm_user_policies`.`subject` = 'profile:anon') OR (`idm_user_policies`.`subject` = 'profile:shared') OR (`idm_user_policies`.`subject` = 'profile:standard') OR (`idm_user_policies`.`subject` = 'profile:admin') OR (`idm_user_policies`.`subject` = 'role:ROOT_GROUP') OR (`idm_user_policies`.`subject` = 'role:ADMINS') OR (`idm_user_policies`.`subject` = 'role:c52b69aa-cc4a-43ba-97a2-f5f90548b1fc')) AND (`idm_user_policies`.`resource` = `t`.`uuid`) AND (`idm_user_policies`.`action` = 'READ'))) AND ((`t`.`uuid` = `n`.`uuid`) AND (mpath1 LIKE \"1.%\") and t.level = 2 AND (`n`.`leaf` = 0))) ORDER BY `n`.`name` ASC LIMIT 1000", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"GgEvSAI="}]}
2019-02-08T10:25:25.290+0100	DEBUG	pydio.rest.user	Received User.Get API request	{"q": {"Queries":[{"GroupPath":"/","NodeType":1},{"AttributeName":"hidden","AttributeValue":"true","not":true}],"Limit":50,"Operation":1}}
2019-02-08T10:25:25.299+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE (EXISTS (SELECT 1 FROM `idm_user_policies` WHERE (((`idm_user_policies`.`subject` = '*') OR (`idm_user_policies`.`subject` = 'user:admin') OR (`idm_user_policies`.`subject` = 'profile:anon') OR (`idm_user_policies`.`subject` = 'profile:shared') OR (`idm_user_policies`.`subject` = 'profile:standard') OR (`idm_user_policies`.`subject` = 'profile:admin') OR (`idm_user_policies`.`subject` = 'role:ROOT_GROUP') OR (`idm_user_policies`.`subject` = 'role:ADMINS') OR (`idm_user_policies`.`subject` = 'role:c52b69aa-cc4a-43ba-97a2-f5f90548b1fc')) AND (`idm_user_policies`.`resource` = `t`.`uuid`) AND (`idm_user_policies`.`action` = 'READ'))) AND (((`t`.`uuid` = `n`.`uuid`) AND (mpath1 LIKE \"1.%\") and t.level = 2 AND (`n`.`leaf` = 1)) AND ((`t`.`uuid` = `n`.`uuid`) AND NOT EXISTS (SELECT * FROM `idm_user_attributes` AS `a` WHERE ((`a`.`uuid` = `t`.`uuid`) AND (`a`.`name` = 'hidden') AND (`a`.`value` = 'true')))))) ORDER BY `n`.`name` ASC LIMIT 50", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"GgEvSAE="},{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"KgZoaWRkZW4yBHRydWVQAQ=="}]}
2019-02-08T10:25:25.338+0100	DEBUG	pydio.rest.user	GetACLsForRoles	{"acls": [{"ID":"25","Action":{"Name":"parameter:core.conf:lang","Value":"\"en-us\""},"RoleID":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","WorkspaceID":"PYDIO_REPO_SCOPE_ALL"}], "roles": [{"Uuid":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","Label":"admin role","UserRole":true,"LastUpdated":1549617472,"AutoApplies":[""],"Policies":[{"id":3,"Resource":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","Action":2,"Subject":"profile:standard","Effect":1},{"id":4,"Resource":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","Action":3,"Subject":"user:admin","Effect":1},{"id":5,"Resource":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","Action":3,"Subject":"profile:admin","Effect":1}]},{"Uuid":"d5f1d4e8-e01d-487f-844a-956c006034e1","Label":"pydio.anon.user role","UserRole":true,"LastUpdated":1549617480,"AutoApplies":[""],"Policies":[{"id":8,"Resource":"d5f1d4e8-e01d-487f-844a-956c006034e1","Action":2,"Subject":"user:pydio.anon.user","Effect":1},{"id":9,"Resource":"d5f1d4e8-e01d-487f-844a-956c006034e1","Action":2,"Subject":"profile:admin","Effect":1},{"id":10,"Resource":"d5f1d4e8-e01d-487f-844a-956c006034e1","Action":3,"Subject":"profile:admin","Effect":1}]}], "actions": [{"Name":"parameter:*"}], "t": "2.874424ms"}
6

I just tried with current HEAD but the result remains the same.

The logs also look different:

mine:

pydio.rest.user Received User.Delete API request (LOGIN) {"login": "group-users", "request": "/user/group-users/"}

versus yours:

pydio.rest.user Received User.Delete API request (GROUP) {"login": "grouptestname", "crtGroup": "/", "request": "/user/grouptestname%2F"}

Differentiation between a user and a group delete request appears to be done by the request suffix - in my requests the path ends in “/” while in yours it ends in “%2F” (encoded “/”). The latter is what is checked for in code.

I can not tell why in my case the trailing “/” is appened unencoded (tried different browsers), but after extending the test to also check for “/” in both relevant files (idm/user/rest/rest.go and cmd/ctl/cmd/user.go) deleting groups succeeds.

Also checking for “/” may just be a workaround, but I will issue a PR anyway because finding the root cause will probably take longer. This even may be browser issue (browser, version, os) with javascript implementations behaving different.

7

maybe we have a misunderstanding,
when you talk about groups are you referring to this kind of group,

32
Screenshot 2019-02-08 at 12.06.32.png3390×164 8.84 KB

8

yes, it’s about deleting IDM groups

9

… forgot to mention, I operate a reverse proxy (apache httpd) in between that appears to receive the correct request URI. It should not perform any rewriting or decoding on the URI though :thinking:

10

could be the proxy altering with the request,
maybe you trying without the proxy will give you another results, then atleast we would locate where it changes.

11

Appears to be the proxy - for what reason ever. However, Charles just merged the fix into master as it will not do any harm and will prevent others from running into the same issue. I will try to find out what happens with the proxy request, but as this will take time I have to postpone it for a while.

12

yeah,
thanks again for your feedback and help.