Unable to delete user groups (1.2.5 / dev HEAD)


#1

Just wanted to clean up the test installations for 1.2.5 and dev HEAD. While removing users worked, removing groups (no matter if empty or not) did not.

2019-02-05T21:09:24.947+0100	DEBUG	pydio.rest.user	Received User.Delete API request (LOGIN)	{"login": "group-users", "request": "/user/group-users/"}
2019-02-05T21:09:24.963+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'group-users') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Cghncm91cC1idA=="}]}
2019-02-05T21:09:24.966+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'group-users') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Cghncm91cC1idA=="}]}
2019-02-05T21:09:24.967+0100	DEBUG	pydio.grpc.user	SHOULD DELETE THESE	{"usersGroups": null}
2019-02-05T21:09:24.967+0100	DEBUG	Delete	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'group-users') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC"}

Line 4 in the log lines above (… SHOULD DELETE THESE …) should contain the details of the group to be deleted.

Below the same for a user deleted successfully:

2019-02-05T21:11:20.562+0100	DEBUG	pydio.rest.user	Received User.Delete API request (LOGIN)	{"login": "test-user", "request": "/user/test-user"}
2019-02-05T21:11:20.571+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'test-user') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Cgl0ZXN0LXVzZXI="}]}
2019-02-05T21:11:20.587+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'test-user') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Cgl0ZXN0LXVzZXI="}]}
2019-02-05T21:11:20.590+0100	DEBUG	pydio.grpc.user	SHOULD DELETE THESE	{"usersGroups": [{"Uuid":"18305106-0787-41bf-9b62-9e0000e72b95","GroupPath":"/","Attributes":{"profile":"standard","pydio:labelLike":"test-user"},"Roles":[{"Uuid":"ROOT_GROUP","GroupRole":true},{"Uuid":"18305106-0787-41bf-9b62-9e0000e72b95","Label":"test-user","UserRole":true}],"Login":"test-user","Password":"sha256:1000:b09qTVNZbURPTWpUclFaeHlzQVVMRUxVSXVRbWp1bWU=:kPaneuVCjoq6tahICUMFK9mJ1X6dsltB"}]}
2019-02-05T21:11:20.591+0100	DEBUG	Delete	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (`n`.`name` = 'test-user') AND (`n`.`leaf` = 1)) ORDER BY `n`.`name` ASC"}
2019-02-05T21:11:20.621+0100	DEBUG	pydio.grpc.workspace	DELETING POLICIES ON EVENT	{"event": "Type:DELETE User:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" GroupPath:\"/\" Attributes:<key:\"profile\" value:\"standard\" > Attributes:<key:\"pydio:labelLike\" value:\"test-user\" > Roles:<Uuid:\"ROOT_GROUP\" GroupRole:true > Roles:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" Label:\"test-user\" UserRole:true > Login:\"test-user\" Password:\"sha256:1000:b09qTVNZbURPTWpUclFaeHlzQVVMRUxVSXVRbWp1bWU=:kPaneuVCjoq6tahICUMFK9mJ1X6dsltB\" > ", "subject": "user:test-user"}
2019-02-05T21:11:20.644+0100	DEBUG	pydio.grpc.workspace	DELETING POLICIES ON EVENT	{"event": "Type:DELETE Role:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" Label:\"User test-user\" UserRole:true LastUpdated:1549397467 AutoApplies:\"\" Policies:<id:219 Resource:\"18305106-0787-41bf-9b62-9e0000e72b95\" Action:READ Subject:\"profile:standard\" Effect:allow > Policies:<id:220 Resource:\"18305106-0787-41bf-9b62-9e0000e72b95\" Action:WRITE Subject:\"user:test-user\" Effect:allow > Policies:<id:221 Resource:\"18305106-0787-41bf-9b62-9e0000e72b95\" Action:WRITE Subject:\"profile:admin\" Effect:allow > > ", "subject": "role:18305106-0787-41bf-9b62-9e0000e72b95"}
2019-02-05T21:11:20.642+0100	DEBUG	DELETING POLICIES ON EVENT	{"event": "Type:DELETE User:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" GroupPath:\"/\" Attributes:<key:\"profile\" value:\"standard\" > Attributes:<key:\"pydio:labelLike\" value:\"test-user\" > Roles:<Uuid:\"ROOT_GROUP\" GroupRole:true > Roles:<Uuid:\"18305106-0787-41bf-9b62-9e0000e72b95\" Label:\"test-user\" UserRole:true > Login:\"test-user\" Password:\"sha256:1000:b09qTVNZbURPTWpUclFaeHlzQVVMRUxVSXVRbWp1bWU=:kPaneuVCjoq6tahICUMFK9mJ1X6dsltB\" > ", "subject": "user:test-user"}

#2

Hi,
are there any user inside the group while you were deleting it?


#3

Hi zayn,

no, the groups are empty. I actually tried both (empty and not empty) with the same result.


#4

@zayn did you reproduce that?


#5

In my case, on the latest pull as of today (8 feb, 10PM) deleting groups does work.

if you want to take a look at the logs falk.john here’s a peek,

2019-02-08T10:25:25.178+0100	DEBUG	pydio.rest.user	Got Claims	{"claims": {"aud":"cells-front","iss":"http://192.168.0.172:8080/auth/dex","sub":"CiRjNTJiNjlhYS1jYzRhLTQzYmEtOTdhMi1mNWY5MDU0OGIxZmMSBXB5ZGlv","nonce":"c07f725e-2e1f-4f77-8756-dfc0cdc88dde","name":"admin","email":"","profile":"admin","email_verified":true,"roles":"ROOT_GROUP,ADMINS,c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","expiry":"0001-01-01T00:00:00Z","authSource":"pydioapi","displayName":"","groupPath":"/"}}
2019-02-08T10:25:25.187+0100	INFO	pydio.rest.user	Received User.Delete API request (GROUP)	{"login": "grouptestname", "crtGroup": "/", "request": "/user/grouptestname%2F"}
2019-02-08T10:25:25.194+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND (mpath1 LIKE \"1.3.%\") and t.level >= 3) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Gg1ncm91cHRlc3RuYW1lIAE="}]}
2019-02-08T10:25:25.202+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND ((mpath1 LIKE \"1.3.%\") and t.level >= 3 OR (mpath1 LIKE \"1.3\"))) ORDER BY `n`.`name` ASC", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"Gg1ncm91cHRlc3RuYW1lIAE="}]}
2019-02-08T10:25:25.207+0100	DEBUG	pydio.grpc.user	SHOULD DELETE THESE	{"usersGroups": [{"Uuid":"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93","GroupPath":"/","Attributes":{"displayName":"grouptestlabel","pydio:labelLike":"grouptestname"},"IsGroup":true,"GroupLabel":"grouptestname"}]}
2019-02-08T10:25:25.212+0100	DEBUG	Delete	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE ((`t`.`uuid` = `n`.`uuid`) AND ((mpath1 LIKE \"1.3.%\") and t.level >= 3 OR (mpath1 LIKE \"1.3\"))) ORDER BY `n`.`name` ASC"}
2019-02-08T10:25:25.242+0100	DEBUG	pydio.grpc.workspace	DELETING POLICIES ON EVENT	{"event": "Type:DELETE User:<Uuid:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" GroupPath:\"/\" Attributes:<key:\"displayName\" value:\"grouptestlabel\" > Attributes:<key:\"pydio:labelLike\" value:\"grouptestname\" > IsGroup:true GroupLabel:\"grouptestname\" > ", "subject": "role:f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93"}
2019-02-08T10:25:25.258+0100	DEBUG	pydio.rest.user	JWTHttpHandler: Checking JWT
2019-02-08T10:25:25.258+0100	DEBUG	pydio.rest.user	JWTHttpHandler: Checking JWT
2019-02-08T10:25:25.259+0100	DEBUG	DELETING POLICIES ON EVENT	{"event": "Type:DELETE User:<Uuid:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" GroupPath:\"/\" Attributes:<key:\"displayName\" value:\"grouptestlabel\" > Attributes:<key:\"pydio:labelLike\" value:\"grouptestname\" > IsGroup:true GroupLabel:\"grouptestname\" > ", "subject": "role:f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93"}
2019-02-08T10:25:25.262+0100	DEBUG	pydio.grpc.workspace	DELETING POLICIES ON EVENT	{"event": "Type:DELETE Role:<Uuid:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" Label:\"Group grouptestname\" GroupRole:true LastUpdated:1549617915 AutoApplies:\"\" Policies:<id:23 Resource:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" Action:READ Subject:\"profile:standard\" Effect:allow > Policies:<id:24 Resource:\"f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93\" Action:WRITE Subject:\"profile:admin\" Effect:allow > > ", "subject": "role:f5da6167-f2eb-4c50-9b5c-5cca9f0b9c93"}
2019-02-08T10:25:25.267+0100	DEBUG	pydio.rest.user	Got Claims	{"claims": {"aud":"cells-front","iss":"http://192.168.0.172:8080/auth/dex","sub":"CiRjNTJiNjlhYS1jYzRhLTQzYmEtOTdhMi1mNWY5MDU0OGIxZmMSBXB5ZGlv","nonce":"c07f725e-2e1f-4f77-8756-dfc0cdc88dde","name":"admin","email":"","profile":"admin","email_verified":true,"roles":"ROOT_GROUP,ADMINS,c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","expiry":"0001-01-01T00:00:00Z","authSource":"pydioapi","displayName":"","groupPath":"/"}}
2019-02-08T10:25:25.270+0100	DEBUG	pydio.rest.user	Got Claims	{"claims": {"aud":"cells-front","iss":"http://192.168.0.172:8080/auth/dex","sub":"CiRjNTJiNjlhYS1jYzRhLTQzYmEtOTdhMi1mNWY5MDU0OGIxZmMSBXB5ZGlv","nonce":"c07f725e-2e1f-4f77-8756-dfc0cdc88dde","name":"admin","email":"","profile":"admin","email_verified":true,"roles":"ROOT_GROUP,ADMINS,c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","expiry":"0001-01-01T00:00:00Z","authSource":"pydioapi","displayName":"","groupPath":"/"}}
2019-02-08T10:25:25.277+0100	DEBUG	pydio.rest.user	Received User.Get API request	{"q": {"Queries":[{"GroupPath":"/","NodeType":2}],"Limit":1000,"Operation":1}}
2019-02-08T10:25:25.289+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE (EXISTS (SELECT 1 FROM `idm_user_policies` WHERE (((`idm_user_policies`.`subject` = '*') OR (`idm_user_policies`.`subject` = 'user:admin') OR (`idm_user_policies`.`subject` = 'profile:anon') OR (`idm_user_policies`.`subject` = 'profile:shared') OR (`idm_user_policies`.`subject` = 'profile:standard') OR (`idm_user_policies`.`subject` = 'profile:admin') OR (`idm_user_policies`.`subject` = 'role:ROOT_GROUP') OR (`idm_user_policies`.`subject` = 'role:ADMINS') OR (`idm_user_policies`.`subject` = 'role:c52b69aa-cc4a-43ba-97a2-f5f90548b1fc')) AND (`idm_user_policies`.`resource` = `t`.`uuid`) AND (`idm_user_policies`.`action` = 'READ'))) AND ((`t`.`uuid` = `n`.`uuid`) AND (mpath1 LIKE \"1.%\") and t.level = 2 AND (`n`.`leaf` = 0))) ORDER BY `n`.`name` ASC LIMIT 1000", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"GgEvSAI="}]}
2019-02-08T10:25:25.290+0100	DEBUG	pydio.rest.user	Received User.Get API request	{"q": {"Queries":[{"GroupPath":"/","NodeType":1},{"AttributeName":"hidden","AttributeValue":"true","not":true}],"Limit":50,"Operation":1}}
2019-02-08T10:25:25.299+0100	DEBUG	Users Search Query 	{"q": "SELECT `t`.`uuid`, `t`.`level`, `t`.`rat`, `n`.`name`, `n`.`leaf`, `n`.`etag` FROM `idm_user_idx_tree` AS `t`, `idm_user_idx_nodes` AS `n` WHERE (EXISTS (SELECT 1 FROM `idm_user_policies` WHERE (((`idm_user_policies`.`subject` = '*') OR (`idm_user_policies`.`subject` = 'user:admin') OR (`idm_user_policies`.`subject` = 'profile:anon') OR (`idm_user_policies`.`subject` = 'profile:shared') OR (`idm_user_policies`.`subject` = 'profile:standard') OR (`idm_user_policies`.`subject` = 'profile:admin') OR (`idm_user_policies`.`subject` = 'role:ROOT_GROUP') OR (`idm_user_policies`.`subject` = 'role:ADMINS') OR (`idm_user_policies`.`subject` = 'role:c52b69aa-cc4a-43ba-97a2-f5f90548b1fc')) AND (`idm_user_policies`.`resource` = `t`.`uuid`) AND (`idm_user_policies`.`action` = 'READ'))) AND (((`t`.`uuid` = `n`.`uuid`) AND (mpath1 LIKE \"1.%\") and t.level = 2 AND (`n`.`leaf` = 1)) AND ((`t`.`uuid` = `n`.`uuid`) AND NOT EXISTS (SELECT * FROM `idm_user_attributes` AS `a` WHERE ((`a`.`uuid` = `t`.`uuid`) AND (`a`.`name` = 'hidden') AND (`a`.`value` = 'true')))))) ORDER BY `n`.`name` ASC LIMIT 50", "q2": [{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"GgEvSAE="},{"type_url":"type.googleapis.com/idm.UserSingleQuery","value":"KgZoaWRkZW4yBHRydWVQAQ=="}]}
2019-02-08T10:25:25.338+0100	DEBUG	pydio.rest.user	GetACLsForRoles	{"acls": [{"ID":"25","Action":{"Name":"parameter:core.conf:lang","Value":"\"en-us\""},"RoleID":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","WorkspaceID":"PYDIO_REPO_SCOPE_ALL"}], "roles": [{"Uuid":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","Label":"admin role","UserRole":true,"LastUpdated":1549617472,"AutoApplies":[""],"Policies":[{"id":3,"Resource":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","Action":2,"Subject":"profile:standard","Effect":1},{"id":4,"Resource":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","Action":3,"Subject":"user:admin","Effect":1},{"id":5,"Resource":"c52b69aa-cc4a-43ba-97a2-f5f90548b1fc","Action":3,"Subject":"profile:admin","Effect":1}]},{"Uuid":"d5f1d4e8-e01d-487f-844a-956c006034e1","Label":"pydio.anon.user role","UserRole":true,"LastUpdated":1549617480,"AutoApplies":[""],"Policies":[{"id":8,"Resource":"d5f1d4e8-e01d-487f-844a-956c006034e1","Action":2,"Subject":"user:pydio.anon.user","Effect":1},{"id":9,"Resource":"d5f1d4e8-e01d-487f-844a-956c006034e1","Action":2,"Subject":"profile:admin","Effect":1},{"id":10,"Resource":"d5f1d4e8-e01d-487f-844a-956c006034e1","Action":3,"Subject":"profile:admin","Effect":1}]}], "actions": [{"Name":"parameter:*"}], "t": "2.874424ms"}

#6

I just tried with current HEAD but the result remains the same.

The logs also look different:

mine:

pydio.rest.user Received User.Delete API request (LOGIN) {"login": "group-users", "request": "/user/group-users/"}

versus yours:

pydio.rest.user Received User.Delete API request (GROUP) {"login": "grouptestname", "crtGroup": "/", "request": "/user/grouptestname%2F"}

Differentiation between a user and a group delete request appears to be done by the request suffix - in my requests the path ends in “/” while in yours it ends in “%2F” (encoded “/”). The latter is what is checked for in code.

I can not tell why in my case the trailing “/” is appened unencoded (tried different browsers), but after extending the test to also check for “/” in both relevant files (idm/user/rest/rest.go and cmd/ctl/cmd/user.go) deleting groups succeeds.

Also checking for “/” may just be a workaround, but I will issue a PR anyway because finding the root cause will probably take longer. This even may be browser issue (browser, version, os) with javascript implementations behaving different.


#7

maybe we have a misunderstanding,
when you talk about groups are you referring to this kind of group,


#8

yes, it’s about deleting IDM groups


#9

… forgot to mention, I operate a reverse proxy (apache httpd) in between that appears to receive the correct request URI. It should not perform any rewriting or decoding on the URI though :thinking:


#10

could be the proxy altering with the request,
maybe you trying without the proxy will give you another results, then atleast we would locate where it changes.


#11

Appears to be the proxy - for what reason ever. However, Charles just merged the fix into master as it will not do any harm and will prevent others from running into the same issue. I will try to find out what happens with the proxy request, but as this will take time I have to postpone it for a while.


#12

yeah,
thanks again for your feedback and help.