[doc] password encrypted encryption-key: How to decrypt it using the password?

Pydio Cells
#1

Export password-protected encryption key.
I created a key and started using it for a given storage.
I backup my key using the “Export key” feature. There, I’m asked a (possibly empty) passphrase.

For a given passphrase (and supposedly given encryption key) I always receive different encrypted base64 strings.

I wonder, how is the key password-encrypted?
I would like to use openssl to decrypt it locally and be sure that the non-encrypted key is really always the same.

Thank you.

#2

As an additional question: Having the clear key (if possible) and assuming I get an encrypted file via
openstack object save <container> <file>

Which openssl command would allow decryption?

#3

Hi @drzraf,

You can find more information about encryption here: https://pydio.com/en/docs/cells/v2/encryption
If you think that we are missing details do not hesitate to tell us and we will update the doc consequently.

#4

Here are the three question I’m missing answers to from the documentation.

  • For a given passphrase (and supposedly given encryption key) I always receive different encrypted base64 strings when I export the key: Why?
  • How is the key password-encrypted?
  • Could you provide a sample openssl command that, provided a plain-text key, would decrypt an encrypted file managed by Pydio?
#5

File Encryption key (https://pydio.com/en/docs/cells/v2/encryption)

For a given file an unique random AES key of 256 bits is generated and encrypted with the encryption key of the datasource the file belongs to. The datasource encryption keys are generated by the admin and are protected using the default keyring of the system Cells is installed in.

SQL

  1. `SELECT key_data FROM idm_user_keys;
    This is my 80 characters base64-encoded of the 60 bytes master key. Right ?
  2. SELECT HEX(key_data), block_header_size, block_data_size FROM enc_node_keys NATURAL JOIN data_meta NATURAL JOIN enc_node_blocks WHERE data = '"test4.rtf"';
    • F2A703227282D4F6D54722FC134973652655A191E04F7D8ABDAEAB54B5F2BE90B00401AF4219BAA4AF7BEB1CBE00B3754388C26E0F9245D290972CAE # This is the 60 bytes file-specific unique key (120 characters HEX-encoded) which, is encrypted (using the data-source key).
    • 26 # block_header_size
    • 1751 # block_data_size
      These sole values are expected to allow me to decrypt the file, right?
  3. openstack object save pydio test4.rtf to get the encrypted file (here on OVH Swift+S3). 1777 bytes = 1751 + 26

Key decryption

According to SetupEncryptMode definition and call, encryptionKeyPlainBytes is passed coming from keyProtectionTool.GetDecrypted.
The first 12 bytes being a nonce.

Encrypted file format

The 1777 encrypted bytes are composed of:

  • A 26 bytes clear-text header
  • The rest of encrypted data
    • containing the a 16 bytes AES-GCM tag placed at the end of the data
  • From this code I assume that the 12 bytes nonce is actually represented by the 12 first bytes of the header.

Code

Contrary to libressl, openssl enc does not support AES-GCM but using Python Cryptodome should be.

from Cryptodome.Cipher import AES
import binascii, base64, sys

master_key = base64.b64decode(sys.argv[1])
encrypted_file_key = binascii.unhexlify(sys.argv[2])
file_key_nonce, encrypted_file_key = encrypted_file_key[:12], encrypted_file_key[12:]
cipher = AES.new(master_key[0:32], AES.MODE_GCM, nonce = file_key_nonce)
decrypted_file_key = cipher.decrypt(encrypted_file_key) # = 48 bytes

HeaderSize = 26 # hardcoded
with open(sys.argv[3], "rb") as f:
    data = f.read()
header = data[:26]
nonce, tag = header[:12], data[-16:]
data = data[26:-16]
cipher = AES.new(decrypted_file_key[:32], AES.MODE_GCM, nonce)
cipher.decrypt_and_verify(data, tag)

Run

$ aes-256-gcm-decrypt "$K" "$FK" test4.rtf

  • With “$K” from SELECT key_data from idm_user_key
  • And “$FK” the above file key from SELECT hex(key_data) FROM enc_node_keys
  • test4.rtf the encrypted local file

=> ValueError: MAC check failed (Cryptodome/Cipher/_mode_gcm.py :: decrypt_and_verify)

The decryption failed.
Why?
As you can see above I trim both the master key and the decrypted file key to their first 32 bytes but I feel bad about this. Is this the reason for the error? what the adequate use of these keys would be?

Thank you

(@charles?)

#6

Hi drzaf,

Everything in the code seems good but I suspect you are working with the wrong master_key content. Indeed make sure the content of $master_key is the decrypted version of the key you exported from the admin interface. Which in your case I would call it the datasource key.

And to answer your questions:

For a given passphrase (and supposedly given encryption key) I always receive different encrypted base64 strings when I export the key: Why?

  • Because a random parameter is used to cook the export output.

How is the key password-encrypted?

  • I am not sure I understand the question.

Could you provide a sample openssl command that, provided a plain-text key, would decrypt an encrypted file managed by Pydio?

  • I am affraid. In addition of the payload there are additional info that can be handled with a simple command

Regards

#7

Hi @jabarkarim,
Let me first thank you for helping on this.

I obtain the master key using SELECT key_data FROM idm_user_keys. I only have one data-source, and thus only one master-key. Downloading the encrypted file (test4.rtf) using pydio UI brings a readable file so we can safely assume that there is no problem with the instance configuration.

This master key within the {idm_user_keys} table is 80 characters long and base64 encoded. Once decoded, this give me the 60 bytes representing the cleartext master key.

  1. Could you confirm that what I’m getting above is right?

  2. Since AES-256 expects a 32 bytes key, can you confirm that only the first 32 bytes of the cleartext master key are to be used to decrypt the file key?

  3. Once the file-key (decrypted using the plaintext masterkey), is, once again, larger than 32 bytes (= 48). Could you confirm that I have to strip it again to only use the first 32 bytes?

  4. About the master-key export, you said output is cooked. Can we say that it’s symmetrically encrypted using the “user-provided”, “per-export” password and a random salt? If yes, then that what’s the algorithm? (so that, with this information at hand, I could obtain back what’s in {idm_user_keys} in order to verify the process)

Thank you

#8

I’m sorry to insist (it’s a moot point to confirm that crypto works in a secure and standard way and that we could decrypt files if anything goes wrong).

I see that encryption use:


And https://golang.org/src/crypto/aes/cipher.go shows that it’d fail if the key were not among 16, 24, 32 (and I guess it has always been that way).

func (kt *userKeyTool) keyByID(ctx context.Context, id string) ([]byte, error) {
   return base64.StdEncoding.DecodeString(rsp.Key.Content)
}
func (kt *userKeyTool) GetDecrypted(ctx context.Context, keyID string, encrypted []byte) ([]byte, error) {
   keyBytes, err := kt.keyByID(ctx, keyID)
   return crypto.Open(keyBytes, encrypted[:12], encrypted[12:])
}
func crypto.Open(key []byte, nonce []byte, cipherData []byte) ([]byte, error) {
	block, err := aes.NewCipher(key)
	aesgcm, err := cipher.NewGCM(block)
	return aesgcm.Open(nil, nonce, cipherData, nil)
}
func aes.NewCipher(key []byte) (cipher.Block, error) {
	k := len(key)
       if k not in [16, 24, 32] return nil, KeySizeError(k)
}

As such when base64-decoding the 80-chars value from {idm_user_keys} I’ve 60 bytes and after subtracting 12 bytes of nonce, I’m still having 48 bytes of master key instead of 32. What am I missing?

#9

Hi drzraf

I am having problems with this also.

I have encrypted files locally that I want to decrypt with my key but no information anywhere on how this can be done. Have you had any luck?

#10

De-obfuscating the encryption process does not seem like an easy task. As you can see, we are waiting for complementary answers from @jabarkarim.

#11

Hi,

I am sorry for the late answer

  1. The master key you obtain by using SELECT key_data FROM idm_user_keys is encrypted. All master keys before they are saved in the database are encrypted with the same unique key. That key is randomly generated at start once for all and is saved in the system keychain. Allow me to call it the cells system key.
    Now I dont know a reliable way to obtain the cells system key, that’s why I suggest you extract the master key from the Cells admin interface. Note that what you extracted is the encrypted version of the master key but this time it’s encrypted with a derived key from the password you provided.
    So to get the plain version of the master key you have to AES-GCM-256 decrypt what you’ve extracted with key generated from the password you provided for extraction.
    Refer to the crypto.KeyFromPassword method to obtain a derived key from a given password.

  2. No you dont have to extract anything. If you follow the steps above you’ll obtain a 32 bytes clear master key.

  3. Once again you dont have to strip anything.

  4. Can we say that it’s symmetrically encrypted using the “user-provided”, “per-export” password and a random salt? - The answer is YES and the algorithm is AES-GCM-256

I hope it helps.

Regards

#12

Thanks to your, @jabarkarim, I finally got it working (using the exported master-key as you suggested).

I hope this will help others pinpoint their encryption/decryption problems.

@charles: Interested in a MR for cell’s ./tools/* ?

(BTW, I think internals of the encryption process needs more documentation)

2 Likes