...
Abstract: Kettle is used more and more in enterprises where the standard obfuscation of credentials is not sufficient enough. There are requirements to use strong encryption methods and even to store internal data encrypted (covered in PDI-6168 and PDI-6170). The above use cases inspired me to create some simple transformations to test and play around with encryption.
Note: Kettle already has job entries to encrypt and decrypt files with PGP.
Authors: Jens Bleuel
License: LGPL
Kettle versions: 4.1 and later
Update since Kettle 4.2: There are two steps in the experimental section: Secret key generator, Symmetric Cryptography that cover this use case.
Attachments
- cryptography.zip - Transformations & sample files
...
Transformation cryptographyCreateSecretKey is used to creating create a secret key.
Use the transformations cryptographyEncrypt and cryptographyDecrypt for testing.
...
To use AES with 192- and 256-bit key sizes, you need unlimited strength cryptography.
Due to import-control restrictions imposed by some countries, the jurisdiction policy files shipped
with the Java 2 SDK, v 1.4 only permit strong cryptography to be used. An unlimited strength version of these files
(that is, with no restrictions on cryptographic strength) is available for download, however.
Instead of storing the decrypted data to a file there are a lof of other options, e.g.:
- use the decrypted data as credentials in subsequent steps or transformations
- put the decrypted data into variables visible in a limited scope (e.g. parent job) and use them as credentials for databases, repository etc. (see PDI-6168)
- and many more options
We may consider:
- Symmetric-
...
- key algorithm vs. asymmetric key algorithms (public-key cryptography)
...
- Diffie-
...
- Hellman key exchange is a specific method of exchanging keys.
...
- Ensure integrity e.g. by hash-codes
...
- Key file handling could be optimized in different ways.
...
- Please keep in mind that unencrypted data is in RAM
...
- (see PDI-6170
...
- for a circumvention to prevent heap dumps)
- Beneath the binary or indexed storage type, an encrypted storage type may be possible in Kettle core.
The key serialization to file is a little trick to obfuscate the key. Other methods can be included instead of the
clear text file output.
Instead of storing the decrypted data to a file there are a lof of other options, e.g.:
- use the decrypted data as credentials in subsequent steps or transformations
- put the decrypted data into variables visible in a limited scope (e.g. parent job) and use them as credentials for databases, repository etc. (see PDI-6168)
- and many more options
For screen shots For screen shots, some further background information and a test run, please see http://kettle.bleuel.com/2011/06/07/security-considerations-and-encryption-with-kettle/