/
Using Secure Keys

Using Secure Keys

In CDAP, you can use secure keys to store sensitive information in a secure and encrypted manner. You might use secure keys for a passphrase, cryptographic key, access token, or any other data that needs to be stored securely.

You can use secure keys in the CDAP Sandbox, in-memory CDAP, and Distributed CDAP. The basic steps for creating secure keys is the same for all three versions of CDAP. The main difference is that CDAP Sandbox and in-memory CDAP use the Sun JCEKS implementation for storing secure keys and Distributed CDAP uses Hadoop KMS (Key Management Server)-backed secure storage. For more information, see Secure Storage.

The secure keys framework is pluggable, and you can also build your own secure key implementation to store the data in encrypted storage of your choice. For example, Cloud Data Fusion, which is the managed version of CDAP on GCP, uses Cloud KMS to store secure keys.

Creating Secure Keys

Secure keys are stored in the namespace where you create them and are unique to that namespace. You cannot share or copy secure keys across namespaces.

After you create a secure key, you can use the secure key in a pipeline or a compute profile.

Creating a Secure Key (CDAP Sandbox and in-memory CDAP)

To create a secure key, complete the following steps:

  1. To configure CDAP to use secure storage, edit the cdap-site.xml and cdap-security files.
    For more information, see the “File-backed Secure Storage” section.

  2. To create a secure key, use the Secure Storage HTTP RESTful API. For more information, see the “Add a Secure Key” section.

  3. After you create the secure key, you can use it in pipelines or a compute profile.

Creating a Secure Key (Distributed CDAP)

  1. Configure CDAP to use Hadoop KMS. For more information on integration with Hadoop KMS, see to Apache Hadoop Key Management Server (KMS).

  2. To configure CDAP to use secure storage, edit the cdap-site.xml and cdap-security files.
    For more information, see the “Hadoop Key Management Server-backed Secure Storage Secure Storage” section.

  3. To create a secure key, use the Secure Storage HTTP RESTful API. For more information, see the “Add a Secure Key” section.

  4. After you create the secure key, you can use it in pipelines or a compute profile.

Using Secure Keys

You can use secure keys in plugins and compute profiles. For pipelines, you add the secure key as a macro in any plugin that requires authentication. Likewise, you can add it to a namespace compute profile or system compute profile. For example, you might create a compute profile for a Dataproc cluster and use a secure key and for the Service Account Key in the compute profile.

Example: Using a Secure Key in a Pipeline

You’re using CDAP and want to create a pipeline that reads from a source database. You also want to create a secure key for your database password. It’s easy to do this in CDAP. The following steps walk you through this example.

To add a secure key to a pipeline, complete the following steps:

  1. In CDAP, open the HTTP interface. Click System Admin > Configuration > Make HTTP Calls.