DataMasque Portal

File connections

File connections define the data sources for DataMasque to read from and data destinations for the masked files to be written.

Supported data sources

DataMasque supports AWS S3 Buckets and Azure Blob Storage as data sources or data destinations.

Supported file types

DataMasque supports masking the following files types:

Object File Types Tabular File Types
JSON CSV
XML Parquet

Add a new file connection

To add a file connection (Data source/Data destination), click the  Add button  button from Data Sources panel or Data Destinations of the File masking dashboard.

Use the form to configure the file connection parameters. You may validate that the connection parameters are correct using the Test Connection button. This will attempt a connection using the provided parameters. If the connection fails, an error will be shown with the failure details. The Test button is also available from the dashboard.

Add connection

Connection Parameters

Standard parameters

Connection name A unique name for the connection. May only contain alphanumeric characters and underscores.
Connection type The type of the data source. Available options: AWS S3 Bucket or Azure Blob Storage.
Base directory The base directory containing target files to mask or for masked files to be written.

Connection Specific parameters

AWS S3 Bucket

For AWS S3 Bucket connections, the following field is required:

Bucket name The name of the S3 bucket where the base directory can be found.
Azure Blob Storage

For Azure Blob Storage connections, the following field is required:

Container name The name of the Azure Blob Storage where the base directory can be found.
Connection string The connection string configured with the authorization information to access data in your Azure Storage account.

View and edit file connections

To edit a connection, click the edit (Edit button) button for the connection you wish to edit in the Connections panel of the File masking dashboard.

A full list of connection parameters can be found under Connection Parameters.

File connections panel

Delete file connections

To delete a connection, open the connection for editing (see View and edit file connections) and click the Delete button. You will be prompted for confirmation before the connection is deleted. A connection can also be deleted from the dashboard by clicking the trashcan icon.Delete button

File connection actions

Note:

  • Deleting a connection only deletes it from the side it was deleted from, e.g. If a connection is present on both
  • sources and destinations, after deleting the connection from the destinations, it will still be present as a source.

Configuring access between DataMasque and AWS S3 Buckets

If access is restricted from DataMasque to the file connection AWS credentials may need to be set up to access the AWS S3 bucket in order to mask the files. To do this, execute the following steps:

For AWS EC2 machines

Your Amazon S3 Bucket must have read-write access to the S3 bucket configured on your connections.

Follow the steps below to configure the AWS EC2 instance that will be accessing your S3 bucket:

  1. Open the Amazon IAM console, create an IAM policy. The following example IAM policy grants programmatic read-write access to the test bucket:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::test"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": ["arn:aws:s3:::test/*"]
    }
  ]
}
  1. Create an IAM role:
  • Select AWS service as the trusted identity, EC2 as the use case.
  • Attach the policy created from the previous step to the role.
  1. Open the Amazon IAM console,
  • Choose Instances.
  • Select the instance that you want to attach the IAM role to.
  • Choose the Actions tab, choose Security and then choose Modify IAM role.
  • Select the IAM role to attach to your EC2 instance.
  • Finally, choose Save.

For more information relating to setting up access between DataMasque and AWS EC2 instances, please refer to the following AWS Documentation

For non-EC2 machines

  1. Access the machine where DataMasque is running, either via SSH or other means.

  2. Install AWS CLI and configure your own credentials:
    curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
    sudo yum install -y unzip
    unzip awscliv2.zip
    sudo ./aws/install
    aws configure

  3. Enter your AWS Credentials.

  4. Test the connection to reach the S3 bucket.