DataMasque Portal

In-Flight Masking API Reference

Overview

Full URLs are not used in this document. The hostname and scheme are omitted from the URLs, so just the path is provided. To get the full URL of endpoints, prepend your DataMasque hostname.

For example, the Swagger documentation URL is listed as /ifm/docs/. If your DataMasque instances was, for example, https://datamasque.example.com, the full URL would be https://datamasque.example.com/ifm/docs/.

Authorization

All in-flight ruleset plans listed on the page require a JSON Web Token (JWT) for authentication. Please refer to the JWT Login documentation for information on how request a JWT with a username and password.

Also see the JWT Refresh documentation for information on using a refresh token.

Once a JWT is received, it must be included in the Authorization header for in-flight masking requests, preceded by the word Bearer. For example:

Authorization: Bearer <your JWT access token>

Common Status Codes

This document does not repeat common failure status codes for each endpoint. Common failure codes are listed below, but the response should checked for detailed errors.

Status Code Description
400 A generic invalid request error. Check the response for more details.
401 The authorization JWT is invalid (bad signature or expired).
402 The DataMasque license is invalid.
403 Permission denied for the particular endpoint. Check that your user has the correct role to access it.
404 Resource/endpoint not found.
422 The uploaded data is not of the correct format, for example, fields are the wrong type or missing.
429 The rate limit(s) have been exceeded.
500 An unhandled error occurred. Check logs for more information.

Error Responses

Depending on the type of error, the response body may contain a JSON payload with error details.

Generally responses with a 400 or 422 status code will contain error information in the body. Other status codes may not.

This table shows the fields that may be available in error responses.

Field Type Description
error string An error message about what caused the response to fail. Generally available for 400 status code responses.
detail array of validation objects When 422 status codes are returned, this field contains information about why the request validation failed. See below.

For 422 responses, each validation object in the detail array contains:

  • type: The type of validation error.
  • loc: Location of the error in the request.
  • msg: Human-readable error message.
  • input: The invalid value that was provided.
  • ctx (optional): More context-specific information about the error.

Below is an example of a validation response for a 422 status code. In this example, the disable_instance_secret was expected to be a boolean and was not.

{
  "detail": [
    {
      "type": "bool_parsing",
      "loc": [
        "body",
        "disable_instance_secret"
      ],
      "msg": "Input should be a valid boolean, unable to interpret input",
      "input": "notabool"
    }
  ]
}

Trailing Slashes in URLs

Please note that all URLs require a trailing /, for example, /ifm/ruleset-plans/ is not the same as /ifm/ruleset-plans. Any URLs without a trailing / will return a 404 Not Found response.

Rate Limits

By default, in-flight masking has rate limiting enabled. To configure the limits, refer to the rate limit(s) documentation.

All HTTP responses contain headers describing the current rate limits.

Additionally, if a rate limit has been exceeded, responses will be returned with a 429 status code, and will include the Retry-After header to indicate when the rate limit expires.

This table describes the purpose of rate-limit related headers.

Header Description
Retry-After The number of seconds to wait before retrying the request.
X-RateLimit-Limit The limit of requests for the applied rate limit.
X-RateLimit-Remaining The number of requests remaining before hitting the rate limit.
X-RateLimit-Reset A unix timestamp indicating when the current rate limit will be reset.

Interactive API Documentation

DataMasque in-flight masking provides two interactive HTML API documents, using Swagger and Redoc. These are automatically generated and allow for interactive testing of in-flight masking, without using tools like Postman or curl.

Swagger Documentation

Viewable in your browser at /ifm/docs/.

Redoc Documentation

Viewable in your browser at /ifm/redoc/.

Ruleset Plans

Endpoints for creating, updating or deleting ruleset plans.

Ruleset Plan Object Formats

Ruleset Plan Detail Object

The following fields are valid in the request body when creating a ruleset plan, and are returned with fetching or listing ruleset plans.

Field Type Required Description
name string Yes The name of the ruleset plan to create, which will be used to build its URL.
ruleset_yaml string Not required if the ruleset is provided as a file upload for a multipart request. The masking YAML ruleset content as a string.
options RulesetPlanOptions object No An RulesetPlanOptions object.

Ruleset Plan Update Object

The following fields are valid in the request body when updating a ruleset plan using PATCH or PUT. Note the ruleset plan name can not be changed after creation.

Field Type Required Description
ruleset_yaml string Not required if the ruleset is provided as a file upload for a multipart request. The masking YAML ruleset content as a string.
options RulesetPlanOptions object No An RulesetPlanOptions object.

Ruleset Plan Create or Update Result Object

These fields are returned when a ruleset plan is updated or created.

Field Type Description
name string The name of the created ruleset plan.
ruleset_yaml string The masking YAML ruleset content as a string.
created_time string The date and time the ruleset plan was created, in ISO-8601 format.
modified_time string The date and time the ruleset plan was modified, in ISO-8601 format.
options RulesetPlanOptions object An RulesetPlanOptions object.
serial integer The serial number (version) of the ruleset plan. Automatically incremented as the ruleset plan is updated.
url string The full URL of the created ruleset plan.
logs Array of Log Message objects Any logs recorded during mask setup. See Log Message Object for the format.

RulesetPlanOptions Object

An object that contains options for a ruleset plan.

Field Type Required Description
enabled boolean No Enable or disable the ruleset plan. Defaults to true.
default_encoding string No Only json encoding is currently supported.
default_charset string No Only utf-8 charset is currently supported.
default_log_level string No Adjust this to set the default level of logging. See Log Levels for valid options. Defaults to INFO. May be overridden in a masking request.

Log Message Object

The following fields are included in log messages.

Field Type Description
log_level string Log level of the message. See Log Levels for a list of levels.
message string The log message.
timestamp string The date and time of the message, in ISO-8601 format.

Log Levels

The following log levels are available, in order of most to least important:

  • CRITICAL
  • ERROR
  • WARNING
  • INFO
  • DEBUG

GET /ifm/ruleset-plans/

Allowed Roles: Admin, Mask Builder or Mask Runner

Get a list of all ruleset plans that are currently loaded.

Returns a list of Ruleset Plan Detail objects.

GET /ifm/ruleset-plans/ Parameters

Field Type Required Location Description
limit integer No Query Parameter Controls pagination size by specifying how many ruleset plans to return. Default: 50
offset integer No Query Parameter Controls pagination starting point by skipping the specified number of ruleset plans. Default: 0

GET /ifm/ruleset-plans/ Response

The response from the ruleset plan list contains pagination metadata and ruleset plan details.

Field Type Description
items Array of RulesetPlanDetail objects The list of Ruleset Plan Detail for the current page.
total integer The total number of ruleset plans in the system.
limit integer Maximum number of items per page.
offset integer Starting position of this page in the full list.

POST /ifm/ruleset-plans/

Allowed Roles: Admin or Mask Builder

Create a new ruleset plan that will receive data and return masked versions.

This URL can accept data in multiple ways:

  • As a JSON encoded body containing a Ruleset Plan Detail object. The ruleset_yaml should be embedded in the JSON and conform to correct JSON encoding standards. This method should be preferred if updating ruleset programmatically, where the ruleset can be JSON encoded easily.
  • As a multipart form data upload, where the Ruleset Plan Detail object is sent as a form field and does not include the ruleset_yaml. Instead, the ruleset is uploaded as a file in the ruleset file field. This method is easier to use when working with curl.

These two methods are covered in detail separately below.

Regardless of the method chosen to create the ruleset plan, a Ruleset Plan Create or Update Result Object is returned.

Ruleset Plan Name Suffix Generation

When a ruleset plan is created, a name for it is specified in the request. To prevent naming collisions, DataMasque appends a random suffix to the name requested. The suffix is a - followed by 6 alphanumeric characters.

The name is returned in the creation response name field. The url of the ruleset plan is also returned, and this will contain the name with the random suffix. This url should be used for all future interaction with the ruleset plan, such as masking or option updating.

The URL of the ruleset plan is also included in the response's Location header.

POST /ifm/ruleset-plans/ with JSON Content

Create a ruleset plan using YAML content embedded inside the Ruleset Plan Detail Object sent in the request body.

The content type of the request must be application/json.

The returned response is a Ruleset Plan Create or Update Result Object.

POST /ifm/ruleset-plans/ with JSON Content curl example
curl -X POST "https://<your-datamasque-host>/ifm/ruleset-plans/" \
     -H "Authorization: Bearer <access-jwt>" \
     -H "Content-Type: application/json" \
     -d '{"name": "from_file_firstname_mask", "ruleset_yaml": "version: \"1.0\"\nrules:\n- masks:\n  - type: from_file\n    seed_file: DataMasque_firstNames_mixed.csv"}'

POST /ifm/ruleset-plans/ with Form Content

Create a ruleset plan using YAML content from an uploaded file. The Ruleset Plan Detail Object is sent in the configuration form field, and the ruleset must be sent as a file upload.

The content type of the request must be multipart/form-data.

The returned response is a Ruleset Plan Create or Update Result Object.

POST /ifm/ruleset-plans/ with Form Content Parameters
Field Type Required Location Description
configuration Ruleset Plan Detail Object Yes Form Field A JSON encoded Ruleset Plan Detail Object, excluding ruleset_yaml.
ruleset File Upload Yes Form Field The ruleset YAML content as a file upload form field.
POST /ifm/ruleset-plans/ with Form Content curl example
curl -X POST "https://<your-datamasque-host>/ifm/ruleset-plans/" \
     -H "Authorization: Bearer <access-jwt>" \
     -H "Content-Type: application/json" \
     -F 'configuration={"name": "from_file_firstname_mask"}' \
     -F "ruleset=@ruleset.yaml"

PUT /ifm/ruleset-plans/{plan_name}/

Allowed Roles: Admin or Mask Builder

Update an existing ruleset plan with new options or a new ruleset, completely overwriting all fields. The request can be sent in two formats:

  1. As JSON with an embedded ruleset YAML string.
  2. As a multipart form with separate fields for the Ruleset Plan Update and ruleset.

Important: PUT requests replace the entire ruleset plan configuration. Any fields not included will be reset to defaults. To update specific fields while preserving existing values, use PATCH instead.

PUT /ifm/ruleset-plans/{plan_name}/ with JSON Content

Update a ruleset plan using YAML content embedded inside the Ruleset Plan Detail Object sent in the request body.

The content type of the request must be application/json.

The returned response is a Ruleset Plan Create or Update Result Object.

PUT /ifm/ruleset-plans/{plan_name}/ with JSON Content curl example

Note the ruleset_yaml must be included otherwise the request will fail validation.

curl -X PUT "https://<your-datamasque-host>/ifm/ruleset-plans/<ruleset-plan-name>/" \
     -H "Authorization: Bearer <access-jwt>" \
     -H "Content-Type: application/json" \
     -d '{"options": {"enabled": false"}, "ruleset_yaml": "version: \"1.0\"\nrules:\n- masks:\n  - type: from_file\n    seed_file: DataMasque_firstNames_mixed.csv"}'

PUT /ifm/ruleset-plans/{plan_name}/ with Form Content

Update a ruleset plan using form data. The Ruleset Plan Detail Object is sent in the configuration form field, and the ruleset must be sent as a file upload.

The content type of the request must be multipart/form-data.

The returned response is a Ruleset Plan Create or Update Result Object.

PUT /ifm/ruleset-plans/ with Form Content Parameters
Field Type Required Location Description
configuration Ruleset Plan Update Object Yes Form Field A JSON encoded Ruleset Plan Update Object, excluding ruleset_yaml.
ruleset File Upload Yes Form Field The ruleset YAML content as a file upload form field.
PUT /ifm/ruleset-plans/ with Form Content curl example

Note the ruleset must be included otherwise the request will fail validation.

curl -X PUT "https://<your-datamasque-host>/ifm/ruleset-plans/<ruleset-plan-name>/" \
     -H "Authorization: Bearer <access-jwt>" \
     -H "Content-Type: multipart/form-data" \
     -F 'configuration={"options": {"enabled": false"}' \
     -F "ruleset=@ruleset.yaml"

PATCH /ifm/ruleset-plans/{plan_name}/

Allowed Roles: Admin or Mask Builder

Update an existing ruleset plan with new options or a new ruleset, overwriting only fields that have been specified. The request can be sent in two formats:

  1. As JSON with an embedded ruleset YAML string
  2. As a multipart form with separate fields for the Ruleset Plan Update and ruleset

Important: PATCH requests only set fields that are included in the request. Any fields that are not included in the request will not be updated. To overwrite any fields that are not specified with their defaults, use PUT instead.

PATCH /ifm/ruleset-plans/ with JSON Content

Update a ruleset plan using YAML content embedded inside the Ruleset Plan Update Object sent in the request body.

The content type of the request must be application/json.

The returned response is a Ruleset Plan Create or Update Result Object.

PATCH /ifm/ruleset-plans/ with JSON Content curl example

Note that the ruleset_yaml is not included in the request, so it will remain unchanged.

curl -X PATCH "https://<your-datamasque-host>/ifm/ruleset-plans/<ruleset-plan-name>/" \
     -H "Authorization: Bearer <access-jwt>" \
     -H "Content-Type: multipart/form-data" \
     -d '{options: {"enabled": false}}'

PATCH /ifm/ruleset-plans/ with Form Content

Update a ruleset plan using form data. The Ruleset Plan Detail Object is sent in the configuration form field, and the ruleset must be sent as a file upload.

The content type of the request must be multipart/form-data.

The returned response is a Ruleset Plan Create or Update Result Object

PATCH /ifm/ruleset-plans/ with Form Content Parameters
Field Type Required Location Description
configuration Ruleset Plan Detail Object No Form Field A JSON encoded Ruleset Plan Detail Object, excluding ruleset_yaml.
ruleset File Upload No Form Field The ruleset YAML content as a file upload form field.
PATCH /ifm/ruleset-plans/ with Form Content curl example

Note that the configuration is not specified, so only the ruleset is updated.

curl -X PATCH "https://<your-datamasque-host>/ifm/ruleset-plans/<ruleset-plan-name>/" \
     -H "Authorization: Bearer <access-jwt>" \
     -H "Content-Type: multipart/form-data" \
     -F "ruleset=@ruleset.yaml"

DELETE /ifm/ruleset-plans/{plan_name}/

Allowed Roles: Admin or Mask Builder

Delete a ruleset plan.

DELETE /ifm/ruleset-plans/{plan_name}/ curl example

curl -X DELETE "https://<your-datamasque-host>/ifm/ruleset-plans/<ruleset-plan-name>/" \
     -H "Authorization: Bearer <access-jwt>"

Masking

Information about masking values.

Masking Object Formats

Masking Request Detail Object

The following fields are valid in the request body when masking values.

Field Type Required Description
data Array of any Yes The data to mask.
log_level string No Logs at this level or higher will be included in the output. Defaults to the log_level specified on the ruleset plan.
hash_values any No Values to use when generating hash seeds, if the hash source is from_request. See Hash Values below.
run_secret string No A 20-128 character string to use as the run secret for this request. See Run Secret and Instance Secret.
disable_instance_secret bool No Disable the instance secret for this request, for consistency across multiple DataMasque instances. Defaults to false.
request_id string No An ID to identify the request and response in logs or in the calling system. If unset, a UUID is generated.
charset string No Currently only utf-8 is supported.
encoding string No Currently only JSON is supported.
Hash Values

Hash values can be provided as either a single value or a list. When providing a single value (which can be a number, string, boolean, or object), it will be converted to a string and used for hashing.

When masking multiple values in the data array with a single hash value, all output values will be identical since they share the same hash.

To generate different output values, provide hash_values as an array. The array must contain the same number of elements as the data array, creating a one-to-one mapping between hash values and data values. If the lengths do not match, the masking will fail with an error.

Run Secret and Instance Secret

If no run secret is specified, DataMasque will generate and use a random run secret. The disable_instance_secret option cannot be set to true unless a run secret is also specified in the request.

There is a performance impact if changing a run secret, or switching disable_instance_secret between true and false. Please refer to Performance Considerations for Run Secrets.

Masking Response Detail Object

The following fields are valid in the response body when masking values.

Field Type Description
data Array of any The data that has been masked. Will not be present in the response if masking failed.
logs Array of Log Message objects Any logs recorded during masking, if their level matches or exceeds the log_level of the request. See Log Message Object for the format.
request_id string The request_id that was specified in the request or randomly generated.
charset string The charset that was specified in the request, or the default for the ruleset plan.
encoding string The encoding that was specified in the request, or the default for the ruleset plan.
ruleset_plan Ruleset Plan Summary object A Ruleset Plan Summary Object with information to identify which ruleset plan masked the data.

Ruleset Plan Summary Object

A simple representation of a Ruleset Plan, intended to contain just enough information to identify which Ruleset Plan masked the data.

Field Type Description
name string The name of the Ruleset Plan
serial integer The serial number of the Ruleset Plan

POST /ruleset-plans/{plan_name}/mask/

Allowed Roles: Admin, Mask Builder, or Mask Runner

Perform masking on data using a created ruleset plan, applying its ruleset.

The request body is a JSON encoded Masking Request Detail Object, and the response is a JSON encoded Masking Response Detail Object.