Governance Toolkit: PDP Automation

Intro

PDP Automation allows a user to configure a DataSet that outlines what PDP should look like for DataSet(s). Each job can run on a schedule and when the configuration DataSet updates.

Accessing the Toolkit

To access the Governance Toolkit, do the following:

1. Go to More > Admin.
2. In the Governance menu, select Toolkit. 

Required Grants

To access the Governance Toolkit a user will need 1 of 2 grants enabled for a user. Grants can be added to a custom user role. By default the Administer Custom Pipeline Process grant is enabled for Admin users.

•  Create Custom Pipeline Processes
  • This grant will allow a user to create, update, share and delete jobs.
  • This grant will only allow a user to see jobs that they created or jobs that have been shared with them.
• Administer Custom Pipeline Processes
  • This Grant will allow a user to create, update, share and delete jobs.
  • this grant will allow a user to see all jobs

To learn more about custom roles and grants see Managing Custom Roles.

Configuration DataSet

• target_ds
  • This is the DataSet id that the PDP policy should be applied to.
  • In a configuration DataSet you can have multiple DataSet ids.
• policy_name
  
  • The name of what the policy is called on the target_ds.
  • If a policy with that name does not exist a new policy will be created with this name.
• policy_column
  • The column name that contains the values you wish to filter by.
• user_group
  • User id or emails.
  • Group id or name.
  • These are the users or groups you wish to have access to the PDP policy.
  • This supports multiple values. Different values should be separated by a pipe ‘|’.
• value
  • Values to filter the policy by.
  • This supports multiple values. Different values should be separated by a pipe ‘|’.

Creating a Job

1. Navigate to the Governance Toolkit and select PDP Automation.
   • https://example.domo.com/admin/governance-toolkit/pdp-automation
2. Click + New Job at the top right.
3. Fill out the following information:
   • Job Name (required)
   • Job Description (optional)
   • Configuration DataSet (required)
     • This is the DataSet that defines what policies you wish to update or create.
   • Orphaned policy management (required)
     • Orphaned policies refer to existing PDP policies on your DataSet defined in your configuration DataSet, but policies that are not being updated in your configuration DataSet.
     • Keep
       • Will leave those existing policies in place with no changes.
     • Delete
       • Will delete those policies that exist on the DataSet but not in the configuration file.
   • Run Logs (required)
     • Each job will keep track of executions of the job and information surrounding that.
     • New DataSet
       • Will create a new DataSet in your Domo instance and all runs of the job will write logging to the new DataSet.
     • Existing DataSet
       • Existing DataSet will allow you to combine multiple PDP Automation jobs into a single logging DataSet.
       • The DataSet must be a DataSet created by the PDP Automation tool.
         • DataSet must also already contain rows of data.
     • Email notifications (optional)
       • You can add user emails to this field.
       • Each email will receive an email notification from Domo each time the job runs.
       • The notification will outline what the job did during the run and any failures that occurred.

Running a Job

Each job has an Enabled Toggle. This will control how a job is run.

• Preview Only
  • The job will only execute when the job is manually run.
  • When the job is run it will not update the actual target DataSet(s) from your configuration file. It will update the log DataSet with all the changes it would have made to the DataSet.
    • This allows you to validate the job before it updates your DataSets.
• Active
  • This will update the PDP policies on your DataSet(s).
  • The job will execute every six hours.
  • The job will execute every time the configuration DataSet is updated.

Example of Job Run

• PDP Before run:
  
  
  
  
• Configuration DataSet:
  
  
  
  
• PDP after run:
  
  
  
  
• Logging DataSet:
  
  

Using Dynamic Policies

Dynamic policies can be configured in your configuration file.

To add a dynamic policy, set the ‘policy_column’ & set the ‘value’ column to the appropriate “Trusted Attribute”. 

List of Trusted Attributes

If you would like to ignore the case on a dynamic policy you can add “:ignore_case” to the end of the trusted attribute in your value column.

Using Multiple Filters on a Single Policy

To add multiple filters, set the ‘policy_name’ & ‘user_group’ to the same values.

NOTE: It is very important to order this DataSet by the `target_ds` column and then the `policy_name` column. This will ensure all the actions happen on the same dataset and the same policy.