Using the Federated Data Solution

Intro

Domo's Federated Data model allows you to directly render cards and other user-level interactions in the Domo platform using a database hosted outside of Domo's cloud. Instead of executing the visualization query against Domo, the query is executed at the target database and the results are sent to Domo to render the visualization. This model is especially relevant for users who would like to keep their data at rest in their infrastructure for security or logistical reasons.

We have a new streamlined experience for cloud-native federated data warehouses, starting with Snowflake. See more details here.

Benefits

This solution provides the following benefits:

• Preserves existing data warehouse investment by allowing for direct connection to customer data lakes and warehouses.

• No upload to cloud storage.

  • Avoids data duplication and costly data transfers.

  • Reduces data latency due to transfers.

• Domo has implemented the dedicated adapters (PostgreSQL, Snowflake, Teradata, Redshift, etc.)

• Customer admins retain control of the data.

• Custom database optimizations can be performed by the customer.

Federated Query may be a great fit if you meet the following criteria:

• You have extensive EDW investments.

• You do not want to move data out of your existing data warehouse for compliance reasons.

Drawbacks

Before using this solution, be aware of the following:

• Query performance depends on the configuration of the customer’s data warehouse, its performance profile and the query load it experiences. This is managed outside of Domo. 

• Domo Federated Query cannot handle any of the following:

  • ETL, including Data Science actions (as an alternative, you can create views within the on-premise DataSet.)

  • DataSet Alerts (Card Alerts are supported on Snowflake federated DataSets.)

  • DataFusions

  • DataSet Views Beta. (Supported on Snowflake Federated DataSets.)

Architecture

Security

Domo Federated Query supports federated queries between the Domo cloud and your on-premise data stores, without duplicating data. This ensures that your data remains protected in your on-premise environment. Only the subset of data that you query is passed to the Domo cloud, after which it is immediately deleted from memory after being processed.

Domo Federated Query supports a defense-in-depth model to help ensure that your Data remains protected in all stages of its lifecycle, including data in-transit and at rest. 

Domo Federated Query allows you to always be in control of your data. You can initiate the connection, determine each type of data to share, and define the exact data time-to-live.

The Federated Model requires the customer to identify and install the agent on a machine in their environment. 

Data in Transit

All traffic between the Domo cloud and your on-premise data warehouse is encrypted. Your own firewalls can control all inbound and outbound traffic and strong authentication is in place to help ensure the integrity of the system accounts. Furthermore, you own and manage the Federated Data Client server, which means that you initiate and control each data action.

Data at Rest 

Domo Federated Query ensures that your Domo cloud data is ephemeral. A TTL (time-to-live) is associated with all customer data, and once the TTL expires, so too does the customer data in memory within the Domo cloud. This customer data TTL is configurable by you and it can be as short as 5 seconds. The customer data TTL has significant benefits as it allows you to simply disable data access, which results in all customer data to be inaccessible in Domo.

Setup and Configuration

Prerequisites

• A machine must be in place for the agent to run on and must have access to the target database.

• The necessary JDBC driver for the target instance must be available on the server. The path to the jar file is required in the configuration.

• You must have a connection string to the target database, including the database URL, username, and password.

• You must have the following Domo connection information:

  • Domo domain (i.e. customer.domo.com)

  • Domo credentials to OAuth 2.0 or a Domo auth token (generated in Admin > Security > Access tokens)

Access and Enablement

Contact your CSM for information about downloading the Federated package.

FAQ

How long does it take to set up the federated agent?

Once the prerequisites are in place, it generally takes under an hour to get up and running.

What databases are supported?

The current list of databases includes the following:

• Amazon Redshift

• AWS Athena

• DB2

• Google BigQuery

• MemSQL

• MySQL

• Oracle

• Postgres

• Presto

• SAP Hana

• Snowflake*

• SQL Server

• Stratio Crossdata

• Teradata

• Vertica

*We recommend using the new cloud native federated experience for Snowflake. See details here.

What if the database that I need is not in the currently supported list?

Discuss with your CSM to find out what options are available. Generally, new adapters take 2-3 weeks to develop. Customers can develop their own adapters if needed.

Will there be a performance impact to the card loading times?

The additional time added by the federated agent is negligible. Any performance impact will be related to the ability of the target server to process the queries.

What options are available to manage the load on the server?

Domo caches the results of queries when possible to be as efficient as possible. Cache settings are configurable in the Federated adapter.

Does using the Federated agent require allowing Domo through our firewall?

No, the Federated agent uses a web socket to establish a connection to Domo. No inbound traffic is required to use the Federated adapter.

What does a Federated DataSet look like in Domo?

It looks like a normal DataSet. It displays row counts and the schema, but the actual data is not copied to Domo.

Where do I go if I have additional questions?

Reach out to Domo Support or your CSM.