Infrastructure Options

👤 This documentation is intended for Site Administrators. For SQL Users looking to query the Cache, refer to the documentation listed here.

Periscope Data offers infrastructure options which include Direct Connect, Cache and Warehouse. To enable Cache or Warehouse, site administrators can reach out to their Customer Success Manager.

<div><UL>
<LI><a href="#DirectConnect">Direct Connect Infrastructure</a></LI>
<LI><a href="#Cache">Cache Infrastructure</a></LI>
<UL>
<LI><a href="#Redshift">Redshift Cache</a></LI>
<LI><a href="#Snowflake">Snowflake Cache</a></LI></UL><LI><a href="#Warehouse">Warehouse Infrastructure</a></LI><LI><a href="#DataEngine">Data Engine on Your Warehouse Infrastructure</a></LI><LI><a href="#Summary">Summary of Infrastructure Solutions</a></LI>
</UL></div>

<HR>

<a name="DirectConnect"></a>

Direct Connect Infrastructure

Instances that do not have a cache or warehouse integration have a direct connect infrastructure. With this infrastructure, queries from Periscope run directly against the origin database. Query speed is dependent upon the load on the database and any downtimes due to maintenance, replication, or other processes. Supported connections can be found here.

<a name="Cache"></a>

Cache Infrastructure

The Cache Infrastructure is part of the Periscope Data Engine and is an add-on feature. In the Cache infrastructure, site administrators have the ability to store copies of their tables in a Redshift Cache or in a Snowflake Cache using a designated caching strategy. A list of supported Cache integrations can be found here.

The Cache infrastructure is preferred among users who seek enterprise-scale hardware, but want to avoid the associated overhead. Users have the option to run their queries on the uncached versions of their tables (similar to a direct connect infrastructure) in addition to running queries on the cache. The Cache infrastructure offers the following benefits:

➤ Businesses that prefer the Cache Infrastructure:
  • Seek enterprise-scale hardware without the associated overhead
  • Require intensive compute resources that cannot be managed by the origin database alone
  • Seek to consolidate insights from multiple data sources

<a name="Redshift"></a>

Redshift Cache

The Redshift Cache refers to the Data Engine functionality built on AWS Redshift clusters. The Redshift Cache is most beneficial to users that:

  • Prefer to be in the Amazon ecosystem
  • Run relatively low complexity analytics
  • Can keep a Redshift node busy for almost 100% of the time

<a name="Snowflake"></a>

Snowflake Cache

The Snowflake Cache refers to the Data Engine functionality built on Snowflake Virtual Warehouses. The Snowflake Cache is most beneficial to users that:

  • Store a lot of data but have relatively low compute demand
  • Have a wide variability in compute demand
  • Need query speed to be as fast as possible

<a href="#top">Back to top</a>

<a name="Warehouse"></a>

Warehouse Infrastructure

The Warehouse infrastructure gives site administrators access to a private AWS Redshift cluster. Through the Cluster Info console, site administrators also have access to cluster credentials to establish ETL pipelines to connect virtually any data source to Periscope. The Warehousing infrastructure is preferred among enterprise businesses, as it offers maximum privacy for companies with strict regulatory and security requirements, and best supports instances with intensive computing requirements. In addition to the benefits associated with the Cache infrastructure, the Warehouse infrastructure offers the following functionalities:

  • Access to cluster credentials to connect virtually any data store to Periscope via a third party ETL tool
  • Ability to customize table ingest methods beyond Periscope's offered caching strategies through ETL tools or custom ingest scripts
  • View dependency mapping
  • Full access to cluster resources
  • Cluster health and alerting
  • Ability to choose a desired AWS region
➤ Businesses that prefer the Warehouse Infrastructure:
  • Have intensive computing requirements that require exclusive access to a private cluster
  • Seek a solution in accordance with strict regulatory and security measurements
  • Require analyses pulling from a wider diversity of data sources through third-party ETL partners

<a href="#top">Back to top</a>

<a name="DataEngine"></a>

Data Engine on Your Warehouse Infrastructure

Periscope currently supports this with two database types:

  1. Redshift
  2. Snowflake

The Data Engine on Your Warehouse infrastructure allows site administrators to leverage their existing warehouse within Periscope. Site administrators retain ownership and full access to their existing warehouse while gaining the ability to materialize views directly on their warehouse with Periscope Views. The Data Engine on Your Warehouse infrastructure is preferred among companies with the most strict data security requirements, companies with a need to maintain an existing source of truth, and companies that prefer to use their own ETL strategies to the caching strategies offered with Cache and Periscope Managed Warehouse. The Data Engine on Your Warehouse infrastructure offers the following benefits:

  • Full ownership of data warehouse
  • Maximum data security and compliance
  • Flexible view materialization scheduling
  • Ability to upload CSVs directly to existing warehouse
  • Ability to maintain custom ingest methods on existing warehouse
➤ Businesses that prefer the Warehouse Infrastructure:
  • Already have a Redshift or Snowflake data warehouse that pulls from a variety of sources
  • Have security and compliance requirements that require data be kept on owned databases
  • Prefer flexible view materialization scheduling to Cache an Warehouse table caching

<a href="#top">Back to top</a>

<a name="Summary"></a>

Summary of Infrastructure Solutions

<div>
<table>
<thead>
<tr>
<th style="width:200px;text-align:center"> </th>
<th style="width:200px;text-align:center">Direct Connect</th>
<th style="width:200px;text-align:center">Cache</th>
<th style="width:300px;text-align:center">Managed Warehouse</th>
<th style="width:300px;text-align:center">Data Engine on Your Warehouse</th>
</tr>
</thead>
<tbody>
<tr>
<td>CSV Uploads</td>
<td></td>
<td align="center">&#10003</td>
<td align="center">&#10003</td>
<td align="center">&#10003</td>
</tr>
<tr>
<td>Full Table Caching</td>
<td></td>
<td align="center">&#10003</td>
<td align="center">&#10003</td>
<td></td>
</tr>
<tr>
<td>Basic Materialized Views</td>
<td></td>
<td align="center">&#10003</td>
<td align="center">&#10003</td>
<td align="center">&#10003</td>
</tr>
<tr>
<td>Scheduled Materialized Views</td>
<td></td>
<td></td>
<td></td>
<td align="center">&#10003</td>
</tr>
<tr>
<td>Integrations With Third-Party ETL Tools</td>
<td></td>
<td> </td>
<td align="center">&#10003</td>
<td align="center">&#10003</td>
</tr>
<tr>
<td>Dependency Mapping</td>
<td></td>
<td> </td>
<td align="center">&#10003</td>
<td> </td>
</tr>
<tr>
<td>Simplified Cluster Health and Alerting</td>
<td></td>
<td> </td>
<td align="center">&#10003</td>
<td> </td>
</tr>
<tr>
<td>Full Cluster Management and Ownership</td>
<td></td>
<td> </td>
<td> </td>
<td align="center">&#10003</td>
</tr>
<tr>
<td>Ability to Select Desired AWS Region</td>
<td></td>
<td> </td>
<td align="center">&#10003</td>
<td align="center">&#10003</td>
</tr>
</tbody>
</table>
</div>

<a href="#top">Back to top</a>

Our support team is ready to help