R and Python Integration

R and Python are powerful languages that can be used for more advanced statistical data manipulation such as predictive analytics or to create more specific chart formats. With the R and Python integration, Periscope will automatically pull the results of your SQL into R and Python to enable statistical analysis, all within the chart editor.

Note: R and Python Integration is an add-on feature. Site administrators can contact their Account Manager for additional information.

<div><UL>
<LI><a href="#Libraries">Available Libraries</a></LI>
<LI><a href="#PythonChart">Adding Python to a Chart</a></LI>
<LI><a href="#RChart">Adding R to a Chart</a></LI>
<LI><a href="#CodeView">Adding Code to a View</a></LI>
<LI><a href="#CodeTemplate">Code Templates</a></LI>
<LI><a href="#LimitPreview">Limit Preview</a></LI>
</UL></div>

<HR>

<a name="Libraries"></a>

Available Libraries

Additionally, you can easily access the libraries available to import from the schema browser and link directly to the library docs.

Python 2.7 and Python 3.6 supported libraries include:

R supported libraries:

More libraries will be added as they are requested and reviewed. Please reach out to Periscope Data support to request a library addition at support@periscopedata.com or over the in-app live chat.

<a href="#top">Back to top</a>

<a name="PythonChart"></a>

Adding Python to a Chart

Once a SQL query has been successfully run for a new chart, select the desired language from the dropdown below the SQL editor and click the '+' icon to add a block of code. Periscope currently supports Python 2.7, Python 3.6 and R (see below). 

For Python, the pandas library has already been imported pd, with the result set of the chart imported as a pandas dataframe variable called "df". 

In the schema, you can easily see the data frame columns along with the data type.

If the code outputs another dataframe (df2) to be used by Periscope Data's charting functionality, the dataframe can be passed to periscope.output with the following syntax:

periscope.output(df2)

Similarly, for code that generates a plot (named plt), the plot can be passed to periscope.output like this, which would replace the common plt.show():

periscope.output(plt)

Data and text can also be printed to the Standard Out. Switching between the Periscope Output and the Standard Out can be done with the tabs below the code block Run button.

To use the Periscope Output or the Standard Out directly in a chart, select "Python Image" or "Python Text" respectively in the chart type drop down.

<a href="#top">Back to top</a>

<a name="RChart"></a>

Adding R to a Chart

Select R from the dropdown below the SQL editor and click the '+' icon to add a block of code.

For R, the result set of the chart is imported as a dataframe variable called "df". 

If the code outputs another dataframe (named df2) to be used by Periscope Data's charting functionality, the dataframe can be passed to periscope.table with the following syntax:

periscope.table(df2)

Similarly, for code that generates a plot, the plot can be output by calling periscope.image() after plotting. For images, the Periscope chart will scale the size of the image according to the chart's bounds. To define an image's height and width, an optional png function call can override the defaults:

plot(df)
png(width=500, height=500, unit='px')
periscope.image()

Lastly, for code that generates text (named txt), the text can be passed to periscope.text like this:

periscope.text(txt)

Data and text can also be printed to the Standard Out. Switching between the Periscope Output and the Standard Out can be done with the tabs below the code block Run button.

To use the Periscope Output or the Standard Out directly in a chart, select "R Image" or "R Text" respectively in the chart type drop down.

<a href="#top">Back to top</a>

<a name="CodeView"></a>

Adding Code to a View

Note: R and Python Integration with Views is only available with the Cache or Warehouse Infrastructure as well as the R and Python Integration add-on feature. Site administrators can contact their Account Manager for additional information.

Adding code to a view is very similar to adding code to charts. Once a SQL query has been successfully run for a new view, select the desired language from the dropdown below the SQL editor and click the '+' icon to add a block of code. Periscope currently supports Python 2.7, Python 3.6 and R.

For Python, the pandas library has already been imported as pd, with the result set of the chart imported as a pandas dataframe variable called "df".

For R, the result set of the chart is imported as a dataframe variable called "df".

The code attached to a view should output another dataframe (df2) which will be saved as a table on the Periscope Cache or Warehouse to be used across the site in charts. This final dataframe can be passed to periscope.materialize() for materialization into the Periscope Cache with the following syntax:

periscope.materialize(df2)

Data and text can also be printed to the Standard Out, which works the same in both charts and views. Switching between the Periscope Output and the Standard Out can be done with the tabs below the code block Run button. The above mentioned periscope.output() function (for Python), and periscope.table(), periscope.image() and periscope.text() functions (for R) will all also be available to enhance the development experience.

The code environment for executing Python and R for views has a 500mb limit for both input data size and output data size. Materialization requires the the input SQL be run on the Periscope Data Engine. For the materialization process, the SQL code and the R or Python code block are each given 30 minutes to run. View previews are given the same time as charts. Views with code are kept fresh with the same refresh logic as views without code.

For more information on views, including how to create a view, please see the documentation here.
For more information on materializing views on the Periscope Data Engine, see the documentation here.

<a href="#top">Back to top</a>

<a name="CodeTemplate"></a>

Code Templates

Code templates are available for common analyses based on popular posts from our Community. These Include the following templates:

  • Regression Analysis
  • Transpose Table
  • Pivot Table
  • Melt Table
  • Ranged Calculations
  • Confidence Interval
  • Funnel Chart
  • Statistical Significance (in R only)

To select a code template, first select the language (Python 2.7, Python 3.6, or R). Then, click on New Analysis and select the desired template. The code for the template will be in the selected language.

Next, click the green plus to add the code for the selected template. In this case, code for regression analysis.

This will display the template code with a brief description and comments to aid in using the template.

To request a template, please post an example in our Community or reach out to Periscope Data support at support@periscopedata.com or over the in-app live chat.

<a href="#top">Back to top</a>

<a name="LimitPreview"></a>

Limit Preview

Most browsers can only handle visualizations of up to 5MB, so to keep dashboards running smoothly, visualization results are capped. Once a code block for R or Python is added to a chart, the “Limit Preview” setting for charts will only apply to the SQL Output preview. The “Limit Preview” setting will not affect the rows or size of the dataframe sent to the R or Python script, allowing the code to run on the full result set within a 500MB limit. If the R or Python code returns over 5MB of data, an error will be surfaced and a hard-coded limit or aggregation, outside of the “Limit Preview” setting, should be added to the SQL or code blocks.

<a href="#top">Back to top</a>

Our support team is ready to help