Git Integration

👤 This documentation is intended for Site Administrators.

With Git Integration, Periscope provides data professionals with complete control over their analytics environment, offering a sophisticated version control system, file-level access to user generated content, and support for release management workflows. Unlike other analytics solutions, Periscope provides a complete bi-directional sync for all user generated content including Dashboards, Charts, Views, Snippets, Filters and Alerts and works with any remote Git repository, including an owned Git server.

Table of Contents:

<UL>
<LI><a href="#GitUseCases">Git Use Cases</a></LI>
<LI><a href="#HowItWorks">How It Works</a></LI>
<LI><a href="#GitConfigure">Configuring Git</a></LI>
<LI><a href="#RepoStructure">Repository Structure</a></LI>
<LI><a href="#ConfigFiles">Configuration Files</a></LI>
<LI><a href="#GitDisconnect">Disconnecting Git</a></LI>
<LI><a href="#GitCommits">Git Commits in the History Tab</a></LI>
<LI><a href="#Notes">Notes</a></LI>
<LI><a href="#AppendA">Appendix A - GitHub</a></LI>
<LI><a href="#AppendB">Appendix B - GitLab</a></LI>
<LI><a href="#AppendC">Appendix C - BitBucket</a></LI>
</UL>

Notes:

<UL>
<LI>Git Integration is available on select plans. Site administrators can contact their Account Manager for additional information.</LI>
With the mid-September 2018 release, Git Integration will support Snippets and Views. Support for all user generated content (Dashboards, Charts, Filters, Alerts) is coming in the last half of Q4 2018.
</UL>

<HR>

<a name="GitUseCases"></a>

Git Use Cases

<OL>
<LI><b>Sophisticated, externalized version control with code review</b></LI>
<UL><LI>Git integration gives users all of the great features and functionality of Git as a version control system, in addition to providing full, external access to all user generated content.</a></LI></UL>
<LI><b>Automate content creation, search & replace, and clone across Spaces</b></LI>
<UL><LI>With user generated content represented externally as a set of files in a Git repository, users have the power and flexibility to perform any kind of batch processing and automation on their data. Some examples include programmatically creating, editing, and deprecating dashboards, finding and replacing across many files, cloning objects across Spaces, and much more.</b></LI></UL>
<LI><b>Release Management</b></LI>
<UL><LI>Ensure a reliable and stable production environment by previewing and collaborating on changes on a branch prior to pushing live to production.</LI></UL>
</OL>

<a href="#top">Back to top</a>

<a name="HowItWorks"></a>

How It Works

<OL>
<LI>User creates an empty remote repository (GitHub, GitLab, Bitbucket, corporate Git Server, etc) and links it to a Space in Periscope (one repository per Space)</LI>
<LI>Once configured, a two way sync is established between the Space in Periscope and the user's remote repository (all supported objects will be synced automatically on initial sync and going forward)</LI>
<LI>Any changes made to the supported objects in Periscope are instantly synced to the user's remote repository</LI>
<LI>Any changes merged into the periscope/master branch in the user's remote repository are instantly synced to Periscope</LI>
</OL>

<a href="#top">Back to top</a>

<a name="GitConfigure"></a>

Configuring Git

<OL>
<LI>Create an empty Git repository (without a README) hosted at GitHub, GitLab, BitBucket or on an owned Git server.</LI>
<UL>
<LI>If starting with a repository that is not empty and there is a desire to merge from <code>origin/master</code> to <code>periscope/master</code>, issue the following command to allow unrelated histories: <code>git merge --allow-unrelated-histories periscope/master.</code></LI>
<LI>Make sure that the repository is set to Private so that access can be controlled (this is a setting that can be changed in the repository).</LI>
</UL>
<LI>Navigate to the Version Control settings page in Periscope</LI>
<UL>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92af4725711df2824da43f_GitVersionControlSetting.png"></div></figure></LI>
</UL>
<LI>Get the SSH URL for the remote Git repository. Formats for common Git providers:</LI>
<UL>
<LI>To see screenshots of how to configure GitHub, please see <a href="#AppendA">Appendix A - GitHub</a></LI>
<UL><LI>GitHub: git@github.com:&lt;organization-name&gt;/&lt;repository-name&gt;.git</LI>
<LI>GitHub Enterprise: git@example.com:&lt;organization-name&gt;/&lt;repository-name&gt;.git</LI>
</UL>
<LI>To see screenshots of how to configure GitLab, please see <a href="#AppendB">Appendix B - GitLab</a></LI>
<UL><LI>GitLab: git@gitlab.com:&lt;organization-name>/&lt;repository-name&gt;.git</LI></UL>
<LI>To see screenshots of how to configure BitBucket, please see <a href="#AppendC">Appendix C - BitBucket</a></LI>
<UL><LI>Bitbucket: git@bitbucket.org:&lt;organization-name&gt;/&lt;repository-name&gt;.git</LI></UL>
</UL>
<LI>Paste the SSH URL into the “SSH URL” field on the Version Control settings page:</LI>
<UL>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92b0ff25711d53654dac91_GitVersionSSH.png"></div></figure></div></LI>
</UL>
<LI>Copy the “Deploy Key” which Periscope provides on the Version Control settings page and paste it into the Settings for the repository:</LI>
<UL>
<LI>Be sure to select the <b><u>Allow write access</b></u> option because Periscope requires this setting in order to make and deploy changes. </LI>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92b0f836e87ed20ae800ea_GitVersionDeploy.png"></div></figure></LI>
</UL>
<LI>Navigate back to the Version Control settings page in Periscope, Copy the “Webhook URL” and “Webhook Secret Token” and paste them into the Settings for the repository:</LI>
<UL>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92b0ef2e799bfa4134fcb4_GitVersionWebhooksectoken.png"></div></figure></LI>
</UL>
<LI>Navigate back to the Version Control settings page and click “Connect”.</LI>
<LI>Once the connection is successfully established, a new <code>periscope/master</code> branch appears in the repository, along with a set of folders, one for each object type that is synced. The Periscope application will sync with the <code>periscope/master</code> branch in the repository as the Production branch. Any changes made in the application will be synced to the <code>periscope/master</code> branch and any changes merged into the <code>periscope/master</code> branch in Git will be reflected in the Periscope application.</LI>
</OL>

<a href="#top">Back to top</a>

<a name="RepoStructure"></a>

Repository Structure

The repository will be structured as follows:

<blockquote><br>
snippets/<br>
├── snippet_name_1/         <br>
|   ├── snippet_name_1.yaml     //The settings for this Snippet <br>
|   ├── snippet_name_1.sql      //The SQL query for this Snippet<br>
├── snippet_name_2/<br>
|   ├── ...<br>
views/<br>
├── view_name_1/<br>
|   ├── view_name_1.yaml     //The settings for this View   <br>
|   ├── view_name_1.sql      //The SQL query for this View<br>
├── view_name_2/<br>
|   ├── ...<br>
</blockquote><br>

<a href="#top">Back to top</a>

<a name="ConfigFiles"></a>

Configuration Files

Configuration files in the Git repository reflect all the settings which can be defined for a given object. It is not necessary to define values for all settings since Periscope will interpret missing settings in the YAML file as being set to their default value. However, there are certain fields that are required to be defined which are denoted below.

View Settings YAML

<br><blockquote><br>
Database Name: warehouse    #Required<br>
Settings:<br>
 Materialize View: True     #Required<br>
 Publish as Dataset: True<br>
 Exclude from Auto-Archive: False <br>
 Metadata:<br>
   Tags:<br>
     - Tag1<br>
     - Tag2<br>
     - Tag3<br>
   Description: This is how I describe this view.<br>
   Owner: owner@periscopedata.com     #Required<br>
</blockquote><br>

Snippet Settings YAML

<br><blockquote><br>
Parameters:<br>
 - numerator<br>
 - denominator<br>
Settings:<br>
 Metadata:<br>
   Tags:<br>
     - Tag1<br>
     - Tag2<br>
     - Tag3<br>
   Description: This is how I describe this view.<br>
   Owner: owner@periscopedata.com    #Required<br>
</blockquote><br>

<a href="#top">Back to top</a>

<a name="GitDisconnect"></a>

Disconnecting Git

This feature is still in development.

Site administrators can contact support@periscopedata.com if they would like to disconnect a repo. Please note that this may take a few days to complete.

<a href="#top">Back to top</a>

<a name="GitCommits"></a>

Git Commits in the History Tab

This feature is still in development, however, is currently available with a limited feature set. Both changes made in-app as well as from your Git repository will appear in the same format in the History tab.

<a href="#top">Back to top</a>

<a name="Notes"></a>

Notes

<UL>
<LI>In order to rename objects in Periscope from Git, change the name of the folder in Git which corresponds to that object. Example: if changing the name of a Snippet named “<code>my_snippet</code>” to “<code>my_new_snippet</code>”, change the name of the “<code>my_snippet</code>” folder in Git to “<code>my_new_snippet</code>”. It is not necessary to change the names of the files within that folder, “<code>my_snippet.sql</code>” and “<code>my_snippet.yaml</code>”. However, if the files are not also renamed, any subsequent save to that object in Periscope will rename those files to match the new name, which will show up in the commit history in Git.</LI>
<LI>Periscope places <code>.gitkeep</code> files in the top-level directories of the repository to ensure that the directories exist even if they're empty, since empty directories do not show by default.</LI>
<LI>Snippets with invalid characters in their name, will not successfully sync to your Git repository. For a list of invalid characters, please reference our Snippets documentation <a href="https://doc.periscopedata.com/article/snippets">here</a></LI>
<UL>
<LI style="list-style: none;">Any Snippets that did not get synced to the Git repository can be found on the Version Control Settings page in Periscope:</LI>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5babfddc3b59cff9f50eb6bb_gitunsuccessfulsyncss.png"></div></figure></LI>
</UL>
</UL>

<a href="#top">Back to top</a>

<HR>

<a name="AppendA"></a>

Appendix A - GitHub

<OL>
<LI>Create an empty Git repository (without a README) hosted at GitHub.</LI>
<UL>
<LI>If starting with a repository that is not empty and there is a desire to merge from origin/master to periscope/master, issue the following command to allow unrelated histories: <code>git merge --allow-unrelated-histories periscope/master.</code></LI>
<LI>Recommendation is that the repository is set to Private so that access can be controlled (this is a setting that can be changed in the repository). Note: A premium GitHub account is required in order to create Private repositories.</LI>
</UL>
<LI>Get the SSH URL for your remote Git repository. Formats for GitHub:</LI>
<UL>
<LI>GitHub: git@github.com:&lt;organization-name&gt;/&lt;repository-name&gt;.git</LI>
<LI>GitHub Enterprise: git@example.com:&lt;organization-name&gt;/&lt;repository-name&gt;.git</LI><br>
<LI>For new GitHub repositories (repos that don't yet contain any new files), GitHub will show the SSH URL as part of the initial setup. On the Code tab, click the SSH button to get the SSH URL, then click the copy button to copy it to your clipboard </LI>
<UL><LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92eec039ac89dcc39cf8cc_GitHubNewSSH.png"></div></figure>
</LI></UL>
<LI>For existing GitHub repositories (repos that already contain files), the SSH URL can be found by clicking on the Code tab, and click the 'Clone or download button'. Make sure to click the 'Use SSH' link, then click the copy button to copy it to your clipboard.</LI>
<UL><LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92eeb704c9740158351ad7_GitHubExistingSSH.png"></div></figure></LI></UL>
</UL>
<LI>Paste the SSH URL from GitHub into the “SSH URL” field on the Version Control settings page:</LI>
<UL>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92b0ff25711d53654dac91_GitVersionSSH.png"></div></figure></div></LI>
</UL>
<LI>Copy the “Deploy Key” which Periscope provides on the Version Control settings page and paste it into the Settings for the repository:</LI>
<UL>
<LI>Be sure to select the <b><u>Allow write access</b></u> option because Periscope requires this setting in order to make and deploy changes. </LI>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92eeca7794ec6b8a74e1b2_GitHubDeployKey.png"></div></figure></LI>
</UL>
<LI>Navigate back to the Version Control settings page in Periscope, Copy the “Webhook URL” into Payload URL and “Webhook Secret Token” into Secret in the GitHub repository settings, either option for Content type will work:</LI>
<UL>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92eed26708aa78e7e0f1cc_GitHubWebhookSettings.png"></div></figure></LI>
</UL>
<LI>Navigate back to the Version Control settings page and click “Connect”.</LI>
<LI>Once the connection is successfully established, a new <code>periscope/master</code> branch appears in the repository, along with a set of folders, one for each object type that is synced. The Periscope application will sync with the <code>periscope/master</code> branch in the repository as the Production branch. Any changes made in the application will be synced to the <code>periscope/master</code> branch and any changes merged into the <code>periscope/master</code> branch in Git will be reflected in the Periscope application.</LI>
</OL>

<a href="#top">Back to top</a>

<HR>

<a name="AppendB"></a>

Appendix B - GitLab

<OL>
<LI>Create an empty Git repository (without a README) hosted at GitLab.</LI>
<UL>
<LI>If starting with a repository that is not empty and there is a desire to merge from origin/master to periscope/master, issue the following command to allow unrelated histories: <code>git merge --allow-unrelated-histories periscope/master.</code></LI>
</UL>
<LI>Get the SSH URL for your remote Git repository:</LI>
<UL>
<LI>GitLab: git@gitlab.com:&lt;organization-name>/&lt;repository-name&gt;.git</LI>
<UL><LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b931b3c109adb570e889c53_GitLabSSH.png"></div></figure></LI></UL>
</UL>
<LI>Paste the SSH URL from GitLab into the “SSH URL” field on the Version Control settings page:</LI>
<UL>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b92b0ff25711d53654dac91_GitVersionSSH.png"></div></figure></div></LI>
</UL>
<LI>Copy the “Deploy Key” which Periscope provides on the Version Control settings page and paste it into the Settings for the repository:</LI>
<UL>
<LI>Be sure to select the <b><u>Write access allowed</b></u> checkbox because Periscope requires this setting in order to make and deploy changes. </LI>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b931b446708aadb74e129d0_GitLabDeployKey.png"></div></figure></LI>
</UL>
<LI>Navigate back to the Version Control settings page in Periscope, Copy the “Webhook URL” into URL and “Webhook Secret Token” into Secret Token in the GitLab repository settings:</LI>
<UL>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b931b4d7794ecd61a750c89_GitLabWebHook.png"></div></figure></LI>
</UL>
<LI>Navigate back to the Version Control settings page and click “Connect”.</LI>
<LI>Once the connection is successfully established, a new <code>periscope/master</code> branch appears in the repository, along with a set of folders, one for each object type that is synced. The Periscope application will sync with the <code>periscope/master</code> branch in the repository as the Production branch. Any changes made in the application will be synced to the <code>periscope/master</code> branch and any changes merged into the <code>periscope/master</code> branch in Git will be reflected in the Periscope application.</LI>
</OL>

<a href="#top">Back to top</a>

<HR>

<a name="AppendC"></a>

Appendix C - BitBucket

Note: For BitBucket, the Webhook Secret Token that Periscope provides is not needed. Additionally, the setup is a little different as the Bitbucket user must be commissioned with the Periscope Deploy Key and the Repository needs to be set to access the Webhook URL.

<OL>
<LI>Create an empty Git repository (without a README) hosted at BitBucket.</LI>
<UL>
<LI>If starting with a repository that is not empty and there is a desire to merge from origin/master to periscope/master, issue the following command to allow unrelated histories: <code>git merge --allow-unrelated-histories periscope/master.</code></LI>
</UL>
<LI>Get the SSH URL for your remote Git repository:</LI>
<UL>
<LI>Bitbucket: git@bitbucket.org:&lt;organization-name&gt;/&lt;repository-name&gt;.gi</LI>
<UL><LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b98305753114f2fa48c0e72_BitBucketRepoCopySSHURL1.png"></div></figure></LI>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b983061607a093d98d1e322_BitBucketRepoCopySSHURL2.png"></div></figure></LI>
</UL>
</UL>
<LI>Paste the SSH URL from BitBucket into the “SSH URL” field on the Version Control settings page. (BitBucket will copy over the ‘git clone’ command if you click the “Copy” button, so please ensure to delete the 'git clone' command from the URL when copying into Periscope):</LI>
<UL>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b98304253114fb5938c0e63_PeriscopeBBsettings.png"></div></figure></LI>
</UL>
<LI>Copy the “Deploy Key” which Periscope provides on the Version Control settings page and paste it into the Settings for the BitBucket User:</LI>
<UL>
<LI>Go to the BitBucket <b><u>User</u></b> settings by navigating to the User in the lower left-hand corner, then to SSH keys under the Security section. Add a new key here:</LI>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b98307853114f50128c0f94_BitBucketUserSettings.png"></div></figure></LI>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b983081ad24e275fff13558_BitBucketAddSSHkey-User1.png"></div></figure></LI>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b98308c518beaf4175ab847_BitBucketAddSSHkey-User2.png"></div></figure></LI>
</UL>
<LI>Navigate back to the Version Control settings page in Periscope, Copy the “Webhook URL” into the BitBucket Repository Settings by going to Settings, then Webhooks under the Workflow Section, then Add webhook:</LI>
<UL>
<LI style="list-style: none;"><figure class="w-richtext-figure-type-image w-richtext-align-center" data-rt-type="image" data-rt-align="center"><div><img src="https://uploads-ssl.webflow.com/58fe8f93dc9e750ca84ebb16/5b98306dd05a700bb5ab4ddf_BitBucketAddWebhookRepoSettings.png"></div></figure></LI>
</UL>
<LI>Navigate back to the Version Control settings page and click “Connect”.</LI>
<LI>Once the connection is successfully established, a new <code>periscope/master</code> branch appears in the repository, along with a set of folders, one for each object type that is synced. The Periscope application will sync with the <code>periscope/master</code> branch in the repository as the Production branch. Any changes made in the application will be synced to the <code>periscope/master</code> branch and any changes merged into the <code>periscope/master</code> branch in Git will be reflected in the Periscope application.</LI>
</OL>

<a href="#top">Back to top</a>

Our support team is ready to help