New products. New navigation bar. Read up here!

Silo Discovery

Silo Discovery leverages pre-built plugins to scan, discover, and automatically catalog all data silos.

In order to keep your data inventory up to date, Transcend automatically detects new data systems at your company, through our set of Silo Discovery plugins.

Transcend's Silo Discovery plugins are the easiest way to ensure your Data Inventory is both comprehensive and up-to-date on an ongoing basis. We’ve built plugins to detect new data systems as they appear in your organization—all you need to do is connect the plugin, turn on Silo Discovery, and we’ll begin scanning for systems.

Some examples of where Transcend can scan for new data systems:

  • Your single-sign-on provider (like Okta) to detect data systems your team can sign in to.
  • Your cloud-hosting provider (like AWS) to detect data systems like databases, buckets, and warehouses.
  • Your customer data platform (like Segment) to detect your source and destination connections.
  • Your codebase (like a JavaScript repo's package.json) to detect vendor SDKs.
  • Your websites to detect new ad pixels, analytics tools, your monitoring stack, and any other client-side tracking technologies.

You can view the full list of integrations that support Silo Discovery in Integrations.

The authentication process may be different from system to system, but the controls for turning on Silo Discovery are the same. Let’s use Okta as an example.

There are two options for hooking your system up for Silo Discovery:

  1. Transcend can either connect to a system directly and query it (pull-based plugins), or
  2. You can query/scan your system yourself and upload the results to Transcend (push-based plugins).

An example of a pull-based plugin is our Segment integration. You can connect this integration and have Transcend query the Segment API to sync your Segment Connections to Transcend.

An example of a push-based plugin is our JavaScript package.json integration. Connecting this integration on Transcend will give you further instructions on how you can use our CLI tool to scan your JavaScript project dependencies for new data silos. Our CLI will scan for vendor SDKs in any package.json files it finds, and upload the entries to the Transcend backend to be classified. Using this approach, it's simple to integrate the Transcend CLI into your CI process.

For pull-based plugins:

  1. In the sidebar of app.transcend.io, click on DSR Automation, then Integrations.
  2. Click the “Add Data Silo” button.
  3. Search for the service you wish to use as a plugin. In this example we’ll search for “Okta”.
  4. Click on the Okta card to open the connection screen.
  5. Scroll to the bottom of the connection screen and click the “Connect” Button
  6. Log in to your Okta account with Administrator privileges.
  7. Grant Transcend permissions to integrate with Okta.
  8. Once Okta is authenticated, and you’re back in Transcend, head to the “Configuration” tab Okta’s integration page.
  9. Turn on the “Silo Discovery” switch. This will queue up your first Silo Discovery scan!

For push-based plugins:

  1. In the sidebar of app.transcend.io, click on DSR Automation, then Integrations.
  2. Click the “Add Data Silo” button.
  3. Search for the service you wish to use as a plugin. In this example, we’ll search for “JavaScript package.json”.
  4. Click on the "JavaScript package.json" card to open the connection screen.
  5. Scroll to the bottom of the connection screen and click the “Add” Button
  6. Head to the “Configuration” tab in the JavaScript package.json’s integration page.
  7. Turn on the “Silo Discovery” switch.
  8. Run yarn tr-scan --scanPath=./myJavaScriptProject --auth=asd123 --dataSiloId=abcdefg from the Transcend CLI. For more information, please see the transcend-io/cli README.

As data silos are discovered by your various connected plugins, you’ll be prompted to review them in the Silo Discovery. In the sidebar of Transcend’s Admin Dashboard, click on “Silo Discovery”.

When you open Silo Discovery, newly-discovered data silos will be accessible for review:

Furthermore, when you have newly-discovered data silos, this button will appear on the Data Silos tab of the Data Inventory.

You can approve discovered data silos in bulk by selecting them with the checkboxes on the left, or individually by hovering over the rows. Marking them as approved will send them to Data Inventory to be configured, marking them as Junk will send them to the Junk tab. You can always retrieve them from Junk at a later time if you mark a data silo as such by mistake. If you see a data silo that is a duplicate of something that already exists in Data Inventory, you can mark it as a duplicate.