Handling Custom Data in a Reporting and Analytics Environment

Authors: Gautham Ramachandran, Wesley Eversole, Angshuman Rudra, Chris Doore, Sameer Saraf

Are you struggling with Custom Data and how to bring it into your reporting tool? Do you want to compare, contrast and integrate these custom datasets with data from standardized data sources such as Google Adwords, Facebook Ads or Google Analytics?Tapclicks and it’s Smart Connector lets you bring in your custom datasets within minutes.

TapClicks have successfully implemented a process that enables you to massage and augment your data to reliably and automatically ingest custom datasets that are not covered by the standard integrations.

With more than 1,400 active Import wizards growing at more than 9.4% YoY on the TapClicks platform, a prominent use case for this requirement is the need to handle custom data reports from specific systems or from a client’s databases and present them in conjunction with other standard datasets in reporting and analytics platforms.

Once the custom data is in TapClicks, we also enable the creation of custom metrics for both the custom dataset and other standard datasets.  

What TapClicks aims for:

The goal is to create a cross-channel dashboard that aggregates data across multiple sources that include both client-specific custom data and standard services (Example: Google Adwords/Facebook Ads).

Below is an example of such a cross-channel dashboard from Salesforce:


TapClicks Salesforce Dashboard
How does this work?

TapClicks has been advising some partners to provide exports to CSVs so that they can be consumed by the import wizard tool. For more detail on how the import wizard is to be set up, refer the import wizard set up documentation. Below is a snapshot on Import Wizard setup:

TapClicks Data Import Tool

Handling Limitations with the data for ingestion:

There is often a need to massage and augment the raw data prior to data ingestion and visualization in the TapClicks ecosystem. Below are some examples of data augmentation that may be required:

Setting flags:

Setting custom flags (0/1) for leads, opportunities, and wins for sales related automated reports that allow us to calculate conversion rates.

Creating categorization of variables from the raw data:

Categorization of variables from the raw data into predefined buckets, examples include buckets of advertisers or categories of MRR.

Aggregating Datasets:

Joining multiple CSV files across sources to be ingested into the Import Wizard using a join key.

Below is the process to augment the data prior to data ingestion in an automated and a periodic manner into the TapClicks environment:


How to Aggregate Data Sets


What this means for the end user is the ability to import in an automated manner any custom dataset into the TapClicks Reporting and Analytics platform to allow easy integration and rendition in Dashboards with standardized data sources.

The above-defined process is a flexible option to handle data ingestion and is meant to be used in situations where the normal engineering process can’t provide the necessary solution. TapClicks already has clean processes to handle data in an automated manned regardless of its source and extending them to build intelligent systems. Where ever possible these existing processes should be first considered before pivoting to this new piloted process.