Data is crucial; and every marketer who manages digital advertising faces the same problem: how to aggregate and analyze data across multiple MarTech and Ad-Tech platforms. For many marketers, this task is a total nightmare. Marketers are spending too much time and money compiling their data just to have the ability to start analyzing stats. Currently they have limited options which require either manual “ETL” (Extract, Transform, Load) processes or support from developers, and some type of reporting platform or Business Intelligence (BI) tool.

For the last 5 years, I’ve chatted with hundreds of advertising agencies and digital marketers about how they approach this everyday issue. In this post, I’ll summarize my thoughts on the different ways that marketers can aggregate data and why we decided to build the first ETL platform specifically for marketers.

Problems Marketers Face with Data Aggregation

  1. Market Differentiation

Today we live in a very fragmented market. According to chiefmartec.com, there are more than 3,800 ad-tech platforms. With so many providers, it makes it nearly impossible for agencies to combine data across multiple sources to get the full picture and draw meaningful insights.

cheifmartec

2. Metric Definitions Vary by Vendors

A metric can be measured in a particular way on one platform and an entirely different way on another platform. A great example of this is video views across different channels like Twitter, Facebook, and Youtube. There are four key factors to consider when understanding video views: Initiation, Time Spent, Viewability, and Platform. Youtube for example qualifies a view when the video is user initiated and has 50% viewability on any device. They represent views as a percentage of total video length. Twitter, however, counts the number of views on auto-played videos but the video has to be watched for at least 3 seconds and be 100% in view on any device (source). Understanding and defining how each platform measures each metric is key.

3. Each Vendor has a Number of Ways to Measure Each Metric

Every client has different requirements for how they choose to measure the effectiveness of their campaigns. Being able to standardize this across all platforms is challenging. Take for example conversions. There are many different types of conversions (installs, purchases, form fills) but also post-click conversions and post-view conversions. There are also different types of attribution models of varying lengths. With so many options, it’s important to have consistency across reporting.

4. Ad Units and Metrics are Evolving Everyday

Platforms are constantly rolling out new ways to measure activity but also updating old metrics for accuracy. Calculations can change over time and staying on top of all those changes across the industry is a lot of work.

Since there are so many dimensions to marketing data, it becomes very difficult to normalize the data across so many platforms. This is why marketers have become dependent on manual processes or their developers to aggregate data.

How Agencies Aggregate Data

Below is a list of all the options agencies and brands currently have to aggregate data.

Manually

Most small to mid-size agencies still aggregate data manually. This process includes exporting data from each platform separately into spreadsheets on a weekly or monthly basis. Once they download the data and add it to a single sheet, they need to create necessary pivot tables and input any calculations.

Pros: There really aren’t any benefits to performing the manual ETL process.

Cons: This process is very time consuming and expensive from an opportunity cost standpoint. Instead of focusing on high value activities such as making strategic decisions and gathering insights, most marketers are focusing on low value activities such as data cleansing. There is also a greater probability for human error when copy and pasting data. The manual process takes up the majority of the time and leaves little time for actual analysis. Marketers need to be able to move quickly and have access to data fast.

Write Their Own Code

If an agency goes this route they will code everything themselves and have the data aggregated into their own data warehouse. From there, they will still need a visualization tool like Tableau or Google Data Studio.

Pros: This route will save marketers a lot of time and give them more opportunity to analyze their data and make decisions more quickly.

Cons: There are quite a few issues with this route:

  • It will require a lot of time for developers to build various API integrations for the data warehouse, and any new API integration requests will compete for development time vs. other projects at the agency. This can often lead to half automated, half manual ETL process. Defeating the purpose altogether.
  • Aside from building out the integrations, the developers will also have to spend a lot of time supporting integrations due to stability issues and release of new features. These stem from Facebook and Adwords API constantly updating for example. It is a maintenance nightmare to take on this task.
  • There are many other nuances that constantly arise like platform limitations, data clearance and normalization, dealing with errors, and more.
  • If lack of experience in creating data warehouses is likely to cause agencies to revisit the design of their warehouse schema, automation could make life a lot easier.

Use Advertising Reporting Platform

With this option marketers can link their ad accounts to a platform’s API and use that providers data visualization. Some examples of providers include Datorama and Origami Logic.

Pros: Some platforms include a good amount of integrations. This method generally doesn’t involve the help of developers.

Cons: Learning curves of how to use the platforms can be steep and setting up reports can be tedious. A lot of times platforms are constricted by features and marketers are bound to view data how that provider shows it. More than that, advertising reporting platforms cannot easily integrate with 3rd party business intelligence tools or any other type of data ( like CRM, HR, Finance, etc.) without the help of developers. The reliance on the development team can cause delays for the marketer in this case.

Use General ETL Provider

Agencies can use ETLs providers, like Stitchdata or Alooma, to aggregate basic advertising information.

Pros: This gives marketers the possibility to merge advertising data with other types of data, like CRM platforms.

Cons: Again, there are quite a few cons associated with using general ETL providers.

  • One major drawback is the lack of Ad-Tech and MarTech integrations. A lot of platforms still require some manual processes.
  • Another problem with this route is that marketers still need to have their own data warehouse and developers who can maintain it.
  • Once the data is in their warehouse, they also need a developer or data scientist to write complicated SQL queries to normalize the data.
  • This process will require developers to support a different type of database because of the nature of MarTech integrations.
  • If lack of experience in creating data warehouses is likely to cause agencies to revisit the design of their warehouse schema, automation could make life a lot easier.

Use Marketing ETL Provider

Until now, there hasn’t been an ETL provider specifically focused on marketers’ needs. That’s why we created Improvado.io. This option puts the marketer in the driver seat for managing their data in the most flexible way possible. By simply connecting to the Improvado platform and mapping metrics, marketers will receive a login and password for SQL access to their data which can be linked to any 3rd party BI tool, like Looker for example. Their data can either be housed on their database or in the Improvado data warehouse.

Pros: There is no need for a development team to aggregate data from multiple platforms into BI, visualization, or spreadsheets. All data is readily available and easy to access. With over 40+ Ad-Tech and Martech integrations, Improvado customers have access to the most important data.

Cons: Marketers still need a visualization or BI tool; however, they choose Improvado’s custom visualization solution developed for marketers who do not have other BI tools available to them.

  •  
  •  
  •  
  •  
  •