It’s evident now the world’s most valuable commodity is no longer oil but data. It’s also clear that the lion’s share of marketing dollars are being siphoned by Google and Facebook, and trusting these adtech companies to police their own businesses is no longer enough. Businesses who need to stay relevant in the next decade cannot stay on the sidelines when it comes to big data analytics. So just how does a business develop big data analytics and attribution capability in-house? What capabilities must it develop? How much can it cost? Must it build or can it buy the same capabilities? Get answers to these questions below.
Opting to “Build”- Fill the gaps in analytical maturity
If building big data analytics and attribution competencies, approach the initiative as a path to increasing the analytical maturity of your business. Work may be needed on any/all of the following four areas.
- People gaps: The team is only as good as the leadership. The team lead has to be someone with an analytics background who can shepherd the analytics function through client needs, and who can lead the attribution effort. Some capabilities to consider for building in-house:
- Consultative analytics experience
- Big data analytics experience (1TB data mart or larger)
- Data visualization + storytelling capabilities
- Domain expertise
- Team leadership skills (5+ years experience in managing small teams)
- Algorithm design experience
- Experience with a high-level programming language
- Post-graduate degree in the technical disciplines is nice to have.
- Technology gaps: It’s one thing to run reports off Google analytics, quite another thing to build a data infrastructure to capture, process and analyze the server traffic to deliver client requests. A typical technology environment would be at least a 1 TB data mart comprising:
- Data capture infrastructure (aggregate server traffic, data from CRM systems)
- Analytical infrastructure (algorithms for attribution, deduplication, cross-device tracking)
- Analyst layer (workstations with ability to download, and analyze data in a high-level language such as Python or a SQL database)
- Presentation layer (software for presenting findings for end-user consumption)
- Process gaps: Processes are what stitch people and technologies together in support of the end results. I will highlight two gaps I have consistently seen in organizations with analytical aspirations
- Analytical methodology: There needs a process framework for addressing client issues through analytics. I recommend issue based consulting and our own home-grown framework – “How to make data driven decisions using Polytab shopper-tracking data“.
- Data management processes: I recommend a review of the trust service principles. Not all of these may be applicable – but it’s a good start for a self-audit by any business aspiring to be big data players.
4. Cost of building: The bare bones costs are laid out below. The estimates would be valid for a small to mid sized merchant. For larger businesses (upwards of $150M in annual revenues) there are more variables to consider. Contact me at varun @ polytab .com if you want to chat about this.
- Salaries: You would be looking anywhere from $200K to $750K p.a. for building up a team with the requisite skills. In addition to a strong analytics lead, teams include big data query specialists, statistical analysts, (possibly) experts in machine learning technologies. You may also need experts in systems administration, database administrators, programmer analysts.
- Hardware: You could host your own servers or look to hosted services like Amazon or Azure. Your biggest costs are likely to be bandwidth charges. You may also need to look into licensing fees for relational database management systems, statistical software, noSQL environments etc. depending on the needs and scope of your analytical teams. Budget anywhere from $50K p.a. to $250K p.a. depending on your data mart and bandwidth utilization.
FAQ: Why does it cost so much when I get everything for free through Google Analytics?
Answer: Google analytics provides summary reports, not the data. Then again, GA only focuses on transactions that lead to a sale, not the 90%+ traffic that does not go anywhere. Arguably, there is an upside in converting the traffic that does not convert.
Even a modest sized merchant with 500K page views/month, at 1KB/page view, needs 500MB diskspace and bandwidth to capture the data for one month. This is the Base.
- Multiply base with 5x for computational space
- Multiply by 36x for 3 years processing to capture seasonality and periodicity effect.
- Multiply by 6x for disaster recovery and Business continuity planning mandated redundancy measures.
The numbers add up.
Opting to “Buy” – On demand cloud based shopper analytics solutions
An alternative to building the capabilities listed above is subscribing to Polytab analytics and attribution software.
Web based Attribution
You gain a world-class attribution and analytics product at a very reasonable price. Polytab’s technology subscription – fully loaded – is typically no more than 1% of the client’s digital marketing budget. Typical subscription ranges from 0.4% to 0.75% of the advertising budgets for most businesses starting from $600/month.
Bigview analytics platform
For businesses with in-house analysts and data scientists, BigView cloud based analytics platform provides access to the shopper journey data on-demand as a hosted solution. It is built on the Apache Spark framework and provides access to the data with the tools for analysis and visualization. Pricing starts from $5000/month.
What is your analytical maturity?
Not sure of the analytical maturity of your business. Download the free maturity calculator by submitting your information in the form below to get a complimentary maturity model and self-assessment scorecard.