Data Analytics: The Ultimate Guide – Big Data, Big Money? Are You Ready?

Data analytics is playing a more important role in organisational and career success.  As organisations become more tech-driven and fast-paced, data analytics is becoming a mission-critical skill.

Analysing data been around for centuries.  But 2 recent changes made data analytics critical to every organisation;

1. the rapid growth of computing power and 
2. the explosion of data availability.

So here’s a quick-guide to everything you need to know about data analytics.

Let’s get started.

Table of contents

• What Is Data Analytics
• What Can Data Analytics Do
• Cheat Sheet to Common Data Analytics Terms
• How Data Analytics Can Benefit You
• How are Organisations Using Data Analytics
• Using Data Analytics Effectively (Step-By-Step Guide)
• 5 Common Errors that Kill Data Analytics (and how to avoid them)
• The 3 Essential Data Analytics tools that You Need
• Next Simple Steps to Develop Your Data Analytics skills

What is Data Analytics?

Data analysis is simply the process of turning data into information and finding actionable insights.

And many of the things we do in business rely on data analytics. Customer satisfaction, business plans, marketing, sales, quality, new products.

Which is why good data analytics enable organisations to make better decisions in all areas.

What can Data Analytics Do

Data analytics can answer 4 critical business questions.

1) What happened? – Descriptive Analytics

This is the most common type of analysis. Descriptive analytics helps us understand what has already happened. Often by summarising raw data.

2) What’s going on? – Diagnostic Analytics

Used to dig deeper into an issue and uncover the root cause of the problem.

For example, problem solving methodologies like Lean Six Sigma Data Analytics tools to fix business problems.

3)  What will happen? – Predictive Analytics 

Used to build models and trends of the future. These tools can predict how likely something will happen in the future.

As a result, they are usually the most valuable use of data analytics.  The more accurate the model, the greater the benefit.  

For example, each day, Zara, the clothes retailer, gathers store till sales information, weather data and fashion trends. Zara then uses a model to predict which clothes will sell tomorrow. Zara then re-stocks each store and orders more of what is selling well.

4)  What should I do? – Prescriptive Analytics.  

Used to sort through a range of options and find the most useful one for you.

For example, Google Directions.  You choose where you’re starting from and where you want to go. Google then figures out the best route for you using distances and traffic data.

Cheat Sheet Guide to Common Data Analysis Terms

Data mining – ways for finding data clusters, often hidden in data. Used to great effect in recent elections in the Brexit vote in UK and the US 2016 elections to spot and target groups of voters.

Machine Learning – methods for automatically spotting data patterns and building these into models. These models get better and better over time.

For example Google’s search suggestions.  Even when I mis-spell words it figures out what I meant to type and offers relevant search results.

Big Data Analysis – so what if your company is making millions of new data points a day and you want to analyse in real-time?  

That’s when you need Big Data tools.  The term ‘Big Data’ is used when the amount of data is too large for normal tools like databases and spread sheets.

Big data tools are used to store your data and to analyse many sources of data that could be spread all around the world.

For example, Facebook’s tools allow it to monitor ad performance in real time. It changes your newsfeed based on what you click on. 

Text mining – yes, data analysis has moved way beyond just numbers.  Tools can analyse all sorts of data; text, pictures, video, opinions, to discover patterns and insights from the data.

For example the software controlling self-driving cars depend on data analytics from millions of photos.

How can Data Analytics benefit me?

Today’s best organisations embrace data analytics.  Companies that don’t, are being left behind.  

Whole industries are being changed rapidly by data analytics.

For example, take the travel industry.

In 2009, Uber set-up an app to link drivers and passengers.  Less than 10 years later, Uber has 15 million trips per day. It has changed the old cab and taxi market.

What can data analytics be used for?

To find insights that can be used to make better decisions and improve future results.

What’s the role of data analytics?

Data analytics helps organisations make better decisions.  It works well with our decision making ability, helping us beat our brain biases and handle complex data sets.

As the world produces more and more data, data analytics is becoming a key enabler for growth and profits.

For example, look at the most valuable companies in the world.  They’ve built a core ability to gather and analyse data more effectively than their peers. 

How do organisations use data analytics?

Data analytics can be applied within all areas of an organisation.  Here are a few of the most successfully applied areas.

Using data analytics to create amazing customer experience

By analysing customer buying patterns and service feedback, organisations can develop powerful insights.  

For example, Amazon uses data analytics to recommend related products to its customers.  “People who bought this also bought this…”

Facebook learns how you use its platform and adjusts your newsfeed so that it’s more relevant to you.

Tesco sends you money off coupons for products that you are likely to purchase based on your previous purchases.

Using data analytics to Improve Performance.

Running a business is like driving a car.  When you’re driving your eyes are giving you real-time feedback on your steering.  If your eyes see that you’re drifting left, your hands react and correct. This happens almost instantly, without us even having to think about it.

Now compare this to the typical business where feedback is given weekly or monthly.

If your eyes did not give you feedback instantly you have only 2 choices:

1. Slow down so that when you do receive feedback you still have time to correct your steering
2. Continue at speed but crash multiple times as you veer off the road over and over again

Organisations have the same 2 choices.  However, the feedback loop is much longer, typically weekly or even monthly.

So when an organisation uses data analytics to speed up the feedback loop, it can operate faster (agile) and has fewer issues.

Increasing profits with data analytics

Manufacturing was one of the first areas to use the power of data analytics.  

Six Sigma, Quality Manufacturing and Lean (Toyota Production System) brought data analytics to the factory floor.  The data analytic tools were used to identify hidden costs and improve processes.

Whole industries were transformed as quality got better and costs dropped.

Over the past decade data analytics has branched out from manufacturing.  Marketing, sales and customer support were some of the first areas to embrace the tools.  With huge impacts.

Now the tools are used widely and brining cost and quality benefits to areas like HR, Finance and IT.

“Without data you’re just another person with an opinion.”

W. Edwards Deming

Using data analytics to reduce risk

Data analytics is being used to reduce 2 types of risk

• 1. Fraud – data analytics can find unusual patterns of activity that could be fraud.  

• For example, credit card companies use real-time data analytics to spot potential fraud use on your credit card.

• Finance auditors use data analytics to protect companies from dishonest employees.

• 2. Future shocks – creating models and running ‘what-if’ scenarios allows firms to identify and plan for future shocks.

• For example, what happens if a competitor launches a new product that is much cheaper than ours?  What could happen and how should we prepare now so that we respond rather than react?

How to use Data Analytics (Step-by-Step guide)

But how can we start using data analytics right now?

Now we know some of the basics of data analytics and how organisations are successfully using it.  

Good news is that data analytics basics is fairly easy.  If you can use excel and have a bit of curiosity then you’re ready to go.

This step-by-step guide will walk you through the main steps

1) Decide:

What’s the question you’d like answered?  This is the key step.  You need to be clear on what it is you want to achieve. Or else your data analytics will not work for you.

For example, let’s look at your salary:

• “How much did I earn last year?” – Descriptive
• “Why did I not earn more?” – Diagnostic
• “How much could I earn next year?” – Predictive
• “How do I double my salary?” – Prescriptive

Each question needs a different set of data to answer.

For this example let’s assume that the question we’d like to answer is “How do I double my salary?“

2) Gather

Next we need to find data sources that may give us the answer to our question.  

They could be internal (within our company), external, quantitative (fancy word for numbers) or qualitative (non-numbers like text or pictures).

To answer our salary question we’d want a number of data sources.  

• Our internal salary grade data from HR
• List of job roles and job titles that pay double our current salary.  
• Interviews with hiring managers and head-hunters for their insights into how we can double our salary (excelling at Data Analytics is probably high on the list)

The wider you can cast your data net, the more likely it is that you will come up with a valuable, and often surprising, insight that you can use.

3) Organise & clean

It’s rare to find data clean that’s ready to use.  We’ll have multiple interviews and job lists that we’ll have to:

• merge the data records (e.g. create a way of linking the data sets together)
• clean the data (e.g. fix misspellings, fill in blanks, standardise job titles)

The goal is to link and fix the data so that we’ll be able to analyse it well.

4) Analyse  

Now the fun part.  This is where you put your Sherlock Holmes hat on.  We’re looking for patterns in the data 

Run your analysis.  Sort, segment and query to look for insights and ‘aha’ moments.  If your data set is large or complex this can feel a bit like searching for the needle in the haystack.  This is when you use data analysis tools to quickly sort through and find that needle.

Next, visualise your data with graphs and tables.  Our eyes are great at spotting patterns quickly when data is presented visually.  Often a hidden trend pops out on a graph as a line going up or down.  More complex visuals allow you to uncover even the smallest of needles. 

We’re looking for things that may help double our salary.  

Are there patterns around skills, or experiences, or qualifications?  What about industries or geographies?

It’s vital to keep an open mind at this stage of your data analysis.  Just follow the data and see what it’s saying to you.

5) Test  

Hopefully you’re seeing a few potential factors popping out of our data.  Before we spend too much time checking further, it’s key to sanity-test each factors.

For example, you might find that jobs in ‘conflict zones’ pay more.  But if this is not of interest to you, then eliminate the factor ‘conflict zones’ and focus on other factors. 

6) Validate

Now you’ll have a rough model that predicts how you can double your salary.  But is it accurate?

We can  find out by comparing your model’s results with data from the real world.

Let imagine your model predicts that your salary would be double if you 

• a) have ‘director’ job title. 
• b) have ‘data analytics’ as a key skill.
• c) have ‘people manager’ experience.

So how could we test this…?

Open up LinkedIn, find people with this profile and see if the model holds.

You should always validate your model against real-world data.

7) Refine

Usually when you validate your first model you’ll find that your model is not quite right.  This is usually because you have not included an important factor in your model.

For example, you may find that education level is an key difference in salary levels but you had not included this in your model.

Refine your model by repeating steps 2 to 7 until you are happy with the results your model is giving you.

8) Use

Now it’s time to enjoy the fruits of your Data Analysis.  Use your new insights to make better decisions and take action. 

5 Common Errors that kill Data Analytics (and how to avoid them)

1)  Bad data

It sounds obvious.  “You get out what you put in”

But bad source data is the #1 reason for poor data analytics. No one intentionally starts with poor data.  So what’s going on?

Often we need to merge data from different sources. Errors creep in.  For example, I recently helped a client fix a data model that was recommending a large expansion into India.  

It just didn’t seem right.  The problem?

They had assumed that revenue numbers were in US dollars. They were actually in local currency. And with 1 US dollar equal to about 100 Indian Rupee it was throwing the model out.

Always check the data source.  What exactly does each data field mean?  Where is the data coming from?  

Always try and validate the source of the data independently.  For example, if you’re evaluating customer survey data, find an original survey form. Then compare it to your data to make sure it makes sense.

2)  Not using all the data

Imagine creating your grocery shopping list for the week. You open the fridge, check and write down what you need to top-up with onto your list.  You close the fridge.  Put your hand on the food cupboard door and…

Stop!  Leave the door closed and go off to the shops.

How accurate do you think your shopping list would be?

Do you think your shopping list would have been better if you had opened the cupboard door and checked what you needed from there too?

You bet.

Yet, in business we do this all the time.  All the data we need to make a great decision is available but we often just settle for the incomplete data that we have.

Resulting in poor decisions.

It’s always worth spending a few minutes to gather all the data you need. Ask people and see who may have the data you need.  

3)  Tradition

This can be a very painful error to correct.

If your organisation has been around for a while, a tradition of ‘how decisions are made’ will have developed.  If it’s data based… lucky you!

But many organisations develop unhealthy practises. For example, deferring to the leader to make certain decisions based on ‘experience and gut-feel.’  

Yes, there is a time and a place for gut-feel.  But it is no substitute for well-performed data analytics.

Next time you are tempted to rely on your ’15 years of expert experience’ instead of what the data is telling you remember…

Most of today’s biggest companies did not exist 15 years ago. Their rapid growth is fuelled by their focus on data driven decisions.

4)  Cooking the Data

Yes, this happens.  A lot more often than you think.

Data analysts can distort data on purpose to support their point of view.  They trust that no one will have the insights and know-how to spot.

But they’re wrong.

With what you’ve learned here, you have the confidence to question the data sources and know how to validate them.

5)  Cognitive Biases:  

Our brains are great at taking in complex data and quickly finding the information that really matters. 

We’re only alive today because our ancestor’s brains used this ability to quickly spot the sabre-tooth tiger hiding in the dense jungle.  

Our brains can do this because they make decision shortcuts. These shortcuts kept us alive.

But these shortcuts play havoc with our data analytics abilities.  

These shortcuts can lead to poor decisions.  

The shortcuts are called cognitive biases.

Often the hardest part about analysing data is controlling these cognitive biases.  But once you’re aware of them, they’re pretty easy to correct for and make better decisions.

Here’s a checklist of the most frequent cognitive biases and how to handle them.

12 Cognitive biases that play havoc with your decisions

1) Anchoring Bias

People are over-reliant on the first piece of information that we hear.  For example, in price negotiations, the first offer whether high or low shifts the range of ‘good’ prices in the other person’s mind.

2) Group Think

The more people who think a certain way, the more likely it is that others will follow their thinking.  Hence why analysis and decisions by committee tend to have poor results.

3) Placebo Effect

Just believing that something works can show real improvements.  From drug trials we know that this effect can account for up to 25% of improvements.  

4) Outcome Bias

Judging a decision based on the outcome instead of the decision process.  For example, comparing a big win at the slot machine versus a low amount of interest from a savings account.  

5) Success Bias

Using only data sets from good outcomes.  For example, comparing stock market funds results.  We might think that the average fund has a 20% growth. But only because we have not included all the funds that failed and closed because they did not perform.

6) Availability Heuristic

We tend to place heavy reliance on data that is readily available to us.  For example, you might believe that smoking is not as bad as it actually is if you’ve got 3 friends who smoke and are in good health.

7) Pattern making

Your brain will try and make patterns of random data.  Most people when asked to place 100 random dots on a blank page will place them evenly on the page.  However if you placed truly randomly, there would be clusters of dots and big areas of blank spaces.  

8) Confirmation Bias

We tend to listen to data that confirms what we already believe. We ignore data that threatens our thinking.  

9) Expert Bias

Our brains over-estimate what we know and under-estimate what we don’t know.  This effect is greater if we believe we are an expert in the subject area. 

10) Post-Choice Bias

Once we make up our minds, it is hard to change our original decision.  We defend our choice even when we find new data or evidence.  

11) Information Bias

Many people find it difficult to make a decision without lots of information.  Even though most of the information is not needed for the decision.  Usually even complex decisions require only a few pieces of data, analysed well.

12) Blinkers Effect

Deciding to ignore uncomfortable or negative data.  For example, not going to see the doctor when your body is showing signs of sickness.

What Data Analytic tools do you need?

Data analytics tools fall into 3 main categories

Data Storage tools –  Databases, spreadsheets, computer systems, servers, etc.  You need someplace to store your data.  Your solution will vary depending on the amount of data and the frequency that you are adding to your data source.

Data Analysis tools – for example mini-tab, Excel’s data analysis tools, computer languages like SQL, PERL, Hapdoop.  These tools do the heavy lifting.  Sorting through your data, finding the patterns.

Data Visualisation tools – Excel graphs, Tableau, mini-tab. These tools help visualise numbers into graphical representations that are easier to understand and make decisions with

Data Analytics next steps

You probably already have the tools you need to get started with.

1) Get good at Data Analytics using Excel

Microsoft Excel is a great all-round tool to get started on and perform very powerful analysis.

You can store, analyse and visualise data-sets with Excel.  Add the free Data Analysis Pak to give you more powerful data analytic tools like regression analysis and modelling.  

Start by creating different types of graphs from data that you already have.  You’re looking for patterns, oddities and clusters.  

There are many good sites on-line that take you through the basics of using Microsoft Excel.

2) Take a Data Analytics Course to accelerate your skills

And if you like data analytics and would like to become better at this increasingly important skill, considering taking a data analytics course.

In a few days you’ll build the skills and confidence to correctly analyse data.  And in the process, improve your decision making skills and career opportunities.

If you’re interested we can help you on that journey, either on one of our scheduled public Data Analytics courses or on a private, In-house training event.

Our trainers are some of Ireland’s leading data analytics experts, currently working on cutting edge applications of data analytics for leading multinationals.  So they will be able to develop and deliver a training experience tailored to exactly what you need; from novice right through to expert.  Get in touch and we’ll send you the details.

Upcoming public data analytics courses:

So where do you believe data analytics is heading?  I’d love to hear your comments below.