The objective of any BI or Analytics initiative is to equip your company with better decision-making capabilities. However, this is only possible if wha you are presenting is trustworthy and credible. In a perfect world, all the data we get would be clean, consistent and credible. Unfortunately, however, we need to deal with the reality of disconnected and siloed data, spread across hundreds of applications. We rely on data that is sometimes hard to or even totally out of our reach. And then, once we manage to get access to all our data, we grapple with the issue of bad data.It is estimated that bad data costs the US economy $3 Trillion dollars a year and that up to 45% of operating expenses are wasted due to bad data.
Start by thinking about the journey that your data takes to get from a source to a dashboard. There are things you can do at each point in this journey to help improve the quality of your data:
Tip 1: Collect good data by getting as close to the source as possible.
Tip 2: Trust the data you collect by setting clear expectations for quality.
Tip 3: Find and fix issues in your data as early as possible.Use the concept of the 1-10-100 rule for data quality.
Tip 4: Continuously improve the quality of your data by measuring for quality and act on recurring issues.
Tip 5: Have a controlled and auditable way to make adjustments and changes to your data
How credible is the data that you are using to create your reports and analytics? Who is supplying the data? Is it coming straight out of a database or a source system or are you relying on data that has been manually prepared by somebody else in a spreadsheet?As a general rule, the less manual hands involved in your source data, the better. Wherever possible, you should seek data that is produced in an automated way.The reason why this matters is simple. You don’t want to be starting with erroneous data. Studies have shown that 90% of manual spreadsheets have errors. A person at Barclay’s bank accidentally deleted instead of hid cells in their workbook, the error cost the bank millions.Similarly, JP Morgan made a cut and paste error in a spreadsheet that broke one of their risk models resulting in a $6 Billion dollar fine