We tend to believe that everything in Information Technology is brand new. Those of us who have been around longer have seen topics revisited and relaunched. Just like in fashion, IT abandons some technologies only to bring them back years later with a new look or reinterpretation. And more than that, in IT, we have this habit of giving things fancy, foreign names — and if it’s an acronym, it sounds even newer!
Data Quality is no different. Many companies worldwide already practice some form of DQ. Remember, there are maturity levels for companies, people, processes, and the data itself when we talk about Data Quality.
For example, any kind of data pre-check, right at the point of entry (even during typing), or a verification (constraint) on a column in an RDBMS table, is already a step toward DQ.
Data Quality (DQ) is not just a software or an isolated technique. It’s a comprehensive process involving tools, people, and techniques — all working together, never as standalone initiatives.
Yet, in Brazil and many other countries, DQ is still in its early stages. Its adoption tends to be very segmented, as shown in the adjacent chart.
Often—and mistakenly—DQ is associated primarily with CRM and marketing, especially as a driver for campaign creation and distribution.
But is that all? No. DQ goes much further, including fraud prevention (a main use in the financial sector), obtaining a 360° customer view, and more.
While there are many applications, the most prominent use of DQ is as a filter for BI (Business Intelligence). Typically, BI tools extract data directly from source systems (databases) to build the BI model. However, these sources may be inconsistent—and generally are.
When dealing with multiple systems or databases, the data retrieved for BI tends to be inaccurate.
According to a SERASA Experian survey in the U.S., 81% of large American retailers found basic BI issues like duplicated customers or multiple addresses for the same customer, even after applying DQ.
No matter how many filters, rules, or transformations are applied during ETL, there’s always a risk of failure, because DQ is not just an extraction tool — it’s a process, and this is often overlooked.
Furthermore, SERASA Experian reports that 87% of U.S. financial companies encountered BI problems due to unconsolidated or inaccurate information.
Thus, the current trend is, when possible, to source data not directly from origin systems, but from repositories maintained (or protected) by a DQ system.
The alliance between BI and Data Quality has been strong in the United States and Europe, and I have even experienced a similar trend here in Brazil. There is still a long way to go, but this fusion between the two fields is definitely here to stay.
Visit our Blog
Learn more about databases
Learn about monitoring with advanced tools
Have questions about our services? Visit our FAQ
Want to see how we’ve helped other companies? Check out what our clients say in these testimonials!
Discover the History of HTI Tecnologia
