< Browse more articles
Data Warehousing Process
Why data warehouses remain a cornerstone of any complete BI solution

It’s a very exciting time for data analytics. The evolution of technologies to help process, understand, and find meaning from big data is happening at breakneck speed, with new capabilities steadily emerging. So what does that mean for the future of the industry and the root methods that dominated just a short time ago?

Chris Klimm, Principal Consultant at Hexagon Safety & Infrastructure, says: “With these advancements, we see terminology, such as Data Lakes, Machine Learning, Artificial Intelligence and Cloud Based Computing leading the conversation in our industry. While these newer technologies offer great promise, the data warehouse remains as much a cornerstone of any complete BI solution as it ever was.”

Klimm asserts that without a data warehouse, we simply have raw data to work with—and while raw data can be useful in the right hands, it assumes a great deal of knowledge on the user’s part.

Nick Heudecker, research director at Gartner concurs with Klimm, stating that: "It assumes users recognize or understand the contextual bias of how data is captured; that they know how to merge and reconcile different data sources without 'a priori knowledge' and that they understand the incomplete nature of datasets, regardless of structure."

Generally speaking, most end-users lack this level of sophistication. End-users are typically not data scientists and nor should they have to be. This is where BI and data warehousing comes in. A data warehouse adds context to the data, allowing it to be inspected, cleansed, and transformed into structures that have meaning and can be easily understood by the business users. “It’s a place where the business can have ‘one version of the truth’ and have a level of confidence that the information they are making decisions on is accurate,” says Klimm.

With Hexagon’s software, called Intergraph Business Intelligence for Public Safety, a data warehouse is delivered with 15 distinct subject areas, each modeled using the Kimball dimensional model, the “gold standard” for data warehousing. Each is also designed to be intuitive to the business user and to accurately answer a different set of business questions.

But does data warehousing in the backend help the BI solution become compatible with other BI visualization tools - such as, Microsoft’s Power BI, QlikView, Birst, SAP Webi, SAP Lumira, and Tableau?

“Yes,” says Klimm. “That is another advantage of a data warehouse. The data warehouse organizes the data into dimensional models, called star schemas. Dimensional models organize the information into clear and coherent business categories. These business categories are our 15 subject areas and they are much easier to read and understand than the source transactional data models used by I/CAD and WebRMS.”

According to Klimm, the simplicity of these models makes it very efficient for third party business tools to connect and use this data, allowing Hexagon’s Business Intelligence solution to be compatible with most major BI tools.

The data warehouse is also optimized for performance. “The simple, predicable and controlled nature of star schemas means databases can easily optimize query performance,” he says. “In addition, databases, such as SQL Server, have included technologies like columnstore indexes that specifically target the data warehouses and star schemas.”

As Microsoft states: “This new [columnstore] index, combined with enhanced query optimization and execution features, improves data warehouse query performance by hundreds to thousands of times in some cases, and can routinely give a tenfold speedup.”

And, of course, that’s not all. The data warehouse is also scalable and extensible, with the dimensional models allowing for additional data sources to be added and the models extended as necessary. “Existing BI applications, queries, reports and analytics will continue to work unaffected by the changes,” says Klimm.

To find out more about the importance of a data warehouse when it comes to your Business Intelligence solution, please visit: http://go.hexagonsi.com/EVOLUTION-BI-Webinar