DataBytes   |   3423 Piedmont Rd NE Atlanta, GA 30305   |   (404) 400-7477   |   Email

Visualizing Big Data

Companies in the data visualization space are maintaining their competitive edge by adding functionality to support the demands of today’s rapid fire collection of data. Having evolved from Excel based charts and pivot tables for one-off analysis to more flexible and content rich data visualizations that integrate multiple data sources, dashboards today are being used to uncover business insights quickly, efficiently and interactively in the Big Data arena.

In the first of a two-part series, we will identify some pros and cons of using visualizations on top of big data solutions. Part two will dive deeper into specific strategies that can be leveraged to mitigate issues faced when working with large data sets.

A problem facing many companies that have invested in a Big Data solution is how to mine the huge mountains of disparate data being collected and convey insights in an effective way to the business.

Data visualization has played a huge role in bridging this gap.

Some key advantages of using visualization on top of big data solutions are:

  • Rapid insight: Using optimized interactive visualizations to glean insights rather than relying on open sourced tools to query large volumes of data accelerates the time needed to digest and interpret the data. In fact, in terms of speed and ease of use, visualizing big data is a game changer.
  • Business Focus: Staffing analytics teams who are subject matter experts in your business rather than people who must also have deep knowledge of writing SQL to query large volumes of data means keeping expertise focused on business strategy, not how to get at the data.
  • Innovation: Dashboards can marry the wide variety of data being collected into central dashboards where correlations can be made that may have been missed before. These correlations can translate into new products or services for a company and at the same time, elevate the value of your big data solution.

There are some disadvantages to using dashboards that should be considered as well.

  • Handling outlier data: When bringing together large data sets, it is sometimes difficult to identify outliers in the data when it is visualized. It is difficult to separate outlier data on screen where raw SQL queries could easily exclude it. Ensuring data quality through a defined governance approach is critical to ensure accurate data is being viewed.
  • This is a big one: If dashboard KPI’s involve calculated fields that mask complex formulas leveraging multiple fields, using visualization as a front-end for big data sets can be problematic.  Keeping visualizations fast means having the compute performed on the cluster itself. This means the complex formulas must be coded in the query feeding the visualization. If data is being filtered in any way inside an interactive dashboard, the calculated field(s) must also be able to transform.

To sum it up, companies who leverage visualization techniques to find value in their big data solutions will reap the benefits of the technology. Visual solutions served up faster and to a wider audience can enable a business to stay focused more on strategy and less on tactics, enable faster decision making and deeper insights, and ultimately empower the decision makers.