Handling Big Data with Qlik
by Marina James, on Oct 29, 2018 8:48:26 AM
According to a new report from IBM Marketing Cloud, “10 Key Marketing Trends For 2017,” 90% of the data in the world today has been created in the last two years alone, at 2.5 quintillion bytes of data a day!
But, does that matter to me?
Many organizations are interested in how Big Data impacts them, and how they will the harness this information for reporting. Hadoop/Big Data should be on everyone radars as huge business gains can be gleaned from mining the information with machine learning and visual analytics as well as overall cost and computing savings. However, most of the traditional visualization tools (non-Hadoop origins like, Birst, DOMO, Tableau, etc.) encounter challenges when interacting with large amounts of data.
The article highlights Qlik’s approach. The piece isn’t to endorse a particular tool but help educate on a market leader. Each situation is different and should be evaluated as such. If you need assistance with your BI or data environment decisions, please contact firstname.lastname@example.org.
Qlik is heavily investing in their Big Data Indexer, and current has two common approaches for handling massive data sets, 1) Direct Discovery (QABDI Live) and 2) On Demand App Generation (ODAG). Qlik is continuing to invest to scale and enhance platform.
The below video overviews a solution with open source Taxi Cab data.
Qlik solves most use cases that I have seen in the field from a reporting perspective. It is typically not incredibly useful to see detail information until you are deeper in your analysis, and summary views are needed to guide user. Though ZoomData and Arcadia Data may have some advantages with streaming visualizations, Qlik’s total platform and features make it a better choice for most (+80%) of use cases.
Qlik ETL engine also allows it to ingest other information (Spark, Hive, ODBC, etc.) to augment analysis. Qlik’s API layer can also integrate with other elements of your environment (ie. call a python model or write back). Compared to the other visualization or in-memory tools, Qlik appears to be the farthest along today.