In order to try and provide a consistent result, the same environment was used for all the tests. One dashboard was created per software with a changing data source consisting of a range of records between 1 to 30 million. Supplemental information about the data cardinality, software version etc. is detailed at the bottom of this post.
This comparison is used to provide a high level analysis. It uses random generated data, and therefore cannot replace a test with your own data and requirements. Each software handles data in a different manner and therefore results can skew according to the nature of the data.
In order to provide a fair comparison for the performance and resource usage. Three dashboards have been created, providing very similar information. These also provide a quick glance at the different visualization style used by each software.
Qlik Sense Dashboard
Data Size on Disk
Each software saves data extracts in a different fashion, either by external files or embedding the data within the report file. For Tableau the TDE file size was checked, for Qlik Sense the QVF file and for MicroStrategy the MSTR file (embedded in an empty report).
The results showcases that the file size is quite similar between the software up until 10 Million rows. MicroStrategy and Tableau appear quite close together. However Tableau improved in testing done beyond this scope, which included 70 Million rows with only 124MB used by Tableau (the drop in disk space is assumed to be associated to a grouping/compression applied by Tableau).
Each software uses different processes. The tests attempted to single-out the memory used for the dashboards by each software (upon opening the dashboard). For Tableau the memory size was taken from Tableau.exe, for MicroStategy from MicroStrategyDesktop.exe and for Qlik Sense from QlikSenseBrowser.exe & Engine.exe.
The results showcase that Tableau occupied the least amount of memory. Followed by Qlik Sense, which used a reasonable amount of memory. Lastly MicroStrategy occupied a large amount of memory and quickly deteriorated in performance and resource usage upon increasing the amount of rows beyond 10 Million.
Each software uses filters in a different manner. Some use selection on the dashboards, while others use separate panes. For each software I noted the longest time it took to apply a filter in the dashboard based on the amount of records in the dashboard.
Tableau & Qlik Sense had similar performance in applying filters, and managed to handle this task with ease. Tableau was somewhat faster, but the results from Qlik Sense were on par with Tableau. MicroStrategy on the over hand, lagged behind with 23 seconds of wait time for applying a filter on a 5 Million rows dashboard. This continued to deteriorate up to 294 seconds for 30 Million rows.
The last test conducted, checked the amount of seconds it took each software to load the dashboard. This test attempted to single out the time it took to load the dashboard (after the reporting software was already running).
Tableau excelled and did not require additional load time as the data grew. This might be due to the caching mechanism used in Tableau, but is also assumed to be attributed to the fact, that the extract size was not increasing with the increase of rows (beyond 30 Million the extract size remained very similar, possibly due to grouping/compression). Qlik Sense had a very reasonable load time up until 5 Million, however beyond this point the initial load time of the report was getting too long. MicroStrategy fell behind with already 53 seconds for 5 Million rows.
Please note that this comparison is based on out-of-the-box testing. For better results, you should optimize each software using best practices documentation.
The source database is Oracle XE. All data was loaded into data extracts of the reporting software, rather than using a live connection.
The sample data used has a low cardinality, with generated random numbers for Sales_Amount & Revenue_Amount. Full information about the cardinality in each column is detailed below (using an example of the 1 Million rows data set)
The software version used in the testing are:
- Tableau Desktop 10.0.0
- MicroStrategy Desktop 10.4.0
- Qlik Sense Desktop 3.1 SR1