Top 10 Evaluation Criteria for Analytics & BI Platforms

  1. Home
  2. Article
  3. Top 10 Evaluation Criteria for Analytics & BI...


We are seeing huge amounts of disruption in most industries, further escalated with Covid-19. What’s clear is that the companies coming out the other end unscathed or even in a better situation (not including certain industries like travel), are those that have kept pace with innovation and digital transformation. At the heart of this, is data!

With BI adoption rates as low as 21% in an organisation, one of the main areas we will see growth in 2021, is in data democratisation. More specifically, how can you get more people in your business, making better data driven decisions. The next wave of BI solutions will move to a more consumer centric model, and mimic some of the top consumer apps like Netflix, YouTube and Uber. AI will also play an important role in helping your users to surface insights that they might not be able to find themselves – only algorithms and machines can realistically analyse millions of rows of data.

Here we’ll explore the top 10 evaluation criteria, to help you with your BI / Analytics purchasing decision.


  1. Data Modelling
  2. Data Environment
  3. Search Experience
  4. Search Intelligence
  5. Chart Creation
  6. Insights
  7. Speed & Performance
  8. Total Cost of Ownership / ROI
  9. Security & Governance
  10. Training & Support

1. Data Modelling

Too much time is spent by IT & Data teams modelling data and often businesses spend much more time and money on BI services than the actual software itself.

Across all the companies we speak to, the typical pipeline involves tons of data, usually scattered across tables across dozens of data sources that data teams need to extract and put through a complicated ETL process to simplify the data model. Then the data needs to be moved into a data warehouse or data lake for centralised storage, segmented into data marts, and then summarised even further into cubes and views. This process itself can take weeks before the business gets a usable data set they can get insights from.

Some of the data analytics products are schema-aware and able to remove a significant amount of data modelling and complexity. Schema-awareness means the search engine interprets the many relationships between different data sources and can relate them together automatically.

Try to find a product that can free up BI teams to focus on higher value problems like data governance and data quality rather than being bogged down with the complex and mundane tasks mentioned above.

Questions to ask: How long is a typical implementation engagement before users can start getting value from the solution and can it handle the complexity of your data model?

2. Data Environment

With the explosion of data sources we have seen in recent years combined with the complexity of most enterprises, especially those that have grown through M&A, can your BI tool handle all of your data sources?

As well as internal data, businesses also need to gather insights from external data sources just as easily.

The ability to search data at scale from a variety of sources is essential to a productive business user. Google combines search results from across the entire web, in a similar fashion, search-driven analytics solutions should be compatible with your existing data environment and capable of analyzing search results across different types of data sources, as well as different data integration or ETL technologies.

We are now seeing a growing trend of tools that can directly live query your Cloud Data Warehouse (Amazon Redshift, Google Big Query, Azure Synapse, Teradata, Snowflake), where it resides and get granular and up- to-data insights.

Questions to ask: We don’t want to learn different BI products for different data sources, is your solution capable of handling all types of data sources quickly and easily?

3. Search Experience

Most Analytics & BI Platforms these days offer some form of search capability, but not all search is the same. As an example, Google, Amazon and Instagram all offer search capability, but the experience is very different on each platform.

Gartner predict that by 2021, over 50% of analytical queries will be generated via search, natural language processing, voice, or automatically generated.

So it is important to understand how each solution works. For example, some only search pre-built reports and dashboards, some only look at metadata, some merely return a list of matches, some provide a single answer or even worse a long list of ranked search results of pre-built reports that the user has to wade through.

Meanwhile, the newest breed of search-driven analytics engines search through all the underlying raw data, compute results, and then present charts and numbers based on those real-time calculations presented through best fit visualisations, for that particular question.

Questions to ask: Does your search bar, search across all of the underlying data or just across pre built reports?

4. Search Intelligence

No ifs, no buts, a search driven analytics platform should deliver a consistent and reliable answer so that business users can trust the answers they get from a BI solution.

Most businesses run on many different data sources, most users don’t understand how all this data relates to each other, which schema represents the underlying tables or which joins are needed to find an answer. For this reason, all this complexity should be kept away from the user.

A smart search-driven analytics platform should deliver an experience that recognises patterns, has spell check, understands synonyms and offers suggestions as they type based on other users activity – such as how Google’s predictive ‘type-ahead’ feature works. It’s also important that a standard user, can easily query and verify how results were calculated, without technical experience or help from the BI team.

Questions to ask: Can your users easily analyze results at granular level, such as daily, weekly, monthly etc. without involving the BI team and can they do this across billions of rows of data in less than a second?

5. Chart Creation

For a search driven-analytics product to be effective and therefore adopted widely throughout the business, it must remove as many barriers as possible. This isn’t the case with traditional or legacy BI tools where the wait time between the users query and the visualised result is often slow. On average it takes 5 days but we speak to some customers who say it can take weeks and even months before the business user gets the data they are looking for.

Instead, a user should ask a question, just like they do for example when asking Google ‘what is the weather tomorrow?’,¬†and the product itself should automatically create the dashboard on the fly – there should be no wait time. The user shouldn’t have to choose which type of chart they would like the numbers to present, the solution should present them with the best fit visualisation based on the question that was asked, but give the user an option to select different chart types if needed.

It’s reported that 23% of current BI users are comfortable creating charts & graphs. A good analytics or BI solution should democratise access to data, remove this complexity and enable the least technical users to ask questions and get on with their day.

Questions to ask: How are dashboards and charts created, who creates them and how long do they take to create?

6. Augmented Insights

AI Driven insights are all around us in our personal life, we just don’t notice them any more. Think about intelligent spam filters, blocking out unwanted emails or YouTube, where AI learns from your behavior to automatically recommend content that may resonate best for you.

Now think about your organisations data, there’s so much of it, it’s constantly growing, it’s scattered all over the place and with millions or often billions of rows of data, even knowing the right questions to ask becomes tricky. Because of this, real insights in your data are often missed and this can lead to huge amounts of waste and missed opportunity.

But what if your analytics solution could answer the questions you didn’t even think to ask? What if this intelligent machine could access huge data sets, generate thousands of questions across billions of data points and find hidden insights, outliers, anomalies, trends in your data. What then if it could surface these insights to you as a user, because it knows the type of person you are, it understands what’s important to you and it knows when these insights should be delivered. What if you could work with this machine, and help it to learn from you?

This is one of the fundamental shifts we have seen in recent years with analytics and BI solutions, where augmented data discovery is not only saving huge amounts of time and waste, but also making companies more profitable and increasing output.

Questions to ask: Does your solution have any AI, predictive or augmented data discovery features?

7. Speed & Performance

The power of Google is achieved because of it’s ability to search every website across the whole of the web. However some BI tools, often fall short of allowing users to analyse data across the whole of the business with some restricted to files on your local machine. IT and data teams are often left with having to make decisions around which data sets to include, as they know that reports can take overnight to run because of problems with scale and performance.

The key to a successful analytics or BI solution is it’s ability to deliver insights at scale, across all of your data tables and for all of your users. In just a few clicks, your users should be able to get the answers they need across billions of rows of data.

Questions to ask: How much data can your product handle and how many users can work with the data simultaneously?

8. Total Cost of Ownership / ROI

There are many things to think about when considering the cost of a new data analytics solution, naturally the value that can be delivered will depend on your use cases and where you are at in terms of your data journey.

Beside from the cost of the software itself, you should also consider;

  • Implementation Costs – what are the costs to deploy the product into your environment. A good BI solution should work straight out of the box and you should be able to easily connect to your different data sources.
  • Implementation time – is your analytics strategy tied into a new product launch. Time to value should be considered and in some cases it could be best to focus on specific and targeted use cases.
  • Infrastructure Costs – where will the technology be deployed, what hardware will be needed, what about storage, maintenance and support etc. Consider working with a partner that can offer a fully managed service, so you can scale up and down on demand.
  • Resource – how many technical staff will be needed to administer the solution. As we’ve mentioned earlier in the article, try and find a solution that offers true self service for the end user, without the need for a data team to keep churning out reports and dashboards.

These are all factors that should be considered when considering the ROI, and ensuring that your employees, partners and customers can make better fact driven decisions is crucial.

9. Security & Governance

Most data & BI solutions have enterprise grade security and governance built in these days and can integrate into your existing directory services through protocols like LDAP.

But how far does this go, does your product protect your data at the row, column or table level? Can you secure the search intelligence at the user level. Your search bar results should not identify to the user any underlying data that they do not have access to, and your search suggestions should obey security restrictions.

Questions to ask: Do the search box and search results feed into your access rights and rules, so users only see what they have access to see?

10. Training & Support

BI adoption today sits somewhere between 20 and 25% in an enterprise and many traditional & legacy BI solutions require lengthy training courses, for so called ‘Power Users’

At the same time, some of the most innovative and transformational technologies are becoming more and more like consumer facing applications that the end user can simply pick up and use, because it’s familiar to them – through innovations like ‘Search & NLP’.

This approach is offering an experience that is more user friendly and as well as driving transformation by enabling everyone to make better fact driven decisions, the end user training time is massively reduced.

Questions to ask: What is the training time and costs for non technical business users, business analysts and IT/BI teams?


Data is growing faster than ever before, we need to move quicker than ever before and we need to be able to get insights to the right people, across the organization, at the right time. With all the recent changes that have shaped our World throughout 2020, we will see on the one side disruption but on the other side transformation and exciting new businesses and operating models – data will play a key role in your company’s future.

For a full list of BI Platforms, click here.

For further help with selecting the right solution, check out our other blog – Data Analytics & Business Intelligence comparison guide

About Geniusly

Geniusly connects people who need tech, with people who know tech to help them find the right solution for their requirements. Our industry specific advisors will save you weeks of research and avoid costly mistakes. In just a short call, our experts will understand your requirements, narrow down your search and make the necessary introductions. Best of all it’s free.

Get Advice – It’s Free – Speak to one of our experts today.

Related articles