A Data Science Central Community
Analytics projects range in scale and complexity, but every analyst eventually runs into the same problem. Regardless of the kind of data you’re working with or the size of the project, from a mid-sized manufacturer to a project that truly is Big Data, businesses look to analytics and BI tools because the scope and complexity of the data is obscuring critical information. Organizations spend a great deal of time and resources creating models and running queries trying to get a more complete picture of the data set. BI and visualization tools give a visual representation of the data and make the search process faster and more efficient.
Regardless of the tools that are available or the problems that need to be solved, what Big Data is missing is the bigger picture. Data sets don’t come with road maps, so a good deal of every analytics project is data exploration. Each query tells you a little bit more about the data, filling that picture in bit by bit until you find what you are looking for. It’s possible that a combination of luck and a good understanding of your data will get you to an answer quickly, but more often than not much digging is done before results emerge. Modern tools help the user to ask questions faster, but they don’t tell you where to search. The true costs of that searching are difficult to calculate, but recent news stories claim that many organizations are not realizing the expected returns from their Big Data projects.