Subscribe to DSC Newsletter

The next revolution in analytics: it's not about software, it's about data

It is about integrating external data sources in your data warehouse, and leveraging this data to answer questions such as "why are we losing so many users last month" or "why do we have so few new users recently", or "what new product / feature should I produce". The answer (and the cure) might not come from within your internal data, but from the outside:

- what are my competitors up to?
- what do my clients / employees write on Facebook, Twitter or elsewhere?

The external data in question can be gathered and analyzed using web crawling and text mining techniques, or surveys - to automatically find out and summarize what is being said about your company... and about your competitors. Combined with internal data, it could answer critical business questions.

In my opinion, the potential in properly exploiting external data far exceeds results that you could get from improved software (cloud, analytics as a service, hidden decision trees etc.) I believe the future of analytics is more about finding relevant data (and identifying the right metrics - this is absolutely critical and I will discuss it later), than software improvements.

Views: 1458


You need to be a member of AnalyticBridge to add comments!

Join AnalyticBridge

Comment by Jeffrey C Howard on November 22, 2010 at 8:12am
The external data idea is an intriguing one. We have customers that acquire external data (such as telecom tariff tables, etc.) from known, tightly controlled corporate or government agency sources. But how does one vet the quality and reliability of more ad-hoc sources to avoid corrupting the overall quality and credibility of the analytics?
Comment by Vincent Granville on November 20, 2010 at 8:12pm
Ralph: what do you think about techniques such as fuzzy or probabilistic joining, where 2 records from 2 different data sets are assigned a probability to represent the same user? Do you think they are useful to reconcile internal with external data?
Comment by Ralf Klinkenberg on November 20, 2010 at 7:20pm
Solutions and services like RapidSentilyzer [ ] automatically collect this kind of data from online media like news, internet discussion groups, forums, web blogs, etc. to gain customer insight and competitive intelligence allowing you to better understand and react faster to the needs and desires of your customers and to detect trends, risks, chances, and issues very early.
Comment by Ralph Winters on November 20, 2010 at 10:45am
For internal Surveys, yes. You can always tie that back to the individual. For the other kind of data, web crawling, sentiment etc. that is important too, but is somewhat disengaged from your internal data if you do not have a key. Somewhat like having two independent samples. The big challenge in data warehouse is tying the two together. I consider the software a separate issue, I don't see what that has to do with the data.
Comment by BR Deshpande on November 19, 2010 at 4:32am
I second this opinion. There are also a couple of excellent discussion items in the LinkedIn sister group which indicate this trend among practioners.

One key challenge that people seem to be facing is determining answers to qustions like what data do we need, what are the right tools to use to get the answers we want, how to interpret/communicate the answers in terms that are easily comprehended by the ultimate users of analytics - management.
Comment by Jaime Fitzgerald on November 19, 2010 at 3:05am
I could not agree more. The frontier of innovation has moved from software design and data sourcing, data management, and to your point, monetization by doing the right analysis and deriving the right metrics to achieve your business goals. I like your post a lot.

On Data Science Central

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service