A Data Science Central Community
Second in a two-part series
Businesses keep vast troves of data about things like online shopping behavior, or millions of changes in weather patterns, or trillions of financial transactions — information that goes by the generic name of big data.
Now, more companies are trying to make sense of what the data can tell them about how to do business better. That, in turn, is fueling demand for people who can make sense of the information — mathematicians — and creating something of a recruiting war.
DJ Patil, with venture capital firm Greylock Partners, is on a perpetual manhunt, looking for a rare breed: someone with a brain for math, finesse with computers, the eyes of an artist and more.
"There's one common element across all these people that stands out above everything, and that's curiosity," Patil says. "It's an intense curiosity to understand what's behind the data."
He compares raw data to clay: shapeless until molded by a gifted mathematician. A good mathematician can write algorithms that can churn through billions or trillions of data points and show where patterns emerge.
For example, patterns indicated early on that mothers were heavy users of social networks, which in turn led to the creation of social circles. Patil says a good mathematician can figure out what matters and what doesn't in a huge trove of data.
Dylan Field, a junior at Brown University who is being recruited by big data companies, created this visual analysis of course data at Brown with fellow students. Each dot represents a course, and its size indicates how interdisciplinary it is. Similar courses are grouped together, showing how different academic fields relate. The information is then used for CourseKick, which helps students discover new courses to take.
The Climate Corp. uses 50 terabytes of live data — including weather measurements from 2.5 million locations and 150 billion soil observations — to create weather insurance for farmers. Its founders learned big data principles from their time at Google and now predict risk for weather events like droughts or flooding. This graphic is the first of its kind to visualize soil type; darker blue coloring shows a higher water holding capacity, better for growing crops, while lighter colors have less capacity.
"Everybody's looking for these people, because they know these individuals can move the needle by themselves. They're that impactful," he says.
Firms are going beyond offering piles of cash and equity to get these people. The chief executive officer at TellApart, a company that helps online marketers profile its customers, invited one math recruit to dinner with other tech luminaries to sell him on the company. Another California firm, Cataphora, which helps firms monitor employee behavior, opened a satellite office in Ann Arbor, Mich., so it could recruit from the University of Michigan's math department.
Often, executives say it's not just about money; they have to appeal to an ideal.
David Friedberg says his company does that by touting the power of risk management to change agriculture. Friedberg, chief executive officer ofThe Climate Corp., says crop insurance is basically nonexistent in large parts of Africa and Southeast Asia. He says the ability to better model changing climate patterns means his company can provide insurance to small farmers who otherwise might not take on the risk of farming more land.
"We can actually encourage agricultural development and provide a sustainable living for them," Friedberg says. "And that's one of the long-term missions of our organization."
The type of people drawn to this work aren't necessarily what you might expect. Greylock Partners' Patil says his successful recruits have included an oceanographer and a neurosurgeon, as well as people who barely graduated from high school but were brilliant at math. He approaches math majors the way baseball scouts look for young stars.
"I have a list that I track. There's one student who I think is phenomenal; I've been tracking him since he was 16," Patil says.
Read full article at http://www.npr.org/2011/11/30/142893065/the-search-for-analysts-to-...