Subscribe to DSC Newsletter

The screen shot below says it all: candidates for data science positions are plentiful, considerably more abundant than job openings: more than 600 LinkedIn users applied for this data science job. Even less prestigious companies routinely attract more than 50 applicants per job opening.

In view of this, you would thing that getting an expensive analytic degree is a waste of money (except a degree from MIT, Stanford, CMU, Northwestern and a few other select schools), and applying for a data science position is a waste of time (you would be competing with top candidates who apply to all the advertised positions). The problem is actually a bit more complicated. The root causes and solutions are as follows:

  • Companies want candidates with very deep rather than broad expertize, are not willing to accept telecommuting, and will not train a new employee, and in some cases only hire Ivy league candidates. In doing so, they drastically restrict the pool of of potential employees
  • Many university curricula are outdated, so despite the volume of applicants, hiring managers complain that very few have the right skill set. Few candidates are willing to acquire these new skills (e.g. Mapreduce), although it can be learned at no cost if you have an Internet connection and a browser.
  • Data Scientists are not properly used or hired. In the case of Facebook for instance, one might ask how - despite all the great scientists and great data that they collect about users - they generate so little revenue per page impression. They should generate 10 times more revenue if data science was fully and properly leveraged by top management, read comments here for details. In this case, the issue is probably poor communication between top management and data scientists: a solution to significantly increase ad revenue by optimizing ad relevancy is described in my free eBook, so there is no excuse for poor performance. The same can be said about many companies (Google under-utilizing its Internet real estate, Microsoft having very poor marketing campaigns and not hiring the right people) and many problems such as spam detection or fraud detection. 
  • There are many alternate options outside traditional employment for data scientists, and as a data scientist, you should consider these new options.

Finally, an unexpected consequence is a rise in sophisticated fraud, as an oversupply of unemployed math PhD's with great expertize end up working for rogue organizations (their only choice), while government and other organizations fail to hire the best people, for whatever reasons.

Views: 11281


You need to be a member of AnalyticBridge to add comments!

Join AnalyticBridge

Comment by jwinburnnc on June 17, 2012 at 6:55am

In my most humble opinion, I believe our field is still in it's infancy. Most employment opportunities are with newer companies with quantitative methods at their core.

I think current management in many, if not most companies, generally, don't understand the powerful impact of quantitative methods in business. I think this will change in time. Even SAS, until very recently, would frown upon an employee mentioning the word "Bayesian".

Generally, as an executive level manager with data science responsibilities, I tend to select and hire the best "critical thinkers" I can find then develop their quantitative skills... and my experience has been that current candidates with a pure math or statistics education seem to struggle a bit more than candidates with physics /applied math backgrounds. 

Comment by Mike O'Neil on June 17, 2012 at 2:31am

The advert above looks just like a a large fishing net.  A recruiter trying to develop a collection of resumes that can be used as putty. Throw it at the wall and see what sticks.  High volume low success rate approach to making money in the recruitment industry.

Comment by Mike O'Neil on June 17, 2012 at 2:24am

Hey Angela,

you discovered my secret.  I have been A-B testing my job adds for years.  Amazing what a different headline can pull.

What is worse than using the word "analytics" in an advert when there is no analytics required of the role is using "manager" when no management is requred. Just remember the average age of the recruiting industry is 24, the average duration of stay is 14 months, and the industry is heavily biased in hair color, and shortness of skirt.

Comment by Mike O'Neil on June 17, 2012 at 2:17am

600 applicants for a single role is a pain in the butt if you are a hiring manager running the process yourself. Quantity ^= Quality. But then if you are a recruiter who makes money by throwing putty at the wall, hoping that some of it sticks, 600 CV's is a treasure trove.

Data Science has already started to over rate itself.  It is just another skillset. Like all skill sets there is a range of breadth and depth, and there are the fakes. Get over yourselves guys. Next year there will be a new epithet for you to grab and apply to yourself, and we can all move on. 

The key is the value generated.  There are lots of highly qualified individuals doing lovely work producing models with great lift.  When they have no idea how to monetise the insight, the value is just about zero, except that another individual might see how it could be applied and how to make money from it.

And god save us from the certifiers. They are the ones who should be certified. Lock them up in the insane asylum. Long live freedom to work at anything without requiring certification. The certification score is a risk score. It has the same limitations that a risk score produced by an analytical model has.

Vince - keep at it. Love the rants and raves.

Comment by QUYEN KIET on June 12, 2012 at 9:46am

Brian, to take the "curriculum" a step further, data discovery and analytics should be a dedicated major in and of itself; the first 3 years to establish a strong foundation in social/behavior science, math/statistics, algorithm/ICS with the 4th year dedicated to bringing it all together in a clinical environment of data discovery/analytics, which may be further explored in a masters program. Data discovery can even be split into sub-sets of data mining/analysis and predictive algorithms/forecasts. There is just so much there and current college courses are very far behind the curve. Heck, most universities have no courses in any of the popular web programming languages like Ruby, Python or PHP (most are happy to have Java instructors). Only this year did UC Irvine begin teaching Python - and it was by a Java professor learning the language as he went along with the students (a complete failure by all accounts).

Comment by Brian Bailey on June 12, 2012 at 8:25am

Until there are degrees or certifications that address these skills, hiring managers will continue to be confused. These are skills that clearly my math degree did not address, but I wouldn’t say that the typical computer science degree isn’t much better. We need more universities to adapt their curriculum towards data discovery.

Comment by David Rogers on June 11, 2012 at 7:38pm

The premise here doesn't make any sense, given the situation and variables: many people clicked on a job post; therefore, many people have that particular skill; thus, analytics talent is plentiful

So 608 people clicked? Of those, how many intended to apply vs. those that were curious (maybe wanted to see if they could learn more about the company...). And of those that intended to apply, of the 608, how many are really qualified? When LinkedIn let's you apply for a role by just clicking once and sending your profile, doesn't everyone and his brother do so? (Isn't it just like the 1 great university you apply to knowing you will not get into, but hey, what if the stars were aligned...?  It's not like these 2 finger clicks and 4 seconds of your life cost you much.)

I am curious to know about another situation that is not as easy- how many people (for a very similar post) clicked on to apply, left LinkedIn, went to the company site, then applied through a taleo system begin frustrated for 20 minutes navigating through? Similar situation with a not-so-easy application process.

That would be a better measure of someone who at least thought they were qualified and were interested in that sort of a role.... but still does not measure talent- just personal bias of talent at best, and most probable (IMHO) more of a view into the talent one is moving toward, maybe not 100% acquired, yet. Isn't that how many find their next job?

In any case, I don't agree with the author. I have been on the hiring side of (digital) analytics for the past handful of years and have not had it easy with many local qualified people to choose from. In fact, I have seen it worsen as more organizations are getting on board using analytics to drive business. Take a look at some current salary guides as an indication of how well we are paying top analytics talent compared to other roles in different industries with similar schooling and work experience- it's quite shocking... 

Comment by Angela Waner on June 11, 2012 at 3:27pm

Agree that the ad is for a computer programmer with an interest in analytics. The hiring manager probably decided to use the phrase "data scientist" because it is trendy. They want a BS, MS, or PhD in computer science.

My employer has a department of computer programmer/math geeks. Trying to find this combination of skills is like fishing. You have to try different bait (keywords).

Occasionally you will see the same job posted with a different headline two weeks later to get discovered by a different group of people. I have actually seen a company that looked like they did a A/B test with their job ads to see which one got the applicants that "fit" better into the job's needs.

Comment by Dr. Steven Struhl on June 11, 2012 at 2:26pm

First, friends, I would not "thing" anything about this--I'm not trying to offend, but merely to point out that language has largely lost all meaning in job postings, and at times even in discussing the postings. Stated qualifications are usually empty phrases, or not in line with the detailed requirements of the job as stated--so which is inaccurate, the verbiage outlining the job or the bullet points beneath? Worse, if grammar, usage and spelling are given only passing nods (as in letting the computer flag what it considers errors), the chances of communicating anything whatsoever about the job in question are slight. I believe job-seekers have a sense that this is so, and so many will throw a resume at anything that looks even faintly plausible. Since (anecdotally, at least) nearly nobody seems to love her/his job any longer, and there is still historically high unemployment, new-job searches are probably at an all time high. So speculating on the depth of talent based on responses to job postings may prove to be quite difficult. The broader question of what to do about inaccurate job postings, or worse, those that are deliberately misleading, is perhaps another issue.

Comment by Salvatore Polizzi McDonagh on June 11, 2012 at 1:48pm

This is a perennial problem. Charlie and Quyen express the misconception in premise of the article perfectly.

@David - "Educate hiring managers" is not likely to happen in response to any shortage or abundance of skills. Even the biggest and "best" employers base their screening of candidates on criteria that bear little or no correlation to job performance. Until HR becomes a data driven discipline - and I do not see any evidence of this even at places like Google or LinkedIn, although I'd be delighted to be corrected on this (mis)perception - organisations will continue to hire people based on subjective criteria.

The bottom line: Get an introduction to the hiring manager from a mutual contact that (s)he trusts, and make sure you are likeable during the interview(s). The technical abilities in this (and any other) scientific discipline range in orders of magnitude, but are rarely measured objectively. Your bosses opinion of you matters far more than your ability to build useful prediction models when it comes to promotion and recognition. It has little to do with where you got your degree (unless the hiring manager got theirs there too, or has a fear of people who got theirs there), or how good you are at the job. It has everything to do with how much of a perceived threat or career booster you are to the hiring manager, and how well you will get on with them and the rest of the team. Even data scientists are humans, and therefore subject to the same subconscious evolutionary drives, that cause them to make the same irrational decisions, and then backfill the logical story to justify the decision to themselves and whoever else cares.

Disclaimer: This is all subjective opinion based on experience that is biased by my perception of my past employers, my (mis)understanding of the reasons for my application rejection by potential employers, my own hiring of employees for previous employers and in my own businesses, my (mis)interpretation of evolutionary psychology, amongst other biases and cognitive errors and omissions. Therefore it is not empirically valid and should not be construed as scientific truth. Why are you still reading this comment?

On Data Science Central

© 2020 is a subsidiary and dedicated channel of Data Science Central LLC   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service