Subscribe to DSC Newsletter

I've read an article published in multiple newspapers, about a company - Digital Risk - mentioning a product called Veritas that is supposedly better than FICO scores to predict risk of default on a loan. It looked like paid advertising, and the company in question was touting the fact they use "statistical clusters" - a technique that every scoring company (including myself with my scores based on hidden decision trees) extensively use.

The core argument was that FICO scores failed to catch people with very good credit who decided to do a strategic default on their underwater home. I believe that 

  • The situation in question is very rare (this type of market collapse happens every 80 years or so) and thus creates chaos in algorithms - any algorithm.
  • In 60 years you'll have another similar situation where all predictive algorithms fail - maybe a pandemic killing lots of people and forcing them to abandon cities and stop paying mortgage. I believe Digital Risk's algorithms will not work in this case.
  • I also believe that FICO (or any company for that matter) learn from their mistakes and will improve their algorithms. They also have access to a very larhe volume of data.

Note: I'm not associated with FICO nor Digital Risk. I also think that it would be interesting to design an algorithm to detect articles that are being sponsored by a third party, or in order word, how do you detect an hidden sales pitch from a neutral article in major news outlets?

Here's the article

Lenders need to use more sophisticated assessment tools for credit histories, a mortgage analytics firm says

By Kenneth R. Harney.

Do you ever get the feeling that your credit score doesn't adequately portray your true risk as an applicant for a home mortgage?

If your FICO score is a subpar 690 but you know that you are a solid candidate for a loan, do you think that lenders' heavy dependence on credit scores is unfair to you as an individual?

You've got some company. Digital Risk, a mortgage analytics firm, is mounting an unusual frontal assault on one of the lending industry's sacred cows. It argues that credit scores such as FICO failed to predict large numbers of defaults during the mortgage bust years — most notably thousands of “strategic” walkaways by borrowers with high scores — because they could not anticipate homeowners' reactions to economic stress. Unless lenders use more sophisticated assessment tools that incorporate far more than credit histories, Digital Risk says, they may be misjudging not only many of today's high-risk borrowers but other applicants who are safer bets than their credit scores suggest.

Views: 3074

Reply to This

Replies to This Discussion

I guess it depends what you mean by "better". Veritas claims their product is "better" than FICO and then goes on to say they have a completely different product - apple - oranges, and Veritas says so themselves. FICO is a generic credit score, reflecting repayment in general. Veritas says they do better for one very specific problem (high credit rating strategic default) and for one specific kind of laon - mortgages. Any competently built model will do better for a very narrowly-defined problem than a one-size-fits-most solution like FICO.

Sound like advertizing to me - as you point out, only a person would no background in modern modeling would be impressed by the use of decision trees - like promoting an automobile for having electronic ignition instead of a distributor.

Nice post.

I think the other question is "How is it going to be used?" - the fico score only gives a score of a persons current situation(the risk)...which in turn gives only an idea of the likelihood of paying obligations.  The fico score was never meant to be a predictor of default, from what I know (see [1,2,4]). At least that was understood when I worked for an Experian affiliated credit bureau for a few years in the late 90s, things can change of course.  In reference [2], down towards the bottom the site states "They reflect only your likelihood to repay debt responsibly based on your past credit history and current credit status."

Fair Isaac company has another product called Fico Expansion Score which essentially is an odd-to-score or essentially predicts the credit risk [3].

The article is misleading when it says referring to Digital Risk..."...that credit scores such as FICO failed to predict large numbers of defaults ..."  The FICO score alone was never built to predict, it is only used to asses risk.

Usually these larger banks have statisticians that work on credit models for loans and the models must be approved by regulators, FDIC.  See the FDIC website for more information see [4] for a start. 

1.  http://www.fico.com/en/products/scoring/pages/fico-score.aspx

2.  http://www.experian.com/credit-education/understanding-credit-score...

3.  http://www.fico.com/en/products/scoring/pages/fico-expansion-score....

4.  http://www.fdic.gov/regulations/examinations/credit_card/pdf_versio...

Hi Lance,

Do you know why we have regulators to regulate statistical models (or their output)? Are they smarter than the statisticians hired by banks or the private sector, to tell them what is valid or not?

Or is it yet another example of red tape, with useless bureaucrats paid by the taxpayer?

Sorry if I'm butting in here - the question was directed to Lance.

Are the regulators smarter? No. Independent verification? Yes. Financial institutions have even been to disregard the work of their statisticians and assume more risk than is warranted. In any case - whether the statisticians have been ignored or not, the regulators provide peer review.

A lot of the people doing ecompnic models are theoretical phsycists - I'm one of them myself, hence my ionteresat in answering this question. I would expect a physicist to submit a model for peer review - just the same for economic models. Regulatory provides this and it is especially important where financial models modelers or their supervisors are subject to forces other that just getting the best numbers.

Does this regulation apply to banks / credit card companies only? Let say I create my own scoring company (offering generic scores the way FICO does), would my scores have to comply with some laws / governmental compliance?  My guess is NO, but I might not be able to sell my scores to banks. I guess I could offer them to other companies (recruiters, employers, landlords, private lenders, etc.) without government interference.

@Vincent:

You could make a bizillion scoring models and if the banks/anyone can use it properly then all is good.  From the FDIC website, the banks just have to make sure that the scores fit with what they are doing AND that is where my understanding of where the governments step in to give a stamp of approval to the banks/credit unions/credit cards(now per Federal Reserve).

Then again, things can change since we are supposedly trying to come out of a recession.  I do not know what exactly they require as things are change from what they were.

Hi Mirko,

The only thing that I am aware of is what David has said about Independent Verification.  A more full explanation is in the fourth reference above which is on the FDIC.gov website.  The reference gives a lot more information than I could give.

Banks at times get side tracked into making money and NOT paying attention to reality.  This is one of the contributing factors of the current economic crisis with all the OFF balance sheet activities - a quick buck and failing to look at reality of a loan made to a customer.

RSS

Follow Us

On Data Science Central

On DataViz

On Hadoop

© 2017   AnalyticBridge.com is a subsidiary and dedicated channel of Data Science Central LLC   Powered by

Badges  |  Report an Issue  |  Terms of Service