STATISTICA Wins International CIAC Credit Scoring Competition

BRICS-CCI-CBIC-logo-150The BRICS-CCI & CBIC 2013 Congress in Brazil recently published results of its first data mining algorithm competition, announcing that StatSoft’s solution achieved first-place and second-place wins for STATISTICA Data Miner.

The international Computational Intelligence Algorithm Competition (CIAC), co-organized by NeuroTech S.A., focused for the first time on data mining applied to credit scoring. Participants were required to address the effects of temporal degradation of performance and seasonality of payment delinquency. Operating under the “Team Sandvika” name, StatSoft Norway’s Country Manager Knut Opdal and Rikard Bohm submitted a solution that demonstrated the superiority of modern data mining methods over the more traditional logistic regression method.

Their STATISTICA model placed first when judged for fitting of estimated v actual delinquency of approved credit applications, and it placed second when judged for robustness against performance degradation over a multi-year data set.

“We used this competition as an opportunity to compare the performance of different classes of predictive models,” Opdal and Bohm stated. “In particular we wanted to compare the industry standard method (logistic regression) with Boosting Trees, Random Forrests, MARSplines and neural network.”

screenshot from Opdal/Bohm CIAC credit scoring white paperOpen to participants from academia and industry, the competition consisted of two tasks. Task 1, which focused on performance degradation, was evaluated based on the usual area under Receiving Operator Characteristic (ROC) curve metrics. Task 2 focused on fitting of estimated v actual delinquency of those credit applications approved by the Task 1 model. CIAC organizers claimed this second task represents a realistic innovation in data mining competitions worldwide by emphasizing the relevance of the quality of future delinquency estimation instead of the usual lowest future average delinquency.

As a top performer in these two tasks, StatSoft Norway was invited to present its winning solution in a special CIAC track scheduled during the three-day BRICS-CCI & CBIC Congress in September. Opdal arranged via Skype to deliver the white paper, “Benchmarking of Different Classes of Models Used for Credit Scoring,” in which he described the methodology and techniques he and Bohm applied.

In the presentation Opdal and Bohm conclude that, as volume of data and/or variables increases, score model performance using modern data mining techniques (e.g., Boosting Trees, MARSplines, Neural Network) is “significantly better than (with) traditional scorecards.”

Competition results are listed here.

Watch Knut’s oral presentation of STATISTICA’s winning solution here.

About BRICS-CCI & CBIC

BRICS is an acronym for the economic group of Brazil, Russia, India, China, and South Africa, which held its first Congress on Computational Intelligence (BRICS-CCI 2013) during September, alongside the 11th bi-annual Brazilian Congress on Computational Intelligence (CBIC). The Congress’ website describes the objective of BRICS-CCI 2013 is to provide a high-level international forum for scientists, researchers, engineers, and educators to disseminate their latest research results and exchange views of future research directions.

Advertisements

About statsoftsa

StatSoft, Inc. was founded in 1984 and is now one of the largest global providers of analytic software worldwide. StatSoft is also the largest manufacturer of enterprise-wide quality control and improvement software systems in the world, and the only company capable of supporting its QC products worldwide, with wholly owned subsidiaries in all major markets (StatSoft has 23 full-service offices, on all continents), and its software is available in more than 10 languages.

Posted on October 16, 2013, in Uncategorized. Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: