STATISTICA Wins International CIAC Credit Scoring Competition
The BRICS-CCI & CBIC 2013 Congress in Brazil recently published results of its first data mining algorithm competition, announcing that StatSoft’s solution achieved first-place and second-place wins for STATISTICA Data Miner.
The international Computational Intelligence Algorithm Competition (CIAC), co-organized by NeuroTech S.A., focused for the first time on data mining applied to credit scoring. Participants were required to address the effects of temporal degradation of performance and seasonality of payment delinquency. Operating under the “Team Sandvika” name, StatSoft Norway’s Country Manager Knut Opdal and Rikard Bohm submitted a solution that demonstrated the superiority of modern data mining methods over the more traditional logistic regression method.
Their STATISTICA model placed first when judged for fitting of estimated v actual delinquency of approved credit applications, and it placed second when judged for robustness against performance degradation over a multi-year data set.
“We used this competition as an opportunity to compare the performance of different classes of predictive models,” Opdal and Bohm stated. “In particular we wanted to compare the industry standard method (logistic regression) with Boosting Trees, Random Forrests, MARSplines and neural network.”
Open to participants from academia and industry, the competition consisted of two tasks. Task 1, which focused on performance degradation, was evaluated based on the usual area under Receiving Operator Characteristic (ROC) curve metrics. Task 2 focused on fitting of estimated v actual delinquency of those credit applications approved by the Task 1 model. CIAC organizers claimed this second task represents a realistic innovation in data mining competitions worldwide by emphasizing the relevance of the quality of future delinquency estimation instead of the usual lowest future average delinquency.
As a top performer in these two tasks, StatSoft Norway was invited to present its winning solution in a special CIAC track scheduled during the three-day BRICS-CCI & CBIC Congress in September. Opdal arranged via Skype to deliver the white paper, “Benchmarking of Different Classes of Models Used for Credit Scoring,” in which he described the methodology and techniques he and Bohm applied.
In the presentation Opdal and Bohm conclude that, as volume of data and/or variables increases, score model performance using modern data mining techniques (e.g., Boosting Trees, MARSplines, Neural Network) is “significantly better than (with) traditional scorecards.”
Competition results are listed here.
Watch Knut’s oral presentation of STATISTICA’s winning solution here.
About BRICS-CCI & CBIC
BRICS is an acronym for the economic group of Brazil, Russia, India, China, and South Africa, which held its first Congress on Computational Intelligence (BRICS-CCI 2013) during September, alongside the 11th bi-annual Brazilian Congress on Computational Intelligence (CBIC). The Congress’ website describes the objective of BRICS-CCI 2013 is to provide a high-level international forum for scientists, researchers, engineers, and educators to disseminate their latest research results and exchange views of future research directions.