Aim:

              The main of the project is to categorize the dataset using machine learning algorithm and predict the data using Apache Spark ML Lib.

Abstract:

               In the past few years, Peer-to-Peer lending (P2P lending) has grown rapidly in the world. The main idea of P2P lending is disintermediation and removing the intermediaries like banks. For a small business and some individuals without enough credit or credit history, P2P lending is a good way to apply for a loan. However, the fundamental problem of P2P lending is information asymmetry in this model, which may not correctly estimate the default risk of lending. Lenders only determine whether or not to fund the loan by the information provided by borrowers, causing P2P lending data to be imbalanced datasets which contain unequal fully paid and default loans. Imbalanced datasets are quite common in the real worlds, such as credit card fraud in transactions, bad products in the plant and so on. Unfortunately, the imbalanced  data are unfriendly to the normal machine learning schemes. In our scenario, models without any adaptive methods would focus on learning the normal repayment. However, the characteristic of the minority class is critical in the loaning business. In this study, we utilize not only several machine learning schemes for predicting the default risk of P2P lending but also re-sampling and cost-sensitive mechanisms to process imbalanced datasets. Furthermore, we use the datasets from Lending Club to validate our proposed scheme. The experiment results show that our proposed scheme can effectively raise the prediction accuracy for default risk.

Proposed System:

             The P2P lending datasets contain many attributes which are empty for most records. Therefore, we remove these attributes and modify the nominal features by using one-hot- encoding technique that can transform nominal features to be a format suitable for classification. For instance, we have a feature ‘‘purpose of the loan’’ which has string value such as ‘‘Car’’, ‘‘Business’’, and ‘‘Wedding’’. Normally, we use ordinal value to encode these to be numbers such as 0, 1, and 2. However, in machine learning schemes, different categories have the same weight. Thus, the ordinal technique cannot be implemented in machine learning because the lowest and the highest value will affect the classification result. One-hot-encoding uses one Boolean column for each category which has different weight. We use libsvm library to convert the features into encoded format. Then the dataset is sent to training and prediction with analytics.