Please make the changes to the project. I attached the required changes pdf. All the highlighted parts need to be changed or modified. Also attached code file with requirements pdf. Use google colab and Aima code library. please hire Prasun because he did our previous work.
PROJECT#2 – Raisin VERSION 3b updated 12-8, 2022 Collaborative Assignment (groups of 2-3) Use the AIMA-PYTHON library to explore the UCI raisin dataset You will be building classifier models, analyzing results, comparing models and offering improvements. GOAL: Explore AIMA Python modules for classification, UCI dataset, and metrics (AIMA & custom – NO sklearn) REQUIREMENTS: • Create classification models using AIMA learners (simple perceptron, ensemble, neural network); • Make use of AIMA performance metrics (Error ratio, accuracy_score, r2_score, grade_learner) & SciKit-Learn metrics (confusion matrix & classification report). [minimum of 2] • Useful modules will be found in learning4e, neural_networks, deep_learning4e • Sample code can be found in learning.ipynb, test_learning, Learning_RR_modified_Perceptron, Learners… ; • Code must be THOROUGHLY DOCUMENTED, with a complete General Header, and inline comments that will make SEARCHING for tasks completed easy to locate. If the grader is unable to quickly locate supporting code for each task below, the task will get 0 points.; • There will be a written report required, including a results table. TASKS • DATA o Load UCI raisin dataset o Assign the attribute names and source from the .txt file o Display details about the data, using DataSet attributes & assigning known details from UCI repository o Prepare data1 for training & validating models, reserving a subset of rows as a hold-out set for testing (see Data Engineering tips in footnotes) o Unknown data point2 has original feature values: [188888, 666, 333, 0.77, 191919, 0.74, 1666]. • PERCEPTRON Classifiers o Generate Simple PERCEPTRON classifiers from training data & record hyperparameter values & results (to table) for each new model below: ▪ Generate 1st model using all defaults ▪ Next model: change the learning rate, until you see improvement ▪ Next model: change the number of epochs, until you see improvement o Create a final Simple PERCEPTRON model w/best hyperparameters from above, or additional choices if appropriate/available, (fit to training data) o Record results when tested on training data, and separately on test data. Support your results and assessment with at least two different performance metrics, o Predict for Unknown data point. Record the prediction and any speculation on this outcome.2 • ENSEMBLE modeling o Use AdaBoost –or—EnsembleLearner to generate ensemble models, starting with defaults. o Next Models: Alter hyperparameter or learner, as appropriate, and record results (to table) o Record results when tested on training data, and separately on test data. Support your results and assessment with at least two different performance metrics, 1 - As mentioned in TIPS, you should follow best practices, to include: shuffle, convert-classes-to-Numbers, normalize features, and train-test-split. Be aware of when to pass full DataSet, examples only, features only, targets only, training-set only, test- set only. 2 Unknown Data Point is to be used to predict output class for a single data point , as requested in last bullet of the 3 model types. https://archive.ics.uci.edu/ml/datasets/Raisin+Dataset o Predict for Unknown data point. Record the prediction and any speculation on this outcome.2 • NEURAL NETWORK modeling o Generate multi-layer perceptron classifiers & record hyperparameters & results (to table) ▪ Generate 1st model using all defaults o Record results for each of: #hidden layers/nodes is (1,10), (2,10), (3,10) , and
<# of="" your="" choice=""> o Create a final NNet model by selecting the best hyperparameters from above; record results. o Record results when tested on training data, and separately on test data. Support your results and assessment with at least two different performance metrics, o Predict for Unknown data point. Record the prediction and any speculation on this outcome.2 DELIVERABLES: - G# will be your group number. • WRITTEN REPORT (.pdf) o Overview describing your general approach to the problem (1paragraph); o Analysis of performance (w/direct references to table); o Summary of what worked/didn't, and recommendations for how you might improve with future efforts. o Table of results (min. of 1 row per model) • G#p2.ipynb – CLEAN source file. o Include a general header (names, etc.) & generous comments to document your code o You are required to include ALL dependencies in your submission. o Remove any unnecessary code (unused functions, troubleshooting, etc.) o Explicitly note (w/comments) any classes or functions you wrote or modified • G#p2.pdf – .ipynb notebook, o Visible: all code cells & comments, o Visible: outputs from successful run & demonstrations that support your report & results table P2 Written Report Project 2: Raisin Fever Group Number: 02 By:Amritvir Randhawa Written Report The general approach to the problem: This collaborative assignment required us to create classification models using AIMA learners, precisely simple perceptron, ensemble, and neural networks. To accomplish this, we have (1) loaded and prepared the dataset for training; (2) generated a simple perceptron classifier and tuned it based on learning rate and a number of epochs; (3) generated an Adaboost-based ensemble model and tuned it based on the 'K' number; and (3) generated a neural network model and tuned it based on hidden layer sizes, learning rate, epochs, and activation function. Each cell block has been explained using comments, and meaningful variable names have been used. The original paper (Cinar et al., 2020), documents available on the aimacode/aima-python repository, and other websites have been consulted to derive the solution. Analysis of performance: (i) The good performance (85.8% on ALL datasets) of the simple perceptron model is an indication of its power. The high precision (0.90) shows the reproducibility of the results. Table 1 shows the experiments with the learning rate. The accuracy stagnates at about 0.1. Table 2 shows the experiments with epochs. The accuracy does not appreciably increase after 700 epochs. With these hyperparameters tuned, the accuracy of the perceptron model increased to 87.2% on the unseen validation dataset. (ii) The performance of the untuned adaboost-based ensemble classifier (85.7%) was similar to that of the perceptron model. 10 different values of the hyperparameter K were tried. It was seen that a plateau of accuracies formed around K = 35 and K = 40 (Table 3). By setting K to 35, the accuracy of the ensemble model increased to 87.2% on the unseen validation dataset. (iii) A multi-layer perceptron (here referred to as a neural network) classifier gave 85.4% accuracy when trained using 80% of the data and tested on ALL the data values. Multiple values of hyperparameters such as hidden layer sizes (Table 4), learning rate (Table 5), epochs (Table 6), and activation function (Table 7) were tried and the ones giving the highest accuracies were applied to the final model. The final model accuracy (85.6%) was a mere 0.2% up from the initial run. Summary and Recommendations: This study showed how to implement 3 different classifiers and how to tune their hyperparameters. An interesting finding is that although ensemble-based and neural network-based classifiers added more complexity to the classification, a simple perceptron model still gave comparable accuracies without much tuning. The neural network hyperparameter tuning requires more work. It has the capability to give better results. The approach adopted to derive the best hyperparameter values is not correct. A grid-based approach, that is, a combination of all hyperparameters much be tried to see which combination of hyperparameter values gives the best results. Finally, the results could also be compared by implementing some standard machine learning classifiers such as random forest, decision trees, and support vector machines - so as to support/refute the findings of Cinar et al., 2020. Table of results (1 row per model): (1) For a simple perceptron model, the hyperparameter learning rate Learning rates = [0. 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9] Accuracies = [0.5, 0.8655555555555555, 0.86, 0.8488888888888889, 0.8366666666666667, 0.83, 0.8255555555555556, 0.8211111111111111, 0.82, 0.8166666666666667] (2) For a simple perceptron model, hyperparameter epochs Epochs = [ 0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400] Accuracies = [0.5, 0.8533333333333334, 0.8644444444444445, 0.8611111111111112, 0.8644444444444445, 0.8666666666666667, 0.8688888888888889, 0.87, 0.87, 0.8711111111111111, 0.87, 0.8688888888888889, 0.8677777777777778, 0.8677777777777778, 0.8677777777777778] (3) For the ensemble model, hyperparameter K K = [ 5 10 15 20 25 30 35 40 45 50] Accuracies = [0.8533333333333334, 0.8566666666666667, 0.8566666666666667, 0.8566666666666667, 0.8577777777777778, 0.8566666666666667, 0.86, 0.86, 0.8577777777777778, 0.8566666666666667] (4) For the neural network model, hyperparameter hidden layers/nodes Hidden layer choices = [[7], [14], [20], [7, 14], [7, 14, 7], [14, 28, 14]] Accuracies = [0.8533333333333334, 0.8588888888888889, 0.86, 0.8555555555555555, 0.5, 0.6411111111111111] (5) For the neural network model, the hyperparameter learning rate Learning rates = [0. 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19] Accuracies = [0.5, 0.8522222222222222, 0.86, 0.86, 0.8488888888888889, 0.8422222222222222, 0.8344444444444444, 0.83, 0.8244444444444444, 0.8244444444444444, 0.8222222222222222, 0.82, 0.82, 0.82, 0.82, 0.8188888888888889, 0.82, 0.82, 0.8155555555555556, 0.82] (6) For the neural network model, hyperparameter epochs Epochs = [100 200 300 400 500 600 700 800 900] Accuracies = [0.8488888888888889, 0.8688888888888889, 0.8722222222222222, 0.8733333333333333, 0.8744444444444445, 0.8744444444444445, 0.8733333333333333, 0.8744444444444445, 0.8744444444444445] (7) For the neural network model, the hyperparameter activation function Activation functions = ['sigmoid', 'relu', 'tanh', 'elu', 'leaky_relu'] Accuracies = [0.8555555555555555, 0.5, 0.8522222222222222, 0.5, 0.5]