4/5/23, 12:36 PM Lecture-11-K Nearest Neighbors-Part2 - Jupyter Notebook
Lecture 11-K Nearest Neighbour-Part 2
K Nearest Neighbors with Python
Import Libraries
In [10]: 1 import pandas as pd
2 import seaborn as sns
3 import matplotlib.pyplot as plt
4 import numpy as np
5 %matplotlib inline
Get the Data
In [11]: 1 df = pd.read_csv('Downloads/KNN_Project_Data')
In [12]: 1 df.head()
Out[12]:
XVPM GWYH TRAT TLLZ IGGA HYKR EDFS
0 1636.670614 817.988525 2565.995189 358.347163 550.417491 1618.870897 2147.641254 33
1 1013.402760 577.587332 2644.141273 280.428203 1161.873391 2084.107872 853.404981 44
2 1300.035501 820.518697 2025.854469 525.562292 922.206261 2552.355407 818.676686 84
3 1059.347542 1066.866418 612.000041 480.827789 419.467495 685.666983 852.867810 34
4 1018.340526 1313.679056 950.622661 724.742174 843.065903 1370.554164 905.469453 65
Standardize the Variables
In [4]: 1 from sklearn.preprocessing import StandardScaler
In [5]: 1 scaler = StandardScaler()
In [13]: 1 scaler.fit(df.drop('TARGET CLASS',axis=1))
Out[13]: StandardScaler()
localhost:8890/notebooks/Lecture-11-K Nearest Neighbors-Part2.ipynb 1/6
4/5/23, 12:36 PM Lecture-11-K Nearest Neighbors-Part2 - Jupyter Notebook
In [16]: 1 scaled_features = scaler.transform(df.drop('TARGET CLASS',axis=1))
In [20]: 1 df_feat = pd.DataFrame(scaled_features,columns=df.columns[:-1])
2 df_feat.head()
Out[20]:
XVPM GWYH TRAT TLLZ IGGA HYKR EDFS GUUB MGJM
0 1.568522 -0.443435 1.619808 -0.958255 -1.128481 0.138336 0.980493 -0.932794 1.008313
1 -0.112376 -1.056574 1.741918 -1.504220 0.640009 1.081552 -1.182663 -0.461864 0.258321
2 0.660647 -0.436981 0.775793 0.213394 -0.053171 2.030872 -1.240707 1.149298 2.184784
3 0.011533 0.191324 -1.433473 -0.100053 -1.507223 -1.753632 -1.183561 -0.888557 0.162310
4 -0.099059 0.820815 -0.904346 1.609015 -0.282065 -0.365099 -1.095644 0.391419 -1.365603
Train Test Split
In [21]: 1 from sklearn.model_selection import train_test_split
In [22]: 1 X_train, X_test, y_train, y_test = train_test_split(scaled_features,df['TA
2 test_size=0.30)
Using KNN
Remember that we are trying to come up with a model to predict whether someone will TARGET
CLASS or not. We'll start with k=1.
In [23]: 1 from sklearn.neighbors import KNeighborsClassifier
In [24]: 1 knn = KNeighborsClassifier(n_neighbors=1)
In [25]: 1 knn.fit(X_train,y_train)
Out[25]: KNeighborsClassifier(n_neighbors=1)
In [26]: 1 pred = knn.predict(X_test)
Predictions and Evaluations
Let's evaluate our KNN model!
In [27]: 1 from sklearn.metrics import classification_report,confusion_matrix
localhost:8890/notebooks/Lecture-11-K Nearest Neighbors-Part2.ipynb 2/6
4/5/23, 12:36 PM Lecture-11-K Nearest Neighbors-Part2 - Jupyter Notebook
In [28]: 1 print(confusion_matrix(y_test,pred))
[[109 45]
[ 33 113]]
In [29]: 1 print(classification_report(y_test,pred))
precision recall f1-score support
0 0.77 0.71 0.74 154
1 0.72 0.77 0.74 146
accuracy 0.74 300
macro avg 0.74 0.74 0.74 300
weighted avg 0.74 0.74 0.74 300
Choosing a K Value
Let's go ahead and use the elbow method to pick a good K Value:
In [30]: 1 error_rate = []
2
3 # Will take some time
4 for i in range(1,40):
5
6 knn = KNeighborsClassifier(n_neighbors=i)
7 knn.fit(X_train,y_train)
8 pred_i = knn.predict(X_test)
9 error_rate.append(np.mean(pred_i != y_test))
localhost:8890/notebooks/Lecture-11-K Nearest Neighbors-Part2.ipynb 3/6
4/5/23, 12:36 PM Lecture-11-K Nearest Neighbors-Part2 - Jupyter Notebook
In [31]: 1 plt.figure(figsize=(10,6))
2 plt.plot(range(1,40),error_rate,color='blue', linestyle='dashed', marker='
3 markerfacecolor='red', markersize=10)
4 plt.title('Error Rate vs. K Value')
5 plt.xlabel('K')
6 plt.ylabel('Error Rate')
Out[31]: Text(0, 0.5, 'Error Rate')
localhost:8890/notebooks/Lecture-11-K Nearest Neighbors-Part2.ipynb 4/6
4/5/23, 12:36 PM Lecture-11-K Nearest Neighbors-Part2 - Jupyter Notebook
In [32]: 1 # FIRST A QUICK COMPARISON TO OUR ORIGINAL K=1
2 knn = KNeighborsClassifier(n_neighbors=1)
3
4 knn.fit(X_train,y_train)
5 pred = knn.predict(X_test)
6
7 print('WITH K=1')
8 print('\n')
9 print(confusion_matrix(y_test,pred))
10 print('\n')
11 print(classification_report(y_test,pred))
WITH K=1
[[109 45]
[ 33 113]]
precision recall f1-score support
0 0.77 0.71 0.74 154
1 0.72 0.77 0.74 146
accuracy 0.74 300
macro avg 0.74 0.74 0.74 300
weighted avg 0.74 0.74 0.74 300
localhost:8890/notebooks/Lecture-11-K Nearest Neighbors-Part2.ipynb 5/6
4/5/23, 12:36 PM Lecture-11-K Nearest Neighbors-Part2 - Jupyter Notebook
In [35]: 1 # NOW WITH K=3
2 knn = KNeighborsClassifier(n_neighbors=23)
3
4 knn.fit(X_train,y_train)
5 pred = knn.predict(X_test)
6
7 print('WITH K=30')
8 print('\n')
9 print(confusion_matrix(y_test,pred))
10 print('\n')
11 print(classification_report(y_test,pred))
WITH K=30
[[114 40]
[ 20 126]]
precision recall f1-score support
0 0.85 0.74 0.79 154
1 0.76 0.86 0.81 146
accuracy 0.80 300
macro avg 0.80 0.80 0.80 300
weighted avg 0.81 0.80 0.80 300
In [ ]: 1
localhost:8890/notebooks/Lecture-11-K Nearest Neighbors-Part2.ipynb 6/6
