Diagnosing Diseases Using kNN

An Application of kNN to Diagnose Diabetes

Elena Boiko, Jacqueline Razo (Advisor: Dr. Cohen)

2026-11-01

Introduction

In healthcare, kNN has shown promise in predicting chronic diseases like diabetes (Suriya and Muthu 2023) and hypertension (Khateeb and Usman 2017).

In this project, we focus on how kNN can be applied and optimized to predict diabetes, a critical and growing public health issue.

kNN is a well-known algorithm that’s already been applied in medical research, including disease prediction for conditions like hypertension and diabetes. In this project, we explored how kNN behaves with health data and how we can optimize it to improve predictive accuracy for diabetes.

Why This Matters

- Diabetes affects millions worldwide. Early detection can improve outcomes.

- Machine learning, especially interpretable models like kNN, can support diagnosis.

- Our project explores:

- How different k values, distance metrics, and preprocessing techniques affect kNN’s performance.

- Whether kNN is competitive with other models for this task.

Diabetes affects over 37 million people in the U.S., and many don’t know they have it. Early detection is critical to avoid complications. Machine learning can help — especially models like kNN that are easy to understand and implement. We tested how different settings, such as the number of neighbors, distance metrics, and preprocessing techniques impact performance. We also compared kNN with models like decision trees and random forests to see how it holds up.

Why We Chose kNN for This Project

Well-suited for medical datasets with small to medium size

Easy to interpret — great for health professionals

Flexible with minimal assumptions

Can impute missing data and detect patterns

We chose kNN because it’s simple, interpretable, and works well on structured health data like surveys. Unlike more complex models, kNN doesn’t make strong assumptions — it just compares similar cases. That makes it more transparent, which is valuable in healthcare, where decision-making needs to be explainable.

Our Approach

• Clean and preprocess real-world survey data.
• Train kNN models with various configurations.
• Evaluate performance and compare with tree-based models.

Our project followed a structured approach. We used the CDC’s diabetes health indicators dataset, which includes over 250,000 survey responses. After cleaning and preparing the data, we trained multiple versions of kNN by changing key parameters like the number of neighbors and the distance metric. We then compared kNN 4 (SMOTE + feature selection) with decision trees and random forests to see how it performed in a real-world healthcare prediction task.

Method: kNN Overview

k-Nearest Neighbors (kNN) is a non-parametric, instance-based learning algorithm

It is a lazy learner — no explicit training phase is required

Instead, it classifies new data based on similarity to existing labeled points (Zhang 2016)

Classification Process:

1. Distance Calculation:
Measures similarity using metrics like Euclidean or Manhattan distance

2. Neighbor Selection:
Hyperparameter k defines how many nearby points to consider

3. Majority Voting:
The most frequent class among the k nearest neighbors determines the prediction

So, let’s talk about how kNN actually works kNN is known as a lazy learner - it doesn’t train a model in advance. Instead, it stores all the data and makes predictions based on similarity. When a new data point comes in, kNN finds the closest examples from the training set - based on distance - and predicts the most common label among them. This makes it intuitive and highly adaptable, which is why it’s useful in clinical applications where transparency matters.

The process starts by calculating the distance - we used both Euclidean and Manhattan distances in our tests. Then, the model looks at the closest k data points — and this k is something we tune to get better results. Finally, it uses majority voting: whichever class appears most often among the neighbors becomes the prediction.

Distance Calculation:

kNN identifies the nearest neighbors by calculating distances between points.

Euclidean distance: (Theerthagiri et al. 2022) \[ d = \sqrt{(X_2 - X_1)^2 + (Y_2 - Y_1)^2} \]

Manhattan distance: (Aggarwal et al. 2015) \[ d = |X_2 - X_1| + |Y_2 - Y_1| \]

To find “closeness,” kNN uses distance metrics. We tested two types: The most common is Euclidean distance, which measures straight-line distance, and Manhattan distance, which works like a city grid. Since kNN relies on distance, the choice of metric — along with scaling — can significantly affect results. This diagram shows both — we tested both in our project to compare results.

Classification Process

The red square represents a data point to be classified. The algorithm selects the 5 nearest neighbors within the green circle—3 hearts and 2 circles. Based on the majority vote, the red square is classified as a heart.

Figure 1. kNN with k=5

Here’s a simple example of how kNN makes a prediction. The red square is a new point. It looks at the 5 nearest neighbors — in this case, 3 hearts and 2 circles. Since hearts are the majority, the red square is predicted as a heart. This simple majority voting process makes kNN easy to understand and explain - a big advantage in medical settings.

Strengths and Weaknesses of kNN

Strengths

• Simple, intuitive, and non-parametric — no assumptions about data distribution
  
• No training phase — the algorithm learns during prediction
  
• Performs well on small to medium datasets, especially when features are well-scaled
  
• Easy to understand and implement — ideal for baseline models or educational use

Weaknesses of kNN

• Slow prediction time on large datasets due to distance calculations (Deng et al. 2016)
• Sensitive to feature scaling and distance metric choice (Uddin et al. 2022)
• Choosing the right ‘k’ is critical — too low or high can reduce performance
  
• Affected by irrelevant or correlated features, which may distort neighbor similarity

Here’s a quick summary of what makes kNN useful, and what challenges come with it. It’s simple, doesn’t need training, and works well with smaller datasets when features are properly scaled. That’s why it’s often used for quick testing or in educational settings. But it can struggle with large datasets or noisy features. It’s also sensitive to scaling, and choosing the right k value is really important. That’s why preprocessing and tuning are so important - especially in healthcare, where accuracy and fairness matter.

Analysis and Results

Data Source and Collection

Data Source: CDC Diabetes Health Indicators

Collected via the CDC’s Behavioral Risk Factor Surveillance System (BRFSS)

Dataset contains 253,680 survey responses

Covers 21 features: demographics, lifestyle, healthcare, and health history

Target: Diabetes_binary

(0 = No diabetes, 1 = Diabetes/Prediabetes)

For this project, we used real survey data from the CDC’s BRFSS program. The dataset includes over 250,000 adult responses and covers a wide range of features like age, BMI, physical activity, and general health. Our target was a binary variable indicating diabetes or prediabetes. The dataset’s size and feature variety made it an ideal test case for evaluating how kNN behaves in a real-world health context.

Data Challenges

Data Quality
No missing values
24,206 duplicate rows detected

Outliers & Scaling Sensitivity
BMI, MentHlth, PhysHlth had extreme values
kNN is highly sensitive to scale

Feature Relationships
No strong multicollinearity (max r = 0.52)
All features retained for now

Early Insight
Higher BMI in diabetic cases, but overlapping range
Used as a predictor along with other features

Before modeling, we looked closely at the raw dataset. There were no missing values, but nearly 10% of the data was duplicated - those could bias the model if not removed. BMI and other health features showed outliers, and because kNN uses distance, that’s a problem - large values can dominate. We also checked for correlation but didn’t find any features that were too closely related. One early pattern we noticed: people with diabetes generally had higher BMI, but it wasn’t only factor.

Class Distribution:

Diabetes Class Imbalance

Key Points
Significant class imbalance observed
Majority class: No Diabetes (0) – 86.07%
Minority class: Diabetes (1) – 13.93%

Impact on Modeling
Imbalance can bias predictions
Models may underpredict diabetes cases

Another major challenge was class imbalance. About 86% of cases were non-diabetic, and only 14% were diabetic or prediabetic. This imbalance can cause models to favor the majority class and miss early diabetes cases. We addressed this in preprocessing.

Preparing the Data

• Removed 24,206 duplicate rows
  Diabetic class increased from 13.9% → 15.3%

• Kept ordinal features as numeric
  Age, Education, Income, and GenHlth retained due to natural ordering

• Scaled Features with Outliers
  Features scaled with StandardScaler or RobustScaler (fitted on training data)

• Handled class imbalance
  Applied SMOTE to generate synthetic diabetic samples

➡️ Final dataset: clean and scaled; SMOTE applied to training data only

To get the data ready, we removed duplicates and kept ordinal features like age, education, and income as numeric. Features were scaled with StandardScaler or RobustScaler, fitted on the training data, to ensure equal contribution during distance calculations, a critical aspect for kNN’s accuracy. To fix the class imbalance, we used SMOTE, which creates synthetic diabetic cases to help the model learn both classes equally. After these steps, the data was cleaned and scaled, and SMOTE balanced the training set for kNN 3 and kNN 4.

kNN Model Setup

• Explored different k values: 5, 10, 15
  
• Compared distance metrics: Euclidean vs. Manhattan
  
• Evaluated weighting methods: uniform vs. distance
  
• Tested multiple scaling techniques
  
• Included variations with SMOTE and Feature Selection

Model	k	Distance	Weights	Scaler	SMOTE
kNN 1	5	Euclidean (p=2)	Uniform	StandardScaler	No
kNN 2	15	Manhattan (p=1)	Distance	RobustScaler	No
kNN 3	10	Euclidean (p=2)	Uniform	StandardScaler	Yes
kNN 4	15	Euclidean (p=2)	Distance	StandardScaler	Yes (Feature Selection)

To better understand kNN’s behavior, we designed four model variations. We changed the number of neighbors, distance type, and weighting method. We also experimented with different scaling techniques and applied SMOTE to handle class imbalance. In one version, we also used feature selection to reduce dimensional noise. kNN 4 combined SMOTE with chi-square feature selection.

Performance of kNN Variants

Table 4: Performance Comparison of kNN Models

Model	k	Distance	Weights	Scaler	SMOTE	Accuracy	ROC_AUC	PR_AUC	Precision_1	Recall_1	F1_1
kNN 1	5	Euclidean (p=2)	Uniform	StandardScaler	No	0.832	0.705	0.281	0.406	0.206	0.274
kNN 2	15	Manhattan (p=1)	Distance	RobustScaler	No	0.841	0.752	0.334	0.446	0.155	0.230
kNN 3	10	Euclidean (p=2)	Uniform	StandardScaler	Yes	0.691	0.733	0.286	0.278	0.641	0.388
kNN 4	15	Euclidean (p=2)	Distance	StandardScaler	Yes (FS)	0.731	0.722	0.300	0.303	0.579	0.397

• Best ROC-AUC among kNN: kNN 2 (0.752)
  RobustScaler, Manhattan distance, distance weighting

• SMOTE raised recall from ~0.2 to 0.58–0.64
  at the cost of lower precision

• Feature selection (kNN 4): 12 features instead of 21, mixed results vs. kNN 3

Table 4 shows the results for each kNN model. As you can see, model performance varies significantly depending on configuration. For instance, KNN 1 and 2 without SMOTE had higher accuracy but poor recall - meaning they missed a lot of diabetic cases. kNN 2 had the best ROC-AUC among kNN models. SMOTE in kNN 3 and kNN 4 tripled recall for diabetic cases, but lowered precision. Feature selection in kNN 4 reduced the features from 21 to 12 with mixed results. We use kNN 4 for the comparison with tree models because, among the configurations that detect most diabetic cases, it had the best balance between precision and recall (highest F1, 0.397).

Comparing kNN with Tree Models

Table 5: kNN 4 vs. Tree-Based Models

Model	SMOTE	Accuracy	ROC_AUC	PR_AUC	Precision_1	Recall_1	F1_1
KNN 4	Yes	0.731	0.722	0.300	0.303	0.579	0.397
Decision Tree	Yes	0.671	0.756	0.317	0.281	0.740	0.407
Decision Tree	No	0.849	0.794	0.388	0.517	0.154	0.237
Random Forest	No	0.854	0.813	0.432	0.599	0.134	0.219

- Random Forest had the best ROC-AUC (0.813) and PR-AUC (0.432)

- Decision Tree with SMOTE had the highest recall (0.740) and F1 (0.407)

- Random Forest had highest accuracy, but poor recall (0.13) shows it struggled to detect diabetic cases

Tree-based models offer interpretability, but may need tuning or resampling for minority detection

We also compared kNN 4 to Decision Trees and Random Forests. Random Forest had the highest accuracy and the best ROC-AUC — but at the 0.5 threshold it missed most diabetic cases. The Decision Tree with SMOTE had the highest recall. kNN 4 did not reach the performance of the tree-based models.

ROC_AUC curves comparison

ROC curves of kNN 4 and the tree-based models on the same test set.
Random Forest has the highest AUC (0.81), kNN 4 reaches 0.72.

ROC Curve

This ROC curve shows how well each model separates diabetic from non-diabetic cases. Random Forest had the highest AUC of 0.81, followed by the Decision Tree. kNN 4 reached an AUC of 0.72. After correcting data leakage in the evaluation, kNN did not outperform the tree-based models.

Conclusion

• Preprocessing changes kNN’s behavior: configurations with SMOTE had ~3× higher recall for diabetic cases (~0.6 vs. ~0.2); RobustScaler, Manhattan distance, k = 15 and distance weighting (kNN 2) improved ROC-AUC (0.70 → 0.75).
  

• On this large dataset, tree-based models achieved higher ROC-AUC: Random Forest had the best ROC-AUC (0.813), Decision Tree with SMOTE the highest recall (0.740).
  

• No single best model: the choice of model and metric should follow the goal of the analysis.
  

• Accuracy alone is misleading for imbalanced medical data.
  

• Revision 2026: the original evaluation of kNN 4 (AUC 0.88) contained data leakage; all models were re-evaluated on one untouched test set.
  

In conclusion, preprocessing choices clearly change how kNN performs, and the configurations with SMOTE detected far more diabetic cases. However, after correcting the evaluation, tree-based models - especially Random Forest - performed better than kNN on this dataset. The original result of AUC 0.88 was inflated by data leakage; the corrected results provide a valid estimate of performance.

References

Aggarwal, Charu C et al. 2015. Data Mining: The Textbook. Vol. 1. Springer.

Deng, Zhenyun, Xiaoshu Zhu, Debo Cheng, Ming Zong, and Shichao Zhang. 2016. “Efficient kNN Classification Algorithm for Big Data.” Neurocomputing 195: 143–48.

Khateeb, Nida, and Muhammad Usman. 2017. “Efficient Heart Disease Prediction System Using k-Nearest Neighbor Classification Technique.” Proceedings of the International Conference on Big Data and Internet of Thing, 21–26.

Suriya, S, and J Joanish Muthu. 2023. “Type 2 Diabetes Prediction Using k-Nearest Neighbor Algorithm.” Journal of Trends in Computer Science and Smart Technology 5 (2): 190–205.

Theerthagiri, Prasannavenkatesan, A Usha Ruby, and J Vidya. 2022. “Diagnosis and Classification of the Diabetes Using Machine Learning Algorithms.” SN Computer Science 4 (1): 72.

Uddin, Shahadat, Ibtisham Haque, Haohui Lu, Mohammad Ali Moni, and Ergun Gide. 2022. “Comparative Performance Analysis of k-Nearest Neighbour (KNN) Algorithm and Its Different Variants for Disease Prediction.” Scientific Reports 12 (1): 6256.

Zhang, Zhongheng. 2016. “Introduction to Machine Learning: K-Nearest Neighbors.” Annals of Translational Medicine 4 (11).