Psychiatry Research: Neuroimaging 346 (2025) 111903

Contents lists available at ScienceDirect

Psychiatry Research: Neuroimaging

journal homepage: www.elsevier.com/locate/psychresns

Machine learning based classification of excessive smartphone users via

neuronal cue reactivity
Jailan Oweda a,b,* , Mike Michael Schmitgen a , Gudrun M. Henemann a , Marius Gerdes b,
Robert Christian Wolf a
a
Department of General Psychiatry, Heidelberg University Hospital, Germany
b
Karlsruhe Institute of Technology, Germany

A R T I C L E I N F O A B S T R A C T

Keywords: Excessive Smartphone Use (ESU) poses a significant challenge in contemporary society, yet its recognition as a
fMRI distinct disorder remains ambiguous. This study aims to address this gap by leveraging functional magnetic
Cue-reactivity resonance imaging (fMRI) data and machine learning techniques to classify ESU and non-excessive smartphone
SVM
users (n-ESU) based on their neural Cue-Reactivity (CR) signatures. By conducting a CR task and analyzing brain
RFE
activation patterns, we identified spatial similarities between addictive smartphone use and established addictive
PCA
Excessive smartphone use disorders. Our approach involved employing Support Vector Machines (SVM) for classification, enhanced with
feature selection methods such as Recursive Feature Elimination (RFE) and Model-based Selection and dimen­
sionality reduction methods such as and Principal Component Analysis (PCA) to mitigate the challenges posed by
limited dataset size and high dimensionality of fMRI data. The results demonstrate the effectiveness of our
classification model, achieving accuracies of up to 79.9 %. Furthermore, we observed region-specific activations
contributing significantly to classification accuracy, highlighting the potential biomarkers associated with ESU.
External validation on longitudinal data revealed the necessity for larger training datasets to improve model
generalizability. Additionally, feature selection techniques proved crucial for optimizing model performance,
particularly in datasets with combined information from multiple sources. Our findings underscore the impor­
tance of incorporating more data to enhance model stability and generalizability, with implications for
advancing the understanding and treatment of ESU and related disorders. Overall, our study demonstrates the
promise of machine learning approaches in elucidating neural correlates of ESU and informing targeted in­
terventions for affected individuals.

1. Introduction smartphone use and Internet Gaming Disorder (IGD), has shown strong
internal consistency and reliable test-retest performance in previous
Excessive Smartphone Use (ESU), sometimes also referred to as studies (Pavia et al. 2016; Lin et al. 2014). Also, the Smartphone
“smartphone addiction”, remains debatable and is not yet recognized by Addiction Scale-Short Version (SAS-SV) is measured through a self­
the DSM-5 (DSM-5TR). The main reason is that smartphones can be used reported questionnaire and assesses different domains such as daily-life
for various purposes and thus include several addiction types, including disturbances, withdrawal symptoms, tolerance, and virtual social re­
Internet Gaming Disorder (IGD) and related syndromes, e.g. Internet lationships, which are affected by excessive smartphone use (Hamamura
Addiction Disorder (IAD) (Lawrence Robinson and Jeanne Segal 2023). et al. 2023). Additionally, tracking the number of hours spent on
However tools have been created that can estimate ”excessiveness” of smartphones daily can serve as a further indicator of excessive use. In
smartphone usage. For example, the Smartphone Addiction Inventory this context, ”nonexcessive users” refers to average smartphone users,
(SPAI) (Lin et al. 2014) is a 26-item self-report tool designed to assess who may not exhibit problematic behaviors but still engage in regular
problematic smartphone use based on criteria like compulsive behavior, daily smartphone use.
tolerance, withdrawal, and negative impacts on health and social In a recent study (Mike M. Schmitgen et al. 2020), an fMRI experi­
functioning. The SPAI, developed with reference to excessive ment was conducted to compare ESU and non-excessive smartphone

* Corresponding author.
E-mail addresses: jailan.oweda@gmail.com, jailan.oweda@zi-mannheim.de (J. Oweda).

https://doi.org/10.1016/j.pscychresns.2024.111903
Received 23 April 2024; Received in revised form 19 September 2024; Accepted 20 September 2024
Available online 2 October 2024
0925-4927/© 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
J. Oweda et al. Psychiatry Research: Neuroimaging 346 (2025) 111903

users (n-ESU) during a Cue reactivity (CR)-task, which consisted of generating new images. While traditional 2D scans can be manipulated
stimuli that were either pictures of smartphones (turned on or off) or through spatial transformations like rotation and scaling, fMRI data is 4-
neutral pictures (Fig. 1). Significant CR-related activity was observed in dimensional (3D + time), making data augmentation more complex
the medial prefrontal cortex (mPFC), occipital cortex, temporal cortex, (Zhuang et al., 2019). The distinctive characteristics of fMRI, along with
Anterior Cingulate Cortex (ACC), temporoparietal regions, and cere­ the high correlation between neighboring voxels and time steps, pose
bellum when contrasting images of smartphones vs. neutral stimuli challenges for standard data augmentation techniques (Ghassemi et al.,
(Mike M. Schmitgen et al. 2020). Additionally, for the contrast between 2020). These techniques may struggle to capture spatial and temporal
active vs. inactive smartphones, differences were found in the frontal dependencies adequately, potentially resulting in unrealistic or inap­
operculum/anterior insula and precentral gyrus. These results indicate propriate variations in the data. Commonly Generative Adversarial
that there are spatial similarities in cue-reactivity-related brain activa­ Networks can be used for realistic augmentations, however, as this is a
tion between addictive smartphone use and other well-known addictive deep learning method it already requires a high number of samples
disorders (Mike M. Schmitgen et al. 2020). It is unknown so far, whether (Ghassemi et al., 2020; Zhuang, Schwing, and Koyejo 2019).
such activity patterns could be used for classification purposes, i.e. for When faced with smaller datasets, it is conventional to use simpler
approaches that seek to delineate distinct patterns of neural activity that machine learning approaches that can perform well with fewer samples
could reveal group-specific biological signatures. Such signatures could and avoid overfitting. The choices include:
decisively inform biological models of ESU and related conditions, such
as IGD or other technology-related addictive behavior. (a) Support Vector Machines (SVM) work by finding a hyperplane
This paper aims to train machine-learning models that classify the that maximizes the margin, which refers to the maximum width
ESU and n-ESU subjects based on their neural Cue-Reactivity signature of the boundary that separates the different classes, as it tends to
and to test the model on a separate dataset. For this purpose task-based generalize better on unseen data. The support vectors, which are
functional MRI (fMRI) is used, which captures brain activity by data points closest to the decision boundary, control the margin
measuring bloodoxygen-level-dependent (BOLD) signals during specific width. These models are well-suited to high-dimensional data
tasks. because they can handle complex, non-linear relationships using
Preprocessing steps are essential to prepare fMRI data for analysis. kernel functions (Steinwart et al. 2008; Abe 2010).
These typically include Slice Time Correction (Adjusts for differences in (b) Random Forests (RF) is an ensemble learning method that com­
the time it takes to acquire each slice in a volume, ensuring temporal bines multiple decision trees for improved accuracy. This
alignment of the data), Motion Correction and Realignment (Corrects for approach reduces variance and is more robust to overfit- ting and
subject movement, ensuring consistent spatial alignment across images), can handle noisy data providing more reliable predictions and
Segmentation (Separates brain tissues (gray matter, white matter, ce­ making them more suitable for datasets, where the number of
rebrospinal fluid) for more precise analysis), Normalization (Aligns in­ features often exceeds the number of samples. However, it is
dividual brains to a standard template for easier group-level crucial to apply hyper-tuning for example to the number of trees
comparisons), and Smoothing (Applies a spatial filter to reduce noise and their depths for better performance (Kamarajan et al., 2020;
and enhance signal detection). Afis et al., 2024).
After preprocessing, traditional fMRI analysis usually involves two (c) Multilayer Perceptrons (MLP) are a type of neural network with
stages First-Level and Second-Level Analysis. In First-Level Analysis of one or more hidden layers. While they are more complex than
task-based fMRI, models are applied to each participant’s data to create SVMs and Random Forests, they can still be effective with smaller
contrast images, which show differences in brain activity between fMRI datasets, especially when carefully regularized. They learn
different conditions. These contrast images are then used in Second- complex representations by transforming input features through
Level Analysis to make comparisons across a group of participants. nonlinear activation functions. Their success depends on the
However, machine learning can replace Second-Level Analysis by availability of labeled data, model architecture, and training
looking for patterns directly across participants’ data. Instead of aver­ strategies, which is why they need careful hypertuning (Unzueta
aging, algorithms find subtle patterns and make predictions for indi­ 2023; Benoit Liquet and Nazarathy 2023; Afis et al., 2024).
vidual cases, capturing more complex in- formation and providing Another approach to overcome the limited database obstacle,
deeper insights. as well as its high dimensionality, sparsity, and noise is applying
A common issue with fMRI data arises from the limited database size, feature selection or dimensionality reduction. To test this theory
primarily due to the high cost associated with its acquisition. To address the following methods were chosen as an extension to the pre­
this challenge, data augmentation is often employed to expand the viously described machine learning models. As described further
dataset by applying random transformations to existing samples and in Section 2.8 the ML models were trained with and without these
methods to highlight their effects.
(d) Recursive Feature Elimination (RFE) starts by training a model on
the entire set of features and ranking the features based on their
importance scores obtained from the model’s coefficients. Then,
the least important features are removed from the dataset. The
process is repeated iteratively until the desired number of
remaining features (in this case chosen empirically) is reached. It
is particularly effective because it considers the interactions be­
tween features and captures their combined predictive power
(Ranjan and Singh 2023).
(e) ”Select From Model” also relies on feature importances, selecting
the most important features according to a specified threshold
and discarding the rest. This approach is useful when the model’s
coefficients or feature importances provide insights into which
features are most influential for prediction. Both of these methods
can be used in combination with Support Vector Machines (SVM)
Fig. 1. Schematic overview of the cue reactivity task (see supplementals for and Random Forests (RF) as these models return feature
image examples from each group). coefficients.

2
J. Oweda et al. Psychiatry Research: Neuroimaging 346 (2025) 111903

(f) Principal Component Analysis (PCA) is a dimensionality reduc­ of them were female and 12 were male. Their mean age was 22.8 3.14
tion technique used to transform high-dimensional data into a std, the youngest being 18 and the oldest 30 years old. The mean
lower-dimensional space while preserving most of the variance in Smartphone Addiction Inventory (SPAI) -Score of the ESU group was
the data. This model-independent method works by identifying 56.95 10.2 std, with a minimum score of 38 and a maximum of 82. The
the principal components, which are linear combinations of the mean SPAI-Score of the control group was 36 6.9, with a minimum score
original features that capture the maximum variance. These of 27 and a maximum of 55.
principal components are orthogonal to each other, meaning they In the Longitudinal dataset, 36 subjects were kept after data quality
are uncorrelated. By retaining only a subset of the principal inspection (19 ESU, 17 HC). 20 of them were female and 16 were male.
components that explain most of the variance, PCA reduces the Their mean age was 22.7 1.93 std, the youngest being 18 and the oldest
dimensionality of the data while minimizing information loss 26 years old. The mean SPAI-Score of the ESU group was 59.2 13.2 std,
(Xie et al. 2009). with a minimum score of 35 and a maximum of 90. The mean SPAI-Score
of the control group was 40 10.47, with a minimum score of 28 and a
We predicted that ESU will exhibit distinct neural activation patterns maximum of 66.
during the cr-task compared to n-ESU and that these patterns will be The cross-sectional and longitudinal datasets contained different
detectable and classifiable using machine-learning models. We also ex­ subjects; in both datasets, there was an ESU and an HC group. Subse­
pected that feature selection and dimensionality reduction techniques, quently, during machine learning training the major goal was to classify
such as RFE, Model-based Selection, and PCA, will enhance the classi­ these 2 groups using several distinct approaches. We used one dataset
fication accuracy of the machine-learning models by identifying the and then mixed both of them to compare the results. In all cases, we split
most relevant neural features from the fMRI data. the data to separate the test data from the training data strictly.
Here, we trained machine-learning models on regional activations
extracted from fMRI data from a CR-task to predict ESU or n-ESU state in 2.3. Data acquisition
a sample of young adult smartphone users. We tested SVM, MLP, RF. As
SVM demonstrated the best performance, it was chosen for further an­ A 3-T Magnetom TIM Trio MR Scanner manufactured by Siemens in
alyses in this study. MLP and RF results were added to the supplemental Erlangen, equipped with a 32-channel head coil, was utilized in the
material. cross-sectional study to acquire comprehensive whole-brain structural
and functional scans in a dimly lit environment. For the longitudinal
2. Methods study a 3-T Magnetom Prisma Fit Scanner, also manufactured by
Siemens, was used. Participants’ heads were securely fixed in the head
2.1. Participants coil using foam cushions.
The scanning protocol of the cross-sectional study comprised four
Participants in this study were subsets of previous CrossSectional sequential, functional measurements, specifically a resting-state scan,
studies by Horvath et al. (2020) and Schmitgen et al. (2020) and an three experimental paradigms, and a structural scan, as Horvath et al.
ongoing, longitudinal study of the same group. The two datasets consist 2020 and Mike M. Schmitgen et al. 2020 outlined. In the longitudinal
of completely different in- dividuals. Their recruitment was conducted study, the same acquisitions were reacquired after the participants
through flyers, posters at Heidelberg University, the city center, and abstained from their phones for 72 h. The results of the different experi-
social media advertisements. Two user groups, excessive smart- phone ments and their combinations were already reported in several studies
users (ESU) and controls (n-ESU), were defined based on the Smart­ (Mike M. Schmitgen et al. 2020; Horvath et al. 2020; Mike M Schmitgen
phone Addiction Scale-Short Version (SAS-SV) (Hamamura et al. 2023) et al. 2022; Hirjak et al. 2022; Henemann, Mike M Schmitgen, Wolf,
with cutoff values of >31 for males and >33 for females resulting in 20 Hirjak, Kubera, Sambataro, Bach, et al. 2023; Henemann, Mike M
ESU and 22 n-ESU in the cross-sectional dataset and 19 ESU and 17 Schmitgen, Wolf, Hirjak, Kubera, Sambataro, Lemenager, et al. 2023).
n-ESU in the longitudinal dataset.
Before the MRI scans were acquired, participants com- pleted as­ 2.4. CR-task
sessments including the Smartphone Addiction Inventory (SPAI) (Lin
et al. 2014), Beck Depression Inventory (BDI)-II (Gellman and Turner A modified CR task, adapted from Beck et al. 2012, was employed to
2013), and Barratt Impulsive- ness Scale version 11 (BIS-11) (Barratt examine cue-specific brain activation in both datasets. The task involved
1975). These mea- sures showed satisfactory reliability and validity for the presentation of images depicting neutral, non-modern, non-­
assessing smartphone addiction, depression, and impulsivity, respec- media-related stimuli such as furniture, plants, landscapes, and animals
tively (Hamamura et al. 2023; Lin et al. 2014; Gellman and Turner without humans (NEU). Additionally, images included smartphones in a
2013). nonoperating state (OFF) and smartphones in use (ON) displaying main
Participants were required to abstain from smartphone use during or dialing screens, or apps from various categories. The stimuli were
psychometric and MRI assessments. In the longitudinal study, they were standardized in size (1024 × 768 pixels) but were not matched for
also required to abstain from smartphone use for 72 h before running a valence or physical image properties. The order of condition blocks and
second MRI acquisition. Approved by the Ethics Committee of the pictures within the blocks was randomized across subjects. Images from
Medical Faculty at Heidelberg University, the study adhered to the each condition were presented in blocks of five (20 s per block), sepa­
Declaration of Helsinki. All participants provided written informed rated by a 4.8-second presentation of a fixation cross (ITI) without jit­
consent and received monetary compensation for their participation. tering in the interstimulus intervals (see Fig. 1) (Beck et al. 2012).

2.2. Subject demographics 2.5. Data preprocessing

Participants with poor data quality were excluded after visually Imaging data were pre-processed using the Nipype (Nipype: Neuro­
inspecting the fMRI scans for artifacts, such as signal intensity distor­ imaging in Python Pipelines and Interfaces, 2023) module in Python
tions and inconsistencies throughout the entire time series. Additionally, (https://www.python.org/) and more specifically the SPM12 (SPM12:
subjects with head movements exceeding 3 mm or 3◦ were excluded Statistical Parametric Mapping, 2023) interface package. Nipype was
from the analysis. used to create a pipeline that realigned the functional data, coregistered
This left us with 42 subjects in the cross-sectional dataset (20 it with structural images, segmented for normalization to standard MNI
Excessive Smartphone Users (ESU), 22 from the control group (HC)). 30 space, and smoothed with a 9-mm Gaussian kernel at full width at half

3
J. Oweda et al. Psychiatry Research: Neuroimaging 346 (2025) 111903

maximum (FWHM). Subjects with head motion >(3 × 3 × 3) mm or 2.8. Feature selection
3◦ were excluded from further analysis.
The study utilized a general linear model (GLM) to detect blood As the used dataset contains 125 features and only 42 samples in one
oxygen level-dependent (BOLD) activation. Four regressors were dataset and 36 in the other, the model’s performance has been compared
defined: ITI, NEU, and images with phones with screens turned on (ON) to when RFE or Select from Model methods are applied for feature se­
and phones with screens turned off (OFF). These regressors were created lection or when PCA is applied for dimensionality reduction.
by convolving the timing of these stimuli with a standard hemodynamic To reduce the risk of overfitting, the number of data points should be
response function (HRF). Additionally, six realignment parameters were at least 10 times the number of features (Smolic 2024). This is why we
included as nuisance regressors. A high-pass filter (128 Hz) was applied chose to compare the models with 4 (since one dataset includes 40
to eliminate low-frequency signal drift. Con- trast images comparing subjects), 8 (since the 2 datasets combined include 80 subjects), and 16
neutral and phone images (phone (ON+ OFF) >NEU) and also images (to verify whether the increase of features will lead to overfitting)
turned on and off (phone ON >phone OFF) were generated to investi­ selected features.
gate cue-induced brain activation. These maps were then utilized to
classify the subjects. 2.9. Evaluating ML models
Before passing to the classification task, region-specific activations
were extracted from the contrast images by masking them with the In addition to accuracy, which is the number of correct predictions
Neuromorphometrics atlas (Neuromorphometrics,Inc. - Building a over the total number of predictions other scores have also been eval­
Model of the Living Human Brain, 2023) using the Nilearn (Nilearn uated. Sensitivity or Recall (Re) is the proportion of ESU who test pos­
2024) module in Python, resulting in 125 features per subject per True Positives
itive: P = True Positives+False Negatives.Precision (Pr) is the proportion of
contrast (136 before removing masks not including grey matter, such as True Positives
correctly identified ESU P = True Positives+False Positives. The harmonic mean
ventricles and cerebrospinal fluid). In this case, two contrasts are used
of both metrics can be used to balance them P = 2Pr∗Re
Pr+Re , also called the
for each subject: ’Phone (ON+OFF) > NEU’ and ’Phone ON > Phone
OFF’ resulting in 250 features per subject. F1-score. Specificity is the proportion of n-ESU who test negative: P =
True Negatives
True Negatives+False Positives.
Moreover, receiver-operating-characteristic (ROC) analysis and the
2.6. Grid search area under the ROC curve (AUC) were used to evaluate the performance
of the classifiers. AUC represents the classification power of a classifier.
Region-specific activations were used to train a Support Vector Ma­ The values of AUC range from 0 to 1 and larger AUCs indicate better
chine model that classifies ESU and n-ESU subjects (Random Forests and classification abilities.
Multi-layer Perceptrons were also trained but SVM proved to be the most The regional contributions to the classification model were ranked in
efficient, the results can be found in the supplemental material). The descending order, by extracting the feature coefficients from the models.
model was developed in Python (2023) using the following modules:
Scikit-learn - Machine Learning in Python (2023), Keras (2023) and 3. Results
Tensorflow (2024). The model was hyper-tuned using a k-fold (k = 10)
cross-validation method to find the parameters that deliver the best 3.1. Single dataset
accuracy during validation. The k-folds are repeated n times (n = 5)
while shuffling the data to guarantee the sustainability of the obtained In our classification task distinguishing between ESU vs. nESU from a
accuracies. The hyperparam- eter was kernel type which represents the single cross-sectional dataset, the optimal model emerged as utilizing an
hyperplane function and the regularization parameter (C), which is a SVM with a linear kernel and a regularization parameter of C = 0.005
measure of the tolerance for misclassification allowed in the model. (Table 1). This model showcased superior performance when integrated
The model was trained and tested three times using: (i) The cross- with the Recursive Feature Extractor methodology, specifically
sectional dataset (42 subjects (20 ESU, 22 HC (nESU)), (see supple­ employing 16 selected features. This approach yielded a mean accuracy
mentals for further demographics about each class)) for both training of 79.9 % (18.0 % standard-deviation).
and testing (with cross-validation). (ii) The cross-sectional dataset for It was noted that specific regions demonstrated greater significance
training and the longitudinal (36 subjects (19 ESU, 17 HC)) for testing in the classification task, and these regions manifested from different
(in the supplemental material) (iii) A combination of both datasets for contrast images. Notably, in the Phone (ON + OFF) > NEU contrast
training and testing (cross-validation). In all three combinations, we (Fig. 2a), the Right Anterior Cingulate Gyrus (ACgG), Right Angular
ensured that training and test data were strictly separated. Testing was Gyrus (AnG), Right Superior Frontal Gyrus (SFG), and Right Superior
conducted exclusively on subjects not part of the training sample. For Frontal Gyrus Medial Segment (MSFG) were identified as the most sig­
the splitting the stratified method was used to preserve the proportion of nificant regions, as they were chosen in >40 % of the cross-validation
ESU and HC in both training and test date. (CV) iterations. Conversely, in the Phone ON > OFF contrast (Fig. 2a),
the Left Frontal Operculum (FO) region emerged as the most significant
and was chosen in 30 % of the iterations (Table 3a).
2.7. Choosing the ML model
3.2. Two combined datasets
Using grid search, we optimized the hyperparameters for three
different models: SVM, Random Forest, and MLP. Each optimized model Once two datasets (1 cross-sectional and 1 longitudinal) were com­
was then trained on the three train/test combinations described in bined for cross-validated training and testing, the most effective model
Section 2.6 and evaluated as outlined in Section 2.9. emerged employing a regularization parameter C = 0.025 within a
SVM outperformed the other two models, particularly after applying linear SVM framework (Table 2). Utilizing the RFE method and selecting
feature selection, as detailed in Section 2.8. Due to this, we chose to 16 features yielded the highest accuracy of 78.9 % (12.8 %).
focus on the SVM for the results and discussion, since discussing all Here, in the Phone (ON + OFF) > NEU (Fig. 3a) contrast the Right
models would overwhelm the manuscript and appear redundant, espe­ ACgG, Right Cerebellum, Left Posterior Insula (PIns), and Right SFG
cially when some models, such as those with an accuracy of around 50 were highlighted as the most significant with a selection rate higher than
%, indicated random classification. However, the results from RF and 40 %. In the Phone ON > OFF contrast (Fig. 3b) the Right Orbital Part of
MLP are included in the supplemental material for reference. the Inferior Frontal Gyrus (OrIFG) and Right Planum Polare (PP) held

4
J. Oweda et al. Psychiatry Research: Neuroimaging 346 (2025) 111903

Table 1
Cross-validated mean (stand-deviation) scores from support vector machine model (regularization parameter: C = 0.005) trained on the cross-sectional dataset with
different feature selection or dimensionality reduction methods; green color indicates mean accuracies above 75 %.
Feature Selection / Dimensionality # Accuracy Precision Recall F1-Score Specificity AUC
Reduction? Features

None 250 75.8 % (±18.8 74.3 % (±34.0 67.0 % (±34.1 67.6 % (±30.3 83.0 % (±25.3 75.0 % (±19.6
%) %) %) %) %) %)
​ 4 52.8 % (±6.6 %) 4.0 % (±19.6 %) 2.0 % (±9.8 %) 2.7 % (±13.1 %) 100.0 % (±0.0 51.0 % (±4.9 %)
%)
RFE 8 74.5 % (±14.7 81.3 % (±38.4 49.0 % (±29.1 59.6 % (±30.2 98.7 % (±6.5 %) 73.8 % (±18.8
%) %) %) %) %)
​ 16 79.9 % (±18.0 80.3 % (±34.1 68.0 % (±34.3 70.8 % (±31.0 90.3 % (±22.1 79.2 % (±18.8
%) %) %) %) %) %)
​ 4 52.0 % (±4.0 %) 0.0 % (±0.0 %) 0.0 % (±0.0 %) 0.0 % (±0.0 %) 100.0 % (±0.0 50.0 % (±0.0 %)
%)
Model-Based 8 68.6 % (±13.6 66.0 % (±47.4 35.0 % (±26.9 45.3 % (±33.2 100.0 % (±0.0 67.5 % (±13.5
%) %) %) %) %) %)
​ 16 76.8 % (±19.3 80.7 % (±29.1 69.0 % (±29.8 71.4 % (±25.5 84.0 % (±24.9 76.5 % (±19.5
%) %) %) %) %) %)
​ 4 65.3 % (±18.6 57.3 % (±40.8 50.0 % (±37.4 50.4 % (±34.2 79.0 % (±30.7 64.5 % (±19.5
%) %) %) %) %) %)
PCA 8 69.7 % (±18.3 68.7 % (±36.0 59.0 % (±34.2 60.1 % (±29.9 79.0 % (±27.2 69.0 % (±18.6
%) %) %) %) %) %)
​ 16 73.5 % (±18.1 70.7 % (±38.2 60.0 % (±36.1 61.7 % (±32.8 85.3 % (±27.8 72.7 % (±18.7
%) %) %) %) %) %)

4.1. Extracted features

The most significant brain region that was agreed upon in both SVM
models was the Right ACgG, which is highly related to addiction dis­
orders since it influences various cognitive processes such as decision-
making, emotional processing, inhibition, self-regulation, and motiva­
tion (Posner et al. 2007). Studies have shown that stimulating the ACC
may be effective for treating Substance Use Disorders (SUD) (Zhao et al.
2020).
The following region was the Right SFG, which is associated with
goal-directed behavior, impulsivity, reward processing, craving, and the
ability to resist urges all of which may lead to persistent substance or
behavioral use. Previous studies have demonstrated that the right SFG
could serve as a potential biomarker of IGD and provide clues for its
diagnosis and treatment especially due to impulsivity which is linked to
this specific region (P. Zhang et al. 2023).
Fig. 2. Extracted Regions from the Recursive Feature Elimination (RFE) The Right Lingual Gyrus is another brain region that was selected by
method combined with the Support Vector Machine model applied on the cross- both SVM models. this one is mainly responsible for visual processing
sectional dataset only. The color is an indicator of the rate of its choice since
and in this context may be associated with directing attention to the
RFE was applied on 10 folds and repeated 5 times. Purple is a rate up to 100 %
visual stimuli presented in the trials. Since this region was chosen in the
while yellow is closer to 0 % (Table 3a).
contrast of Phone vs. Neutral it may point out to a higher reward
anticipation resulting from seeing phone images in ESU subjects
significance with selection rates over 68 % (Table 3b).
potentially resulting in craving and cue-induced reactivity. Although
this is not a common biomarker in addiction, the lingual Gyrus may be
linked to emotional abnormalities such as depressive disorders (M.
3.3. Features in common
Zhang et al. 2021).
The left frontal operculum is the final brain region that was selected
Remarkably, among the 16 regions selected for each of the two
by both SVM models. It plays a role in selfawareness, verbal expression,
training scenarios, four regions were common: Right ACgG, Right SFG,
and also motor responses which may be linked to addictive behaviors
and Right Lingual Gyrus (LiG) from the Phone (ON + OFF) > NEU
(Darnai et al. 2019).
contrast, along with the Left FO from the Phone ON > OFF contrast.

4. Discussion 4.2. External validation

In this study, we extracted brain activation during a CR task to One of the initial observations upon validating the SVM model
classify ESU vs. n-ESU individuals using an SVM model. Three main trained on the cross-sectional dataset externally on the longitudinal
findings emerged: 1. specific brain regions, such as the Anterior dataset was a numerical decrease in performance, nearly resembling
Cingulate Cortex (ACC), Superior Frontal Gyrus (SFG), Lingual Gyrus random outcomes (see supplementary material). This decline can be
(LiG), and Frontal Operculum, are significant contributors to ESU clas­ attributed to the size of the training dataset, which fails to introduce
sification, 2. Feature selection, especially Recursive Feature Elimination sufficient characteristics to effectively train the model. The high
(RFE), played an important role in enhancing model performance and standard-deviation of the accuracy in the CV scores which reached up to
improving its accuracy, 3. Larger training datasets are needed to 19.3 % (Table 1, Select From Model 16 Features) highlights the obser­
improve model generalizability and performance. vation that the choice of the split plays a crucial role in the model’s

5
J. Oweda et al. Psychiatry Research: Neuroimaging 346 (2025) 111903

Table 2
Cross-validated mean (stand-deviation) scores from Support Vector Machine Model (regularization parameter: C = 0.025) trained on a combination of the cross-
sectional and longitudinal dataset with different feature selection or dimensionality reduction methods; green color indicates mean accuracies above 75 %.
Feature Selection / Dimensionality # Accuracy Precision Recall F1-Score Specificity AUC
Reduction? Features

None 250 64.8 % (±13.2 69.7 % (±22.7 56.5 % (±22.9 59.7 % (±18.6 73.2 % (±19.7 64.8 % (±13.2
%) %) %) %) %) %)
​ 4 74.7 % (±14.1 77.2 % (±16.9 76.3 % (±21.8 74.3 % (±15.3 72.8 % (±22.7 74.6 % (±14.1
%) %) %) %) %) %)
RFE 8 75.5 % (±14.2 78.3 % (±17.8 74.8 % (±21.6 74.3 % (±16.4 75.8 % (±19.8 75.3 % (±14.3
%) %) %) %) %) %)
​ 16 78.9 % (±12.8 82.3 % (±16.7 77.0 % (±21.1 77.5 % (±15.4 80.3 % (±18.8 78.7 % (±12.9
%) %) %) %) %) %)
​ 4 61.1 % (±14.1 65.2 % (±28.8 45.8 % (±23.3 51.3 % (±22.2 76.7 % (±20.0 61.3 % (±14.0
%) %) %) %) %) %)
Model-Based 8 65.4 % (±16.2 68.2 % (±26.2 58.7 % (±25.2 60.7 % (±22.7 72.7 % (±23.0 65.7 % (±16.2
%) %) %) %) %) %)
​ 16 73.4 % (±13.9 78.9 % (±21.1 67.7 % (±23.4 70.0 % (±18.9 79.3 % (±21.4 73.5 % (±13.8
%) %) %) %) %) %)
​ 4 57.5 % (±15.7 59.5 % (±30.0 46.8 % (±27.8 48.9 % (±23.8 67.5 % (±25.1 57.2 % (±15.7
%) %) %) %) %) %)
PCA 8 59.5 % (±14.6 62.9 % (±24.4 57.0 % (±25.9 56.3 % (±19.5 61.7 % (±25.7 59.3 % (±14.7
%) %) %) %) %) %)
​ 16 56.3 % (±16.4 60.0 % (±23.9 55.0 % (±25.5 53.7 % (±19.3 57.2 % (±28.8 56.1 % (±16.5
%) %) %) %) %) %)

from the noteworthy improvement of >10 % when 8 features are

selected with the RFE method in contrast to when only 4 features are
sorted out.
When the model was employed for CV with a fusion of both datasets
(Table 2), the necessity for feature selection became more apparent due
to a notable drop in accuracy to 64.8 %. This decrease in accuracy points
to the difference between activations in both datasets. In this context,
RFE emerged as particularly effective, introducing a substantial per­
formance improvement. It resulted in a 10 % enhancement when 4
features were selected and 14 % when 16 features were chosen. Also
here, it was noted that whether the RFE or Select From Model method
was employed for feature selection, the model performance was better
with more features. While the mean accuracy was increasing the vari­
ance decreased slightly, too.

4.4. Augmentation through dataset combining

Fig. 3. Extracted Regions from the Recursive Feature Elimination (RFE)
method combined with the Support Vector Machine model applied on a com­
bination of the cross-sectional and longitudinal datasets. The color is an indi­ Despite the significant decline in mean accuracy by 11 % when
cator of the rate of its choice since RFE was applied on 10 folds and repeated 5 SVM’s best model is cross-validated with a combination of two datasets
times. Purple is a rate up to 100 % while yellow is closer to 0 % (Table 3b). compared to only one dataset without feature selection and the marginal
decline by 1 % using the RFE method to select 16 features, an important
performance and that more data is needed for better generalizability. benefit emerged: a noteworthy reduction of variance by 5.6 % in the first
Furthermore, the differences between the experiments may also case and 5 % in the second. This reduction suggests a more stable and
contribute to the region-specific contrast activations. Specifically, while reliable predictive performance, promising a more robust and accurate
participants in the first experiment engaged in one-time tasks, those in classification score overall.
the second experiment were aware in advance of the requirement to It is crucial to consider that utilizing 16 features with a dataset
abstain from smartphone usage for 72 h before performing a subsequent containing 78 samples enhances generalizability compared to applying
round of tasks. This anticipation may have induced excitement or the same number of features to a dataset with only 48 samples. This
anxiousness, potentially influencing brain functionality during task larger sample size allows for a more varied representation of the data
execution, resulting in different outcomes, than observed in the first distribution, potentially resulting in a more robust and applicable model
experiment. when deployed in real-world scenarios.

4.3. Effect of feature selection 4.5. PCA & regularization parameter

For the selected SVMs, it was observed, that when a single dataset It was observed, that when two datasets were combined the perfor­
was utilized for CV (Table 1), feature selection proved unnecessary, mance when using PCA for dimensionality reduction was significantly
given the already commendable accuracy of 75.8 %. When feature se­ lower than without applying this method (by 5.3 % to 8.5 % depending
lection was implemented, the model opting for more features numeri­ on the chosen number of components). This may point out that the
cally consistently exhibited superior performance across all selection retained principal components do not adequately represent the class
methods. This points out that not only a higher number of regions are separation.
significant for the classification, but also that the specific combination of The choice of parameter C for SVM models influences the balance
the regions plays a role in the task. This can be especially concluded between training error and margin. A smaller value of C allows for a

6
J. Oweda et al. Psychiatry Research: Neuroimaging 346 (2025) 111903

Table 3 ensuring consistency in experimental conditions is crucial for more

16 Extracted Regions from the Recursive Feature Elimination (RFE) method reliable outcomes.
combined with the Support Vector Machine model. The 2nd and 3rd columns While our approach provides valuable insights, it’s important to note
indicate from which contrast image the region has been selected and the rate of that not all available algorithms were tested, and there may be others
its choice since RFE was applied on 10 folds and repeated 5 times. The green that could potentially perform better. A comprehensive review of every
color highlights the regions that were in common between case (a) application
commonly used method, however, would have been beyond the scope of
on a single dataset and (b) application on two combined datasets.
this report.
(a) Cross-sectional Dataset only.

Region Phone > Phone ON > Phone 5. Conclusions

Neutral OFF

Right ACgG anterior cingulate gyrus 0.96 ​ In conclusion, our data show the ability of simple ML models to
Right AnG angular gyrus 0.9 ​ classify CR-related fMRI based on extracted regional activations. The
Right SFG superior frontal gyrus 0.86
effect of applying feature selection methods such as RFE has been
​
Right MSFG superior frontal gyrus medial 0.4 ​
segment highlighted. We show that introducing more data to the model may
Left FO frontal operculum ​ 0.3 introduce a significant improvement to the performance, stability, and
Right Calc calcarine cortex 0.28 ​ generalizability of the model. This approach may even be used to find
Right OCP occipital pole 0.26 ​
the regions that contribute most to the class separation thus allowing to
Right MFG middle frontal gyrus 0.26 ​
Right MTG middle temporal gyrus 0.2 ​ delineate robust conclusions on neural mechanisms that underlie ESU
Right AOrG anterior orbital gyrus 0.14 ​ and related addictive behaviors.
Left CO central operculum ​ 0.1
Left PT planum temporale 0.1
CRediT authorship contribution statement
​
Right LiG lingual gyrus 0.06 ​
Right IOG inferior occipital gyrus 0.06 ​
Right PCgG posterior cingulate gyrus 0.06 ​ Jailan Oweda: Writing – original draft, Validation, Software,
Right CO central operculum ​ 0.02 Methodology, Formal analysis, Conceptualization. Mike Michael
(b) Cross-sectional and Longitudinal Datasets combined. Schmitgen: Writing – review & editing, Supervision, Methodology,
Region Phone > Phone ON > Phone
Data curation, Conceptualization. Gudrun M. Henemann: Data cura­
Neutral OFF tion, Conceptualization. Marius Gerdes: Supervision, Methodology.
Robert Christian Wolf: Writing – review & editing, Supervision,
Right OrIFG orbital part of the inferior ​ 0.72
frontal gyrus Conceptualization.
Right PP planum polare ​ 0.68
Right ACgG anterior cingulate gyrus 0.68 ​
Right Cerebellum Exterior 0.64 ​ Declaration of competing interest
Left PIns posterior insula 0.42 ​
Right SFG superior frontal gyrus 0.4 ​
Left LOrG lateral orbital gyrus ​ 0.18 The authors declare that they have no known competing financial
Right PoG postcentral gyrus ​ 0.16 interests or personal relationships that could have appeared to influence
Left TMP temporal pole 0.14 ​ the work reported in this paper.
Left Putamen 0.14 ​
Left FO frontal operculum ​ 0.12
Right LOrG lateral orbital gyrus ​ 0.1 Acknowledgements
Right LiG lingual gyrus 0.08 ​
Left Basal Forebrain 0.08 ​ We thank J. Rosero, S. Haage, and M. Imamovic for their assistance
Left AIns anterior insula 0.08 ​
with data collection. We’d also like to express our gratitude to our study
Left PrG precentral gyrus ​ 0.06
participants for their time and interest in this study.

larger margin. It prioritizes a wider separation between classes, even if it Supplementary materials
means more misclassifications on the training data, and is useful when
data points are well-separated and noise/outliers are minimal. On the Supplementary material associated with this article can be found, in
other hand, a larger value of C emphasizes minimizing the training error. the online version, at doi:10.1016/j.pscychresns.2024.111903.
It seeks to fit the training data as accurately as possible, even if it sac­
rifices margin width, and is beneficial when data points are not well- References
separated or noise/outliers are significant.
Since the grid search has found a larger value (C = 0.025) to be more Abe, Shigeo, 2010. Support Vector Machines For Pattern Classification, 2nd ed. Springer
performant when two datasets are combined vs. when only one is used London, London. Advances in Pattern RecognitionISBN: 9781849960984. URL:
https://ebookcentral.proquest.com/lib/kxp/detail.action?docID=602997.
(C = 0.005) it indicates, that combining the two datasets has resulted in
Afis, Ajala, et al., (2024). Autoencoder-based feature extraction and classification for
a more difficult separation of the data and when the observation of the fMRI-based deep brain stimulation parameter optimization for parkinson
decreased performance with PCA is added to that, it means that the textquoterights disease treatment: towards a rapid semi-automated stimulation
optimization. In: medRxiv. DOI: 10.1101/2024.01.11.24301179. URL: https://www.
combination has introduced more variability to the dataset avoiding
medrxiv.org/content/early/2024/01/13/2024.01.11.24301179.
underfitting and making the obtained model more robust and Barratt, Ernest S, 1975. Barratt Impulsiveness Scale. ETS m, 1975.
generalizable. Ed. by Beck Depression Inventory (BDI), 2013. In: Gellman, Marc D., Turner, J.Rick
(Eds.), Encyclopedia of Behavioral Medicine. Springer, New York, pp. 178–179.
https://doi.org/10.1007/978-1-4419-1005-9_441. Ed. bySpringer referenceISBN:
978-1-4419-1004-2.
4.6. Limitations
Beck, Anne, et al., 2012. Effect of Brain Structure, Brain Function, and Brain Connectivity
on Relapse in Alcohol-Dependent Patients. Arch. Gen. Psychiatry 69 (8), 842–852.
Despite the insights gained, there are several limitations to consider. https://doi.org/10.1001/archgenpsychiatry.2011.2026. ISSN: 0003-990X.
The relatively small sample size (48 to 78 participants) constrains the Benoit Liquet, Sarat Moka and Yoni Nazarathy (2023). The Mathematical Engineering of
Deep Learning (2021) - Chapter 4: general Fully Connected Neural Networks. URL: http
generalizability of the findings. Larger and more diverse samples would s://deeplearningmath.org/general-fully-connected-neural-networks.html (visited on
enhance the robustness and applicability of the results. Additionally, 10/28/2023).

7
J. Oweda et al. Psychiatry Research: Neuroimaging 346 (2025) 111903

Diagnostic and statistical manual of mental disorders, fifth edition text revision: DSM-5-TR https://doi.org/10.1016/j.chb.2016.05.039. ISSN: 0747-5632URL: https://www.sci
(2022). Fifth edition, text revision. Washington, DC: american Psychiatric encedirect.com/science/article/pii/S0747563216303661.
Association Publishing. ISBN: 9780890425763. Posner, M.I., Rothbart, M.K., Sheese, B.E., et al., 2007. The anterior cingulate gyrus and
Darnai, Gergely, et al., 2019. Internet addiction and functional brain networks: task- the mechanism of self-regulation. Cogn. Affect. Behav. Neurosci. 7, 391–395.
related fMRI study. Sci. Rep. 9 (1), 15777. https://doi.org/10.3758/CABN.7.4.391Posner.
Ghassemi, Navid, Shoeibi, Afshin, Rouhani, Modjtaba, 2020. Deep neural network with Python (2023). URL: https://www.python.org/ (visited on 11/02/2023).
generative adversarial networks pre-training for brain tumor classification based on Ranjan, Ashish, Singh, Vibhav Prakash, 2023. Affirma- tive and negative sentence
MR images. Biomed Signal Process Control 57, 101678. https://doi.org/10.1016/j. detection in the brain using SVM-RFE and rotation forest: an fMRI STUDY. SN
bspc.2019.101678. ISSN: 17468094. Comput. Sci. 4 (4). https://doi.org/10.1007/s42979-023-01786-1.
Hamamura, Toshitaka, et al., 2023. Validity, reliability, and correlates of the Smartphone SPM12: Statistical Parametric Mapping (2023). URL: https://www.fil.ion.ucl.ac.uk/spm/
addiction scale–short ver- sion among Japanese adults. BMC Psychol. 11 (1), 78. software/spm12/(visited on 11/04/2023).
https://doi.org/10.1186/s40359-023-01095-5. ISSN: 2050-7283. Schmitgen, Mike M., et al., 2020. Neural correlates of cue reactivity in individuals with
Henemann, Gudrun M, Schmitgen, Mike M, Wolf, Nadine D, Hirjak, Dusan, smartphone addiction. Addict. Behav. 108, 106422. https://doi.org/10.1016/j.
Kubera, Katharina M, Sambataro, Fabio, Bach, Patrick, et al., 2023a. Cognitive addbeh.2020.106422.
domain-independent aberrant frontoparietal network strength in individuals with Schmitgen, Mike M, et al., 2022. Aberrant intrinsic neural network strength in
excessive smartphone use. Psychiatry Res. Neuroimaging 329, 111593. individuals with “smartphone addiction”: an MRI data fusion study. Brain Behav. 12
Henemann, Gudrun M, Schmitgen, Mike M, Wolf, Nadine D, Hirjak, Dusan, (9), e2739.
Kubera, Katharina M, Sambataro, Fabio, Lemenager, Tagrid, et al., 2023b. scikit-learn - Machine Learning in Python (2023). URL: https://scikit-learn.org/stable/i
Neurochemical corre- lates of cue reactivity in individuals with excessive smart- ndex.html (visited on 11/05/2023).
phone use. Eur. Addict. Res. 29 (1), 71–75. Smolic, Hrvoje (2024). How Much Data Is Needed For Machine Learning? URL: https://tow
Hirjak, Dusan, et al., 2022. Cortical surface variation in individuals with excessive ardsai.net/p/l/how-much-data-is-needed-for-machine-learning (visited on 08/24/
smartphone use. Develop- Mental Neurobiol. 82 (4), 277–287. 2024).
Horvath, Juliane, et al., 2020. Structural and functional cor- relates of smartphone Steinwart, Ingo, et al., 2008. Support Vector Machines. Information Science and Statistics.
addiction. Addict. Behav. 105, 106334. https://doi.org/10.1016/j. Springer Science+Business Media, LLC, New York, NY. https://doi.org/10.1007/
addbeh.2020.106334. URL: https://www.sciencedirect.com/science/article/pii/S0 978-0-387-77242-4. ISBN: 978-0-387-77241-7.
306460319313802. Tensorflow (2024). URL: https://www.tensorflow.org/(visited on 12/02/2024).
Kamarajan, C., Ardekani, B.A., Pandey, A.K., Kinreich, S., Pandey, G., Chorlian, D.B., Unzueta, Diego (2023). Fully Connected Layer vs. Convolutional Layer: explained. URL:
Meyers, J.L., Zhang, J., Bermudez, E., Stimus, A.T., et al., 2020. Random Forest https://builtin.com/machine-learning/fully-connected-layer (visited on 10/28/
Classification of Alcohol Use Disorder Using fMRI Functional Connectivity, 2023).
Neuropsychological Functioning, and Impulsivity Measures. Brain Sci. 10, 115. Xie, Song-yun et al. (2009). “Brain fMRI processing and classification based on
https://doi.org/10.3390/brainsci10020115. combination of PCA and SVM”. In: pp. 3384–3389. DOI: 10.1109/IJCNN.2009.51
Keras (2023). URL: https://keras.io/(visited on 11/05/2023). 79085.
Lawrence Robinson Melinda Smith, M.A. and Ph.D. Jeanne Segal (2023). HelpGuide.org - Zhang, Mengzhe, et al., 2021. Shared gray matter alterations in subtypes of addiction: a
Smartphone and Internet Addiction. URL: https://www.helpguide.org/articles/addicti voxel-wise meta-analysis. Psychopharmacology (Berl.) 238 (9), 2365–2379. https://
ons/smartphone-addiction.htm (visited on 11/07/2023). doi.org/10.1007/s00213-021-05920-w.
Lin, Yu-Hsuan, et al., 2014. Development and validation of the Smartphone Addiction Zhang, Pengyu, et al., 2023. Impulsivity-related right superior frontal gyrus as a
Inventory (SPAI). PLoS ONE 9 (6), e98312. https://doi.org/10.1371/journal. biomarker of internet gaming disorder. Gen. Psychiatry 36 (4), e100985. https://doi.
pone.0098312. org/10.1136/gpsych-2022-100985. ISSN: 2517-729X.
Neuromorphometrics,Inc. - Building a Model of the Living Human Brain (2023). URL: htt Zhao, Yijie, et al., 2020. Anterior cingulate cortex in addiction: new insights for
p://www.neuromorphometrics.com/(visited on 10/27/2023). neuromodulation. Neuromodulation: J. Int. Neuromodul. Soc. https://doi.org/
Nilearn (2024). URL: https://nilearn.github.io/stable/index.html (visited on 02/28/ 10.1111/ner.13291.
2024). Zhuang, P., Schwing, A.G., Koyejo, O., "FMRI Data Augmentation Via Synthesis," 2019
Nipype: Neuroimaging in Python Pipelines and Interfaces (2023). URL: https://nipype.re IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice,
adthedocs.io/en/latest/(visited on 11/04/2023). Italy, 2019, pp. 1783-1787, DOI: 10.1109/ISBI.2019.8759585.
Pavia, Laura, et al., 2016. Smartphone Addiction Inventory (SPAI): psychometric
properties and confirmatory factor analysis. Comput Human Behav. 63, 170–178.