Exploratory Data

Analysis:
Uncovering
Insights
Exploratory Data Analysis (EDA) is a crucial step in any data
science project. It involves using statistical and visual methods to
understand the data, uncover patterns and trends, and identify
potential relationships. Through EDA, you gain valuable insights
that inform your further analysis and model building, leading to
more accurate and reliable conclusions.
by Ivy Encarnacion
Descriptive Statistics: Summarizing Data
1 Measures of Central Tendency 2 Measures of Dispersion
Mean, median, and mode provide insights into the Standard deviation, variance, and range quantify
central point of your data. The mean represents the spread or variability of the data. A larger
the average value, while the median indicates the standard deviation indicates greater dispersion,
middle value when data is ordered, and the mode while a smaller one suggests data points cluster
reveals the most frequent value. closer to the mean.
Descriptive Statistics: Summarizing Data
3 Quantiles and Percentiles 4 Frequency Distributions
These measures help understand data distribution. Frequency distributions summarize how often each
Quantiles divide the data into equal parts, while value appears in your dataset. They can be
percentiles express the value below which a presented as tables or histograms, visually
specific percentage of data falls. representing the distribution of data.
Data Visualization: Visualizing Patterns
Histograms Box Plots Scatter Plots

Histograms visually represent Box plots offer a concise Scatter plots visualize the
the frequency distribution of a visualization of data distribution relationship between two
single variable. They provide by showing the median, variables, showing the
insights into the shape, center, quartiles, and potential outliers. correlation or lack thereof. They
and spread of the data. They allow quick comparisons help identify potential trends
across different groups. and patterns within the data.
Histograms

https://helpingwithmath.com/histogram/
Identifying Outliers: Anomalies in Data

Visual Inspection Statistical Methods Domain Knowledge

Outliers often appear as unusual Various statistical techniques, like Z- Considering your specific domain
points on graphs like scatter plots, box scores and interquartile range (IQR) knowledge can help determine if an
plots, or histograms. calculations, can identify potential outlier is truly an anomaly or simply a
outliers based on their deviation from valid data point.
the expected pattern.
Exploring Relationships:
Correlations and
Associations
Correlation Coefficient Interpretation

1 Perfect positive correlation

-1 Perfect negative correlation

0 No correlation
Missing Data: Handling
Incomplete Information
Deletion
Remove rows or columns containing missing values, but this can
lead to data loss and bias.

Imputation
Replace missing values with estimates based on existing data,
using methods like mean imputation or k-nearest neighbors.

Prediction
Use predictive models to estimate missing values based on
existing patterns in the data.
Feature Engineering:
Transforming Variables
1 Scaling and Normalization
Transform variables to a common scale, like 0 to 1, to
improve model performance and avoid bias due to
differences in units.

2 Binning
Group continuous variables into discrete intervals, which can
simplify analysis and improve model interpretability.

3 Feature Creation
Derive new features from existing ones, such as interaction
terms or ratios, to capture more complex relationships in the
data.
Dimensionality Reduction:
Simplifying Complex Data
1 Principal Component Analysis (PCA)
PCA identifies principal components, linear combinations of
original features, that capture most of the data's variance,
allowing you to reduce dimensionality while preserving key
information.

2 t-Distributed Stochastic Neighbor Embedding (t-SNE)

t-SNE is a non-linear dimensionality reduction technique that aims
to preserve the local structure of the data, making it effective for
visualizing high-dimensional datasets.

3 Feature Selection
Selects a subset of the most relevant features, removing
redundant or irrelevant ones to improve model performance and
interpretability.
Data Storytelling:
Communicating Findings
1 Visualizations
Use impactful and relevant visualizations to effectively
communicate patterns, trends, and insights. Choose visualizations
suitable for your audience and the type of data you're presenting.

2 Narrative
Create a clear and engaging narrative that guides your audience
through the key findings of your EDA, highlighting the most
important insights and their implications.

3 Context
Provide context to your findings by relating them to the broader
problem or business objective, making them more relevant and
understandable to your audience.
Iterative Process:
Refining and Enhancing
EDA
Exploratory Data Analysis is an iterative process. As you
uncover insights, you may need to revisit your initial
assumptions, refine your analysis techniques, or acquire
additional data. Embrace this iterative nature, constantly
refining your understanding and improving the quality of
your results.