DATA COLLECTION AND ANALYSIS
TCUP Fundamentals of Education Research Workshop
Mercy Mugo
October 31, 2014
Quality Education for Minorities (QEM) Network
The Research Linkages
Identify the
Research
Question(s)
Determine
the Research
Methodology
Collect the
Data
Data
Analysis
and Report
of Findings
What is Data Collection?
A detailed plan of procedures that aims to
gather data for the purpose of answering a
research question(s)
Quantitative Data
Collected in standardized manner
Presented in numerical format
Analyzed using statistical techniques
Results more generalizable
Qualitative Data
Collected in natural setting
Presented in narrative format
Thick in detail and description
Analysis often emphasizes understanding
phenomena as they exist
Data Collection Methods
Quantitative Methods
Surveys
Tests
Rubrics
Checklists
Qualitative Methods
Observations
Interviews
Focus Groups
Document Review/Analysis
Case Studies
Photographs, Videos
Cultural Responsive Data Collection
Methods
Talking Circles
Visiting
Performance-based Assessment
Appreciative Inquiry
Pros and Cons of Quantitative Methods
Method
Advantages
Disadvantages
Surveys
Inexpensive
Self-report may lead
Good for gathering
to bias
descriptive data
Data lack depth
Cover a wide range of No control for
topics
misunderstood
A variety of software for questions, missing
analysis
data
Pros and Cons of Quantitative Methods
Method Advantages
Tests
Objective information on
what the test taker
knows and can do
Can be constructed to
match level of skills
Easy to administer/score
Provide hard data
Accepted by public
Disadvantages
Time consuming
Biased against some
groups
May be subject to
corruption via
coaching or cheating
Pros and Cons of Qualitative Methods
Method
Advantages
Interviews Yield richest data,
insights
Permit face-to-face
contact
In-depth exploration of
topics
Allow interview to
explain or clarify
questions
Disadvantage
Expensive and time
consuming
Need well trained
interviewers
Interviewee may distort
information
Large volume of information
may be difficult to transcribe
Pros and Cons of Qualitative Methods
Method Advantages
Focus
Groups
Useful to gather
different viewpoints
and new insight
Less time required
Subject matter is not
sensitive
Disadvantage
Not suitable for
generalization
Require qualified
facilitator
Peer pressure may
inhibit responses
Pros and Cons of Qualitative Methods
Method
Advantages
Observations Provide direct
information
Permit observer to
enter into and
understand context
Exist in natural, flexible
setting
Good for identifying
unanticipated outcomes
Disadvantage
Expensive
Time consuming
Need well qualified
observers
Selective perception of
observer may distort data
Behavior observed may not
be representative of a
group/situation
Pros and Cons of Qualitative Methods
Method
Advantages
Disadvantage
Document
Analysis
Inexpensive
Available locally
Grounded in setting
and language in
which they occur
Provide information
on historical trends
May be incomplete
May be inaccurate
Challenges locating
suitable documents
Time consuming
When Choosing Methods, Consider
Purpose of study (research questions)
Respondents/data sources
Resources available
Type of information needed
Value of using multiple methods
Importance of ensuring cultural appropriateness
Other Considerations..
Understand the community
Involve community
Allow time to establish relationships
Take care in constructing and asking questions
Respect cultural protocols
Provide incentives
DATA ANALYSIS
What is Data Analysis?
Summarizing data into manageable format to
communicate its meaning
Reflecting on the data and searching for
patterns
Seeking out the story in the results
Types of Statistics in Quantitative Research
Descriptive Statistics
used to
organize,
describe, and
summarize a set
of data.
Inferential Statistics
used to draw
inferences about
characteristics of a
population based on
what is known about a
sample drawn from
that population.
Types of Statistical Analyses in Quantitative
Research
Descriptive Statistics
Measures of central
tendency
Mean,
variance,
standard deviation
Parametric Tests
median, mode
Measures of variability
Range,
Inferential Statistics
t-tests, Analysis of variance
(ANOVA), Regression
analysis
Non-parametric Tests
Chi-Square test; the sign
test
Data Analysis in Qualitative Research
Read all data, get a sense of the whole
Code data, tag items with same meaning using
using unique codes
Identify patterns/themes among the codes
Represent themes (writing, visual, etc.)
Interpret and make meaning out of the data
Qualitative Data Analysis
Mixed Methods Designs
Utilizes both quantitative and qualitative data
collection methodologies
Major designs:
Convergent Design
Explanatory design
Exploratory design
Embedded Design
Convergent Design
Collect
qualitative and quantitative data
concurrently
Analyze the two datasets separately
Mix the two databases by merging results
during interpretation
Creswell. J (2012): Borrowed from Abraham S. Fischler, Presentation Mixed Methods
Explanatory Design
Starts
by collecting and analyzing quantitative
data
Uses quantitative results to inform subsequent
qualitative inquiry
Uses quantitative results to shape the qualitative
research questions, sampling, and data collection
Creswell. J (2012): Borrowed from Abraham S. Fischler, Presentation Mixed Methods
Exploratory Design
Starts
by collecting and analyzing qualitative
data
Utilizes qualitative results to build the quantitative
phase
Connects the phases by using qualitative results to
shape the quantitative research questions,
variables, and instrument
Creswell. J (2012): Borrowed from Abraham S. Fischler, Presentation Mixed Methods
Embedded Design
Quantitative (or Qualitative)
Design
Quantitative (or Qualitative)
Data Collection and Analysis
Qualitative (or quantitative)
Data Collection and Analysis
(before, during, or after)
Interpretation
Validity of Results
Validity
Most
important characteristic of the study/assessment
results
Concerned with the appropriateness of the
interpretations made from assessment results
Specific to the interpretation being made and to the
group being assessed
Reliability of Results
Reliability
Concerned
with how well the results can be
replicated
Would a particular technique (or survey) yield
the same results each time?
Reliability does not ensure accuracy
Working with Identifiable Data
Maintaining confidentiality
Presenting accurate information
How can we reconcile these two conflicting
dynamics?
Ways to Address Small Sample Size
Report results in aggregate --- across several
samples to maintain confidentiality
Allow access of raw data to user who may need it
for e.g. evidence-based decision-making, policymaking, budgeting, proposal development, etc.
IRB
approval
Participants permission to release information/data
Example: Impact of Suppression of
Small Data Cells
Negatively affect underrepresented groups
(loss of information)
Value of data significantly diminished
Prevent access to information essential to
providing highly needed opportunities
A report on the series of outreach meetings on the Impact of the Suppression of Small Data
Cells in the Survey of Earned Doctorates (SED) Report (2009).
Prepared by QEM for NSFs Science Resources Statistics (SRS) Division.
Impact of Suppression Continued
Harm diversity-focused initiatives and minorityfocused programs
Difficult finding role models
Difficult designing intervention strategies
A report on the series of outreach meetings on the Impact of the Suppression of Small Data
Cells in the Survey of Earned Doctorates (SED) Report (2009).
Prepared by QEM for NSFs Science Resources Statistics (SRS) Division.
Questions
Thank you!
