Survey Data in Macroeconomics
II. Survey Datasets and Methodological Issues
Prof. Dr. Lena Dräger
Johannes Gutenberg-University Mainz, GSEFM field course
Email: ldraeger@uni-mainz.de
1 / 47
Outline
1 Survey Datasets
2 Qualitative vs. Quantitative Questions
3 Wording Issues
4 Truncation
2 / 47
Survey Datasets
Relevant Survey Datasets
3 / 47
Survey Datasets
Relevant Surveys
Professional forecasters: SPF US (Philadelphia Fed), SPF Eurozone (ECB),
Consensus Economics & Blue Chip
Households: University of Michigan Survey of Consumers (US), Survey of
Consumer Expectations (US), Joint Harmonised EU Programme of Business
and Consumer Surveys (most EU countries), British Household Panel, DNB
Household Survey (Netherlands), Household Finance and Consumption
Network (ECB)
Firms: ifo Business Survey (Germany), survey of New Zealand Firms
conducted by Coibion, Gorodnichenko & Kumar, Duke University Survey of
Chief Financial Officers (CFO), Industrial Trends Survey by the
Confederation of British Industry (UK), Survey on Inflation and Growth
Expectations (Banca d’Italia)
4 / 47
Survey Datasets
Survey of Professional Forecasters (SPF) in the US
https://www.philadelphiafed.org/research-and-data/
real-time-center/survey-of-professional-forecasters/
Forecasts by private sector economists in businesses, the financial
sector and university research centres
Conducted by the Philadelphia Fed
Quarterly data since 1968q4, data collected by the Philadelphia Fed
since 1990
Publicly available: Mean, median, cross-sectional dispersion and
microdata
5 / 47
Survey Datasets
Survey of Professional Forecasters (SPF) in the US
Sample size around 30, with a panel dimension (anonymous)
Includes only quantitative macroeconomic expectations (forecasts)
Probability distribution forecasts for annual inflation and GDP growth
and the probability of a recession
Forecasters give quarterly projections for 5 quarters and annual
projections for the current year (nowcast) and the next year ⇒ fixed
horizon and fixed event forecasts!
6 / 47
Survey Datasets
Survey of Professional Forecasters (SPF) in the US
Forecasted variables, US Business Indicators:
GDP (growth)
GDP chain-weighted price index
Corporate profits after tax
Civilian unemployment rate
Industrial production index (growth)
Housing starts
Three-month Treasury Bill rates (since 1981q3)
Moody’s AAA Corporate Bond Yield (from 1981q3-2016q4)
10-year Treasury Bond rate (since 1992q1)
7 / 47
Survey Datasets
Survey of Professional Forecasters (SPF) in the US
Forecasted variables, GDP and its components (all since 1981q3):
Real GDP
Real personal consumption expenditures
Real Nonresidential fixed investment
Real residential fixed investment
Real Federal government consumption expenditures & gross investment
Real state and local government consumption expenditures & gross
investment
Real change in private inventories
Real net exports of goods and services
8 / 47
Survey Datasets
Survey of Professional Forecasters (SPF) in the US
Forecasted variables, CPI and PCE inflation (all growth rates):
CPI inflation (since 1981q1)
Core CPI inflation (since 2007q1)
PCE inflation (since 2007q1)
Core PCE inflation (since 2007q1)
Long-term (10-year) forecasts (except CPI all since 1992q1 and
collected only in the first quarter each year):
CPI (since 1991q1)
Real GDP
Productivity growth
Stock return (S&P 500)
10-year Treasury Bond return
3-months Treasury Bill return
9 / 47
Survey Datasets
Survey of Professional Forecasters (SPF) at the ECB
https://www.ecb.europa.eu/stats/ecb_surveys/survey_of_
professional_forecasters/html/index.en.html
Professional forecasters at financial or non-financial institutions in the
European Union
Sample size around 90, with panel dimension (anonymous)
Quarterly, conducted by the ECB since 1999q1
Aggregate and microdata publicly available
Includes quantitative forecasts and quantitative probability distribution
forecasts and point forecasts for underlying assumptions
10 / 47
Survey Datasets
Survey of Professional Forecasters (SPF) at the ECB
All forecasts are for the Eurozone
Forecasted variables:
HICP inflation
Real GDP growth
Unemployment rate
Assumptions on the ECB’s main refinancing rate, oil prices, USD/EUR
exchange rate and labour cost changes
Forecast horizons (mostly fixed event):
Current calendar year
Next calendar year, and calendar year after that
4-5 calendar years ahead
“Rolling horizon” one-year ahead from latest available data publication
11 / 47
Survey Datasets
Consensus Economics
http://www.consensuseconomics.com/
Survey conducted by a private firm, collecting quantitative economic
forecasts from about 700 economists each month
Not publicly available
Forecasts by the same forecaster for several countries
Forecasting firms are identified
Only fixed event forecasts
12 / 47
Survey Datasets
Consensus Economics
Range of macroeconomic and financial forecasts (GDP growth,
unemployment, inflation, interest rates, exchange rates etc.)
Countries and Regions:
G-7 and Western Europe
Eastern Europe
Latin America
USA
Special surveys:
Foreign exchange markets forecasts
Commodity forecasts in energy and metal markets
13 / 47
Survey Datasets
Blue Chip Surveys
Blue Chip Economic Indicators and Blue Chip Financial Forecasts
Financial market and economic forecasts, survey conducted by a
private firm
Forecasts of yields along the yield curve
Forecasting firms are identified
Focus on US economy
Not publicly available
14 / 47
Survey Datasets
University of Michigan Survey of Consumers
https://data.sca.isr.umich.edu/
Large household survey in the US, microdata available since 1978m1
Sample representative of the US population, cross-section of about
500 households
Rotating panel dimension: Every month, about 40% of respondents
are (randomly) chosen to be re-interviewed after 6 months
Aggregate and microdata publicly available
15 / 47
Survey Datasets
University of Michigan Survey of Consumers
Includes quantitative macro expectations (point forecasts) on one-year
ahead and 5-10-years ahead inflation, gas price inflation, house price
inflation
Qualitative macro expectations and attitudes on changes in business
and buying conditions, unemployment, interest rates, inflation
Qualitative and quantitative micro expectations on changes in
personal and family income, financial household situation
Percent chances that certain events occur
Large range of socio-demographic characteristics
Further interesting questions: Index of consumer confidence, news on
business conditions observed, assessment of government’s economic
policy
16 / 47
Survey Datasets
Survey of Consumer Expectations
https://www.newyorkfed.org/microeconomics/sce
Conducted by the New York Fed since 2013m6
Sample representative of the US population, cross-section of about
1,300 households
Rotating panel dimension: Households stay in the panel for up to 12
months
Aggregate and microdata publicly available (with a lag)
17 / 47
Survey Datasets
Survey of Consumer Expectations
Large range of subjective macro and micro expectations:
Inflation, incl. home prices and prices of gas, food, rent, medical care,
education
Unemployment
Interest rates
Stock market and credit availability
Taxes and government debt
Own employment situation
Own income and spending
Includes both point forecasts and probability distribution forecasts and
percent chances of binary outcomes
Forecasts for various ranges (1 year and 3 years)
Additional modules on credit access, labour market and spending
(each quarterly), one special module and yearly modules on the
housing market, consumption and saving
18 / 47
Survey Datasets
Joint Harmonised EU Programme of Business and Consumer
Surveys
https://ec.europa.eu/info/business-economy-euro/
indicators-statistics/economic-databases/
business-and-consumer-surveys_en
Monthly since 1985m1 (not all countries and not all questions)
Representative household samples
Harmonized questionnaire in all EU countries
Aggregate and sub-group data publicly available, microdata depends
19 / 47
Survey Datasets
Joint Harmonised EU Programme of Business and Consumer
Surveys
Macro and micro perceptions (past 12 months) and expectations (next
12 months):
Changes in consumer prices
Major purchases of durable consumer goods
Change in savings
General economic situation
Own financial situation
Qualitative expectations of:
Unemployment
Purchase car or house
Home improvements
Consumer confidence indicator
Only qualitative expectations! Quantitative data fielded since 2003,
but not published. Exception: Swedish Consumer Tendency Survey20 / 47
Survey Datasets
DNB Household Survey (Netherlands)
https://www.centerdata.nl/en/projects-by-centerdata/
dnb-household-survey-dhs
Panel of 2000 Dutch households, yearly data from 1993-2015
Topics fielded include:
Economic aspects of financial behavior including subjective reasons for
investment decisions
Inflation expectations
Employment
Housing and mortgages
Health
Pensions
Income, assets and loans
Psychological characteristics
Personal characteristics
Data publicly available after registration
21 / 47
Survey Datasets
Household Finance and Consumption Network (ECB)
https://www.ecb.europa.eu/pub/economic-research/
research-networks/html/researcher_hfcn.en.html
Collection of household data on finances and consumption, with
detailed information on assets and liabilities
Large panels in each Euro area country
Waves approx. every 2 years, so far 2 waves in 2010-2011 and
2013-2015
Questionnaires are partly harmonized across countries
22 / 47
Survey Datasets
ifo Business Survey
Survey of German firms in trade and services
Monthly since 1991m1 (longest horizon)
Also interesting: ifo Investment Survey, bi-annually since 2007
Confidential data, only some time series publicly available
23 / 47
Survey Datasets
ifo Business Survey
Only partly publibly available, including information on questionnaires
Aggregate time series of qualitative indices available:
Business climate index Germany
Business climate German service industry
Business climate Eastern Germany
Credit constraint indicator
Employment barometer Germany
Export expectations
24 / 47
Survey Datasets
Survey on Inflation and Growth Expectations (Banca
d’Italia)
https:
//www.bancaditalia.it/pubblicazioni/indagine-inflazione/
index.html?com.dotmarketing.htmlpage.language=1
Quarterly since 1999
Panel of 1,000 firms with 50 or more workers in Italy
Firms from industry, non-financial private services and construction
Includes current perceptions and expectations of inflation 6, 12, and
24 months ahead
Data is publicly available
25 / 47
Survey Datasets
Survey on Inflation and Growth Expectations (Banca
d’Italia)
Qualitative and probabilistic expectations on:
General economic situation in Italy and in the firm’s sector
Own investment
Own employment
Own product prices
Own demand
Quantitative inflation expectations with/without anchoring (latest
data given)
26 / 47
Qualitative vs. Quantitative Questions
Qualitative vs. Quantitative Questions
27 / 47
Qualitative vs. Quantitative Questions
Survey Design
Question difficulty
Firms/households vs. professional forecasters
Media coverage
Central bank communication
Answer accuracy
Literacy and numeracy issues
Knowledge about the macroeconomy varies greatly
Non-response bias for inflation expectations greater for some
socio-economic groups (Blanchflower and Kelly, 2008)
Difficulties with working with percentages
28 / 47
Qualitative vs. Quantitative Questions
Illiteracy Measured by Non-Response Rates
Price Wording Inflation Wording
Perceptions Expectations Perceptions Expectations
All 0.06 0.08 0.12 0.09
Male 0.02 0.06 0.06 0.05
Female 0.10 0.10 0.19 0.13
16-29 0.03 0.07 0.16 0.12
30-44 0.04 0.07 0.11 0.07
45-59 0.05 0.04 0.10 0.11
60+ 0.09 0.11 0.12 0.07
Low Education 0.09 0.13 0.25 0.12
Medium Education 0.07 0.09 0.14 0.12
High Education 0.04 0.04 0.04 0.03
0-25% Income 0.00 0.03 0.12 0.11
25-50% Income 0.10 0.08 0.12 0.07
50-75% Income 0.16 0.19 0.16 0.09
75-100% Income 0.00 0.03 0.04 0.04
Source: Dräger and Fritsche (2013).
29 / 47
Qualitative vs. Quantitative Questions
Qualitative vs. Quantitative Inflation Expectations
Figure: Qualitative and Quantitative Inflation Expectations in Sweden
20
4
3
0
−20
2
−40
1
−60
0
1996 1998 2000 2002 2004 2006 2008 2010
quantitative inflation expectations, trunc
qualitative inflation expectations, balance stat
Source: Swedish Consumer Tendency Survey.
30 / 47
Qualitative vs. Quantitative Questions
Aggregating Qualitative Questions – Balance Statistic
Qualitative survey variables are often summarized in the so-called balance
statistic:
b = s1 + 0.5s2 − 0.5s4 − s5 (1)
where s1 , s2 , s3 , s4 and s5 are the shares of answers in the categories prices
will “rise a lot”, “rise moderately”, “rise slightly”, “stay about the same”,
“fall”.
Balance statistic with three answer categories:
b = s1 − s3 (2)
31 / 47
Qualitative vs. Quantitative Questions
The Carlson-Parkin Probability Method
Method for quantifying aggregate qualitative survey expectations
For 3 answer categories by Carlson and Parkin (1975), adjusted for 5
categories by Batchelor and Orr (1988)
Quantification makes strong assumptions about distribution and
baseline information
Not more accurate than balance statistic (Maag, 2009)
Some variants are useful for quantifying qualitative disagreement
(Mokinski et al., 2015)
32 / 47
Qualitative vs. Quantitative Questions
The Carlson-Parkin Probability Method
Assumptions:
1 Expectations are formed according to a subjective probability
distribution
2 Individual distributions can be aggregated to a joint pdf f (xt+1 |Ωt )
(often assumed to be normal), which has finite first and second order
moments where E [xt+1 |Ωt ] =t µt+1 and t µt+1 is the expected value
of x at time t for period t + 1
3 There are symmetric intervals (−δ L , δ U ) and (−εL , εU ) around 0 and
around the subjective mean perceived inflation rate µ̃t such that
respondents answer that prices will:
fall if xt+1 ≤ −δ L ⇒ answer proportion t At+1
stay about the same if −δ L < xt+1 ≤ δ U ⇒ answer proportion t Bt+1
rise slightly if δ U < xt+1 ≤ µ̃t − εL ⇒ answer proportion t Ct+1
rise moderately if µ̃t − εL < xt+1 < µ̃t + εU ⇒ answer proportion t Dt+1
rise a lot if µ̃t + εU ≤ xt+1 ⇒ answer proportion t Et+1
33 / 47
Qualitative vs. Quantitative Questions
The Carlson-Parkin Probability Method
f(xt+1|Ωt)
tCt+1
tBt+1
tDt+1
tAt+1 tEt+1
L
-δt 0 δtU tµt+1 µ~ t-εt µt µ
L ~ ~
t+εt
U xt+1
Figure 1: Quantification of pentachotomous survey data
Source: Nielsen (2003). L
t −t
µ̃
U L 34 / 47
Qualitative vs. Quantitative Questions
The Carlson-Parkin Probability Method
Inverse distribution functions (quantile functions) are defined as follows:
F −1 (t At+1 ) =t at+1 (3)
−1
F (t At+1 +t Bt+1 ) =t bt+1 (4)
F −1 (t At+1 +t Bt+1 +t Ct+1 ) =t ct+1 (5)
−1
F (t At+1 +t Bt+1 +t Ct+1 +t Ct+1 ) =t dt+1 (6)
(7)
35 / 47
Qualitative vs. Quantitative Questions
The Carlson-Parkin Probability Method
Then, we get:
µ̃t (t at+1 +t bt+1 )
t µt+1 = (8)
t t+1 t bt+1 −t ct+1 −t
a + dt+1
−2µ̃t
t σt+1 = (9)
t t+1 t t+1 −t ct+1 −t
a + b dt+1
µ̃t (t at+1 −t bt+1 )
δt = (10)
t at+1 +t bt+1 −t ct+1 −t dt+1
µ̃t (t ct+1 −t dt+1 )
εt = (11)
t t+1 t bt+1 −t ct+1 −t
a + dt+1
(12)
where t µt+1 is the quantified aggregate expectation with standard
deviation t σt+1 . The quantification depends crucially on the assumed
distribution function and the value for µ̃t !
36 / 47
Qualitative vs. Quantitative Questions
Analyzing Microdata of Qualitative Expectations
In microdata, there is no immediate way of quantifying qualitative answers,
but we can use estimators that account for qualitative data:
Convert answers into dummy variables (for bivariate questions or if we
want to look at specific categories)
⇒ Probit/logit or bi-probit estimators
Use ordered probit/logit estimators
⇒ Make sure that answer categories are ordered in a meaningful way
37 / 47
Wording Issues
Wording Issues
38 / 47
Wording Issues
Wordings for Eliciting Inflation Expectations
When fielding macroeconomic expectations of non-experts, survey
wording can be important
NY Fed Project (see Bruine de Bruin et al., 2012) and Hamburg BUS
Survey (see Dräger and Fritsche, 2013):
Significant differences between perceptions and expectations given
when asked about “changes in prices in general” or the “inflation rate”
⇒ Perceptions and expectations are lower and less dispersed when asking
about “inflation”
⇒ Asking about “prices in general” seems to elicit (extreme) personal
price experiences (availability bias)
⇒ Media and central bank communication about “inflation”
⇒ What is the inflation rate that people act upon?
39 / 47
Wording Issues
Wordings for Eliciting Inflation Expectations
Figure: Distribution of Inflation Perceptions and Expectations Across Wordings
Quantitative Inflation Perceptions and Expectations
60
60
50
50
Frequency
Frequency
40
40
30
30
20
20
10
10
0
0
−5 0 5 10 15 20 25 −5 0 5 10 15 20 25
Perceived Inflation Rates, Truncated Perceived Increase in Prices in %, Truncated
60
60
50
50
Frequency
Frequency
40
40
30
30
20
20
10
10
0
−5 0 5 10 15 20 25 −5 0 5 10 15 20 25
Expected Inflation Rates, Truncated Expected Increase in Prices in %, Truncated
Source: Dräger and Fritsche (2013).
40 / 47
Wording Issues
Wordings for Eliciting Inflation Expectations
Table: Summary Statistics Quantitative Inflation Expectations Across Wordings
Mean Median SD IQR
All Price Question 4.73 3.00 4.80 3.00
Inflation Question 3.07 2.00 3.69 1.50
Male Price Question 4.47 3.00 4.57 3.00
Inflation Question 3.01 2.00 3.71 1.50
Female Price Question 4.99 3.00 5.00 3.00
Inflation Question 3.13 2.00 3.70 2.25
Low Education Price Question 6.00 5.00 6.22 5.00
Inflation Question 4.20 2.00 5.45 3.00
Medium Education Price Question 4.73 3.00 4.74 3.00
Inflation Question 3.39 2.00 3.97 1.20
High Education Price Question 4.19 3.00 4.14 3.00
Inflation Question 2.21 2.00 2.08 1.00
0-25% Income Price Question 5.83 3.25 6.64 3.50
Inflation Question 3.11 2.00 5.00 2.00
25-50% Income Price Question 5.38 3.00 5.57 3.00
Inflation Question 3.52 2.00 4.58 2.00
50-75% Income Price Question 4.49 3.00 3.03 3.00
Inflation Question 3.47 2.10 3.38 2.50
75-100% Income Price Question 3.74 3.00 3.10 3.00
Inflation Question 2.19 2.00 1.30 1.00
Source: Dräger and Fritsche (2013). 41 / 47
Truncation
Truncation
42 / 47
Truncation
Truncation of Quantitative Survey Expectations
Estimation results can be biased if the data contains extreme outliers
This also applies to survey data of (non-expert) macro expectations
⇒ Truncation of the data
The University of Michigan Survey of Consumers truncates answers at
(-50,50%) (!)
Researchers typically truncate by dropping the top/bottom 2.5% of the
distribution or by dropping expectations outside the range (-5,30%)
43 / 47
Truncation
Truncation of Quantitative Survey Expectations
Figure: Distribution of 1 Year Ahead Inflation Expectations of US Households
4.0e+04
4.0e+04
3.0e+04
3.0e+04
Frequency
Frequency
2.0e+04
2.0e+04
1.0e+04
1.0e+04
0
0
−50 −40 −30 −20 −10 0 10 20 30 40 50 −5 0 5 10 15 20
Expected Inflation 1 Year Ahead, Not Truncated Expected Inflation 1 Year Ahead, Truncated
Source: Microdata from the University of Michigan Survey of Consumers; sample period: 1978m1-2012m9;
Observations 215,823 and 202,664.
44 / 47
Truncation
Literature I
Batchelor, R. A. and A. B. Orr (1988).
Inflation expectations revisited.
Economica 55 (219), 317–331.
Blanchflower, D. G. and R. Kelly (2008).
Macroeconomic Literacy, Numeracy and the Implications for Monetary
Policy.
Working Paper Bank of England .
Bruine de Bruin, W., W. van der Klaauw, G. Topa, J. S. Down, and
O. Armantier (2012).
The effect of question wording on consumers’ reported inflation expectations.
Journal of Economic Psychology 33 (4), 749–757.
45 / 47
Truncation
Literature II
Carlson, J. A. and M. Parkin (1975).
Inflation expectations.
Economica 42 (166), 123–138.
Dräger, L. and U. Fritsche (2013).
Don’t worry, be right! survey wording effects on inflation perceptions and
expectations.
Macroeconomics and Finance Series University of Hamburg 8/2013.
Maag, T. (2009).
On the accuracy of the probability method for quantifying beliefs about
inflation.
KOF Swiss Economic Institute Working Paper 230.
46 / 47
Truncation
Literature III
Mokinski, F., X. Sheng, and J. Yang (2015).
Measuring disagreement in qualitative expectations.
Journal of Forecasting 34 (5), 405–426.
Nielsen, H. (2003).
Inflation expectations in the eu – results from survey data.
Humboldt-Universität zu Berlin, Sonderforschungsbereich 373:
Quantification and Simulation of Economic Processes 13.
47 / 47
