Zlateva et al.

BMC Health Services Research (2015) 15:226

DOI 10.1186/s12913-015-0893-1

RESEARCH ARTICLE Open Access

Development and validation of the Medical

Home Care Coordination Survey for
assessing care coordination in the primary
care setting from the patient and provider
perspectives
Ianita Zlateva1*, Daren Anderson1, Emil Coman2, Khushbu Khatri1, Terrence Tian1 and Judith Fifield2

Abstract
Background: Community health centers are increasingly embracing the Patient Centered Medical Home (PCMH)
model to improve quality, access to care, and patient experience while reducing healthcare costs. Care coordination
(CC) is an important element of the PCMH model, but implementation and measurability of CC remains a problem
within the outpatient setting. Assessing CC is an integral component of quality monitoring in health care systems.
This study developed and validated the Medical Home Care Coordination Survey (MHCCS), to fill the gap in
assessing CC in primary care from the perspectives of patients and their primary healthcare teams.
Methods: We conducted a review of relevant literature and existing care coordination instruments identified by
bibliographic search and contact with experts. After identifying all care coordination domains that could be
assessed by primary healthcare team members and patients, we developed a conceptual model. Potentially
appropriate items from existing published CC measures, along with newly developed items, were matched to each
domain for inclusion. A modified Delphi approach was used to establish content validity. Primary survey data was
collected from 232 patients with care transition and/or complex chronic illness needs from the Community Health
Center, Inc. and from 164 staff members from 12 community health centers across the country via mail, phone and
online survey. The MHCCS was validated for internal consistency, reliability, discriminant and convergent validity.
This study was conducted at the Community Health Center, Inc. from January 15, 2012 to July 15, 2014.
Results: The 13-item MHCCS - Patient and the 32-item MHCCS - Healthcare Team were developed and validated.
Exploratory Structural Equation Modeling was used to test the hypothesized domain structure. Four CC domains
were confirmed from the patient group and eight were confirmed from the primary healthcare team group. All
domains had high reliability (Cronbach’s α scores were above 0.8).
Conclusions: Patients experience the ultimate output of care coordination services, but primary healthcare staff
members are best primed to perceive many of the structural elements of care coordination. The proactive
measurement and monitoring of the core domains from both perspectives provides a richer body of information
for the continuous improvement of care coordination services. The MHCCS shows promise as a valid and reliable
assessment of these CC efforts.
Keywords: Care coordination, Survey research and design, Primary care, Psychometrics, Program Evaluation, PCMH,
Safety net/Federally Qualified Health Centers

* Correspondence: ZlatevI@chc1.com
1
Weitzman Institute, Community Health Center, Inc., Middletown, CT, USA
Full list of author information is available at the end of the article

© 2015 Zlateva et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution License
(http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://
creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 2 of 11

Background measures. Building on and complementing the Atlas, a

The Patient Centered Medical Home (PCMH) model has systematic review of existing CC measures identified 96
been widely implemented to guide system-wide primary different instruments, the majority of which rely on survey
care redesign [1] because of its emphasis on team-based data (88 %) and are applicable to primary care settings
care, “whole person” orientation, access, self-management, (58 %) [17]. Some of these measures were further included
and coordination of care in a complex health system. in the 2014 update of the Atlas [18]. However, none of
While evidence is still relatively limited, some studies of these tools provide a comprehensive assessment of all
the PCMH model have demonstrated improvements in relevant CC domains in the primary care setting or from
health care quality [2, 3], access to care [4], patient and the perspective of health care professionals.
staff experience [5, 6], and health care expenditures To address these problems, we sought to develop and
[2, 3, 7–9]. Studies on PCMH implementation suggest the validate a new measurement survey to assess the core do-
potential for system-wide benefits, but there are many mains of CC for primary care practices involved in Med-
challenges and barriers that primary care practices must ical Home transformation. This new survey, the Medical
overcome to successfully adopt this model. Home Care Coordination Survey (MHCCS), assesses the
Care coordination (CC), one of the core elements of the perspective of the healthcare team (MHCCS-H) and the
PCMH model, is particularly challenging in the current patient’s perspective (MHCCS-P). Since many elements of
healthcare system due to the disjointed, uncoordinated CC relate to activities best perceived by the healthcare
nature of care between multiple providers including pri- team, and since the ultimate output of a CC program is
mary care, specialists, hospitals, emergency rooms, phar- experienced by the patients themselves, the authors be-
macies and others. As an example, a recent study found lieve that both perspectives are important and needed to
that a typical primary care provider (PCP) shares and co- comprehensively evaluate the coordination of care.
ordinates patient care with 229 other physicians [10]. In
addition, there is an inadequate exchange of patient infor- Methods
mation between PCPs and specialists [11]. Given these dif- We conducted this study in several steps: 1) develop-
ficulties, it is unsurprising that few practices standardize ment of a conceptual model; 2) generation of the item
care coordination processes for patients. Only about 3 % pool; 3) review of the items and establishment of content
of small-to-medium-sized primary care practices use care validity; 4) administration of the survey; and 5) psycho-
managers [12], and 46 % of larger practices coordinate metric structural evaluation. Figure 1 outlines select
care for patients with chronic illnesses [13]. For safety-net methodological steps taken in this project.
community health centers, which often service the sickest
patients, the challenge of coordinating care is further Development of conceptual model
compounded by the psychosocial and financial issues Care coordination is a blanket term that encompasses a
more commonly faced by these patients. Medically under- wide range of elements that may be assessed. To help frame
served patients are more likely to live and cope with pov- our work and inform the process of developing and validat-
erty, inadequate housing, unemployment, limited access ing CC measures for the primary care safety-net setting, we
to specialty care, and linguistic and cultural barriers [14]. created a PCMH CC Conceptual Model (Additional file 1).
These factors contribute to the general poor health that This model uses the consensus AHRQ definition of CC,
characterizes these patients and place added burden on which is “the deliberate organization of patient care
providers seeking to coordinate and manage their care. activities between two or more participants (including the
Implementing a CC process in primary care is further patient) involved in a patient’s care to facilitate the appro-
complicated by multiple models with different conceptual priate delivery of health care services. Organizing care in-
emphases and a surfeit of evaluation frameworks [15]. To volves the marshaling of personnel and other resources that
address the dearth of clarity, the Agency for Healthcare are needed to carry out all required patient care activities
Research and Quality (AHRQ) published the Care Coord- and is often managed by the exchange of information
ination Measures Atlas [16]. In this report, the authors among participants responsible for different aspects of care”
review theoretical frameworks that underscore the study [16]. In developing this model, we conducted an extensive
of CC, develop a standardized definition and outline a literature review and built on concepts from the Care Co-
broad range of measures for evaluating various domains ordination Measures Atlas [16] and the National Quality
of CC. However, none of these tools fully address the as- Forum (NQF) [19].
sessment needs of primary care practices looking to im- We utilized the NQF-endorsed framework for CC that es-
plement and monitor CC programs. Furthermore, while tablishes five domains essential to measurement: healthcare
there are 22 surveys cited within this report that relate home, the use of a proactive plan and follow-up of care,
specifically to the Medical Home, the authors call for fur- communication between all members of the healthcare
ther study to help determine the applicability of such team and the patients, care transitions, and information
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 3 of 11

Fig. 1 Schematic of the project methods and select results

systems [19]. In addition, we based our conceptual model Generation of the item pool
on the dominant theoretical model in health services re- The newly created PCMH CC Conceptual Model was used
search, the Donabedian model [20], which emphasizes a as a foundation to design the survey, with the goal of ensur-
systems-level perspective on the determinants of health- ing each element of the conceptual model was appropriately
care quality. According to this model, care management reflected in the new survey from the perspectives of the pa-
structures combined with defined care management pro- tient, the administrative staff, and the clinical personnel.
cesses produce desired outcomes. We adapted the Dona- Two researchers independently reviewed existing survey
bedian Model by identifying essential CC structures instruments from the Atlas [16] and from an updated
(inputs) and process factors (activities) with the potential
to affect patient and staff satisfaction as well as clinical
and financial outcomes.
The following domains were selected for inclusion in
the measures: (1) Healthcare Home; (2) Plan of Care; (3)
Self-Management; (4) Communication; (5) Patient Assess-
ment and Support; and (6) Care Transitions. Three of the
domains were further divided into subdomains. The
Healthcare Home domain was divided into CC Practice
Infrastructure, Accountability, and IT Capacity subdo-
mains. The Communication domain consisted of the
Interpersonal Communication and Information Transfer
subdomains. The Patient Assessment and Support domain
included the Needs Assessment, Linkage to Community
Resources, and Monitoring, Follow Up, and Responding
to Status Change subdomains (see Fig. 2). After identifying
all domains that could appropriately be assessed by
primary healthcare team members and their patients, we
described the structures (inputs) and processes (activities)
involved in those domains and subdomains, and the pos-
sible short-term and long-term outcomes (see Additional
Fig. 2 Depiction of the PCMH Care Coordination Conceptual Model
file 1).
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 4 of 11

literature search. From these tools, they selected poten- format, language, response options, skip patterns and defi-
tially appropriate items for inclusion. The Atlas review in- nitions used.
cluded 3448 articles, from which 78 potentially useful The Delphi participants confirmed the hypothesized
survey instruments were identified. The literature search domain-subdomain structure for the MHCCS. The
strategy described in the Atlas [16] was used to update the MHCCS - P comprised 45 questions over six distinct
review and identified 861 additional articles (for the period domains, while the MHCCS - H comprised 57 items
from January 2010 to May 2012), from which three add- over six domains. The major difference between the two
itional potentially useful survey instruments were selected. models was the absence of the “Information Technology
Appropriate permission was obtained to include and/or Capacity” subdomain in the patient version.
modify items from the instruments that were selected. Finally, cognitive interviews were conducted with three
From the selected 81 survey instruments, we identified patients who had received CC services at the Community
326 potentially useful items. After removing redundant Health Center, Inc. to verify the ease of comprehension of
items, 119 items remained for possible inclusion. Each the survey items, survey instructions, study information
item was independently mapped by each of the two re- sheet, definition of key terms, skip patterns and response
viewers to one domain in the conceptual model, with a options. Their input led to a final version of the survey in-
consensus process used when differences were observed. strument. All survey questions were worded to be written
New items were developed where important constructs at a fourth grade reading level.
lacked specific measures and to ensure that each domain In both versions of the MHCCS, all care coordination
and subdomain in the model contained at least two items. questions were structured in a 5-point Likert scale format.
Questions were also reworded so that they had a consist- The last few questions of each survey were multiple-
ent structure for the Delphi process. choice and represented demographic and control variable
information. These selected questionnaire items were not
Review of the items and establishment of content validity included in the analyses undertaken to identify the final
In order to examine content validity we adopted an an- domain structures, and instead provided support for the
onymous, web-based Delphi Technique, which is an itera- convergent and predictive validities.
tive method to help derive consensus in areas that lack
sufficient scientific evidence [21–23]. We used a modified Administration of the survey
electronic version of the Delphi technique to obtain expert Patients were recruited from a large, multi-site Federally
opinion and consensus regarding the design of the final Qualified Health Center (FQHC) located in Connecticut.
survey. To recruit participants for the Delphi process, we Community Health Center, Inc. (CHCI) provides compre-
identified experts in the field of CC and PCMH based on hensive primary care services in 12 health centers across
their having significant publications, a national/international the state and over 200 additional sites including school-
profile, and/or substantial clinical/practical experience in based clinics, homeless shelters, and mobile outreach sites.
the field. Of the 16 invited experts, three declined participa- CHCI cares for over 130,000 medically underserved pa-
tion, three did not respond, and ten agreed to participate. tients in the state. Over 60 % of CHCI patients are racial/
To carry out each of the four rounds of the Delphi ethnic minorities; over 90 % are below 200 % of the federal
process we used REDCap (Research Electronic Data Cap- poverty level, 60 % are on Medicaid or state insurance,
ture) [24], a secure, web-based application designed to and 22 % are uninsured. This study was reviewed and ap-
support data capture for research studies. In the first three proved by the Institutional Review Board at CHCI and
rounds, experts reviewed the pool of items and rated, on a conducted from January 15, 2012 to July 15, 2014.
5-point Likert scale, each item’s appropriateness and abil- Inclusion criteria for patients were: age 18 years or older,
ity to assess the indicated element of the conceptual having English as a preferred language and a medical visit
model. Participants could also suggest that an item be at CHCI within the past 12 months. In addition to these
reworded, moved to another domain, or eliminated. After criteria, patients had to have had at least one of the fol-
each round, items that received an “Appropriate” or “Very lowing: 1) two or more emergency room visits in the past
Appropriate” rating from 80 % or more of the experts year; 2) a hospitalization in the past year; 3) diabetes with
were accepted for inclusion in the measures, while items a hemoglobin A1C test result greater than 9 % in the past
that received an “Inappropriate” or “Very Inappropriate” 6 months; 4) four or more of the following chronic ill-
rating from more than 50 % of the experts were removed. nesses: chronic obstructive pulmonary disease (COPD),
The items that did not reach consensus either way were hypertension, asthma, diabetes and coronary artery dis-
presented again to the experts for review in the next ease (CAD). A total of 695 eligible patients were randomly
round. After each round, questions were modified and chosen through the electronic health record system and
reworded based on the qualitative input from the experts. were invited to complete the survey either on paper or on-
In the final round, experts commented on the general line. Patients were incentivized to complete the MHCCS-
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 5 of 11

P, as they would be entered into a raffle to win one of five survey to all PCPs, nurses, and clinical administrative
$50 gift cards. Patients who did not return the survey two staff. After approximately five weeks, the lead contact
weeks after the initial mailing were sent a second survey, person at each site was asked to email a reminder to all
followed by a reminder phone call. Patients who failed to staff to complete the survey. Respondents’ characteristics
complete the survey after this round received a second re- are summarized in Table 2.
minder survey in the mail, followed by a second phone re- Of the 162 invited CHCI staff members, 55 returned
minder. During this final reminder call, patients were also completed surveys for a 34 % response rate. An add-
offered the opportunity to complete the survey over the itional 109 completed surveys were received from the
phone. In total, 232 surveys were completed for a re- other 11 health centers invited to participate. These
sponse rate of 33.4 %. The responders’ socio-demographic health centers were unable to provide the total number
and inclusion characteristics are reported in Table 1. of staff who were offered the survey, preventing calcula-
The healthcare team survey (MHCCS –H) was admin- tions of a final response rate.
istered at CHCI and 11 other large, multi-site FQHCs Respondents to the MHCCS indicated their consent to
across the country. At each site, the Chief Executive Of- participate in the study by completing the survey.
ficer (CEO) or the Chief Medical Officer (CMO) was In addition, clinical and operational data were ob-
asked to email the invitation to complete the online tained from CHCI’s EHR and practice management
system.
Table 1 Patient responder characteristics
Psychometric and structural validation
Total (%)
Data were analyzed using IBM SPSS Statistics 20 Software
(N = 232)
Package [25] and Mplus 7.11 [26]. An initial sense of how
Gender Female 133 (57)
items clustered in domains was provided by comparing
Male 99 (43) correlations of each item with items from the same hy-
Ethnicity Black or African American 44 (19) pothesized domain to correlations with items from other
Caucasian 141 (61) domains, or the within-domain vs. between-domains aver-
Hispanic 38 (16) age correlations. The quantitative validation of the mea-
sures followed current psychometric standards [27, 28],
Other 5 (2)
grounded in classical test theory [29, 30], which involve
Unreported 4 (2)
primarily testing the reliability and validity of the measure.
Age 20-29 6 (3) Since the very structures of the measures were meant to
30-39 13 (6) be validated (i.e., which items cluster under what specific
40-49 48 (21) domain), classic reliability indices like Cronbach’s α alone
50-59 89 (38) would be inadequate for measures that are not first
proven to be unidimensional. The structure of multi-
60-69 51 (22)
dimensional measures (and hence their unidimensionality)
70+ 25 (11)
is best tested in the Structural Equation Modeling (SEM)
Educational Level No schooling 3 (1) framework [31–33]. The SEM approach assumes the
Grade 1 to 12 53 (23)
Table 2 Healthcare team responder characteristics
High school or GED completed 72 (31)
Total (%)
Some college 51 (22)
(N = 164)
Associate’s degree 15 (6)
Gender Male 27 (16)
Bachelor’s degree 14 (1)
Female 97 (59)
Advanced degree 2 (1)
Unknown 40 (24)
Other 14 (6)
Self-Identified Roles Administrator 15 (9)
Missing 8 (3)
Nurse (e.g., RN,LPN) 21 (13)
Inclusion Categoriesa Hospitalized 122 (53)
Nurse Care Coordinator 6 (4)
2 or more ER Visits 69 (30)
Nurse Practitioner (e.g., APRN) 16 (10)
A1C above 9 46 (20)
Physician Assistant 8 (5)
4 Chronic Illnesses† 22 (9)
Primary Care Physician (e.g., MD, DO) 40 (24)
a
Some patients fit two or more inclusion criteria
†
Four or more of the following chronic illnesses: chronic obstructive Other 16 (10)
pulmonary disease (COPD), hypertension, asthma, diabetes and coronary Unknown 42 (26)
artery disease (CAD)
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 6 of 11

existence of unobserved (latent) variables that explain the and the role of the staff in the practice. Discriminant valid-
correlations between a group of effect indicators, i.e., the ity of the CC domains was tested by determining that no
observed items assumed to belong to the same domain relationships existed with unrelated concepts; specifically
[34]. A range of exploratory and confirmatory factor ana- for MHCCS-P, patients’ gender and ethnicity, and for
lyses (EFA and CFA) can be tested on the data to estimate MHCCS-H the gender of staff. Lastly, predictive validity
both the number of domains (or factors) and what ques- or the ability to predict other relevant outcomes was
tions belong to what factor or confirm a specific number tested for MHCCS-P only, with a health rating item (from
of domains, with a more or less specified structure per fac- poor = 1 to excellent = 5).
tor known a priori. We employed such a sequence of
models, starting with a full CFA model to test the struc- Results
ture as initially hypothesized, then moved into a more ad- Survey validation
vanced recent combination of the two, called Exploratory Cronbach’s α of the proposed domains are measures of in-
Structural Equation Modeling (ESEM) [35]. ESEM can ternal consistency, as they reflect average inter-item correla-
group the survey items (indicators) into a pre-specified tions. Pure confirmatory factor analyses of the hypothesized
number of factors (in our case, the hypothesized do- structures of the patient and healthcare team CC measures
mains), while allowing items to belong simultaneously to were rejected, so a sequence of ESEM models were tested
different domains, and incorporating correlations between (syntax and output are available at trippcenter.uchc.edu/
item residuals. We tested ESEMs with increasingly more modeling). When loadings were non-significant (less than .5
domains for both the patient and healthcare team CC do- in standardized values), and R2 were small (less than .5, or
mains, until a clear structure emerged, based on large 50 % of the variance in that item explained by the latent fac-
enough standardized loadings and explained variance (or tor), items were dropped from that factor. CFA models were
R2), which led to final CFA models. The reliability of the iteratively trimmed by dropping items that were not ex-
domains that emerged from the final models was then plained well by the model and regrouping items when less
assessed. Convergent validity of the CC domains was than three items were left in a factor/domain.
tested by confirming sizeable correlations with other mea- Starting with 45 questionnaire items in the MHCCS-P, 32
sures that the domains were expected to correlate with; were dropped from the reliability and internal consistency
specifically for the MHCCS-P, whether patients have a testing. For the MHCCS-H, these numbers were 57 and 25,
care plan, a rating of the level of CC received (agreement, respectively. Thus, the final MHCCS-P consisted of 13
on a 1-5 scale), and a question rating the care received items, and the MHCCS-H consisted of 32 items.
(from poor = 1 to excellent = 5), and for MHCCS-H, a rat- The final results are shown in Tables 3 and 4. The
ing of the care coordination provided (from poor = 1 to MHCCS-P and MHCCS-H can be found in Additional files
excellent = 5), one question asking whether someone in 2 and 3 in their final format. Four distinct patient CC do-
the practice coordinates care (agreement, on a 1-5 scale), mains and eight provider CC domains emerged from the

Table 3 Structure of the final four domain patient survey as emerged from analyses
Care coordination domain Items λ R2
1 Plan of Care (PC) α = .909 My PCT (Primary Care Team) helps me plan so I can take care of my health .93 .87
My PCT follows through with the care plan it creates with me .89 .78
Someone on my PCT helps me set goals for taking care of my health .92 .85
My PCT asks for my ideas when we make a plan for my care .88 .78
2 Communication (Comm) α = .899 Someone on my PCT tells me all my test results, good and bad .97 .94
I get the results of my lab tests in a timely manner .95 .89
Someone on my PCT helps me understand what my lab tests .70 .50
3 Link to Community Resources (ComRes) Someone on my PCT gives me information about services offered at their office or in my .89 .80
α = .893 community
Someone on my PCT asks me about what I need for support .87 .76
Someone on my PCT encourages me to attend programs in my community .78 .60
4 Care Transitions (CT) α = .893 After I leave the hospital, my PCT knows about new prescriptions or if there was a change .94 .89
After I leave the hospital, my PCT helps me get back on my feet .85 .73
After I leave the hospital, my PCT knows about the care I received from the hospital .66 .43
All four domains correlated pairwise with each other significantly (p < .001) and moderately (.44 to .75); the χ2 (54) = 66.7, p = .115, CFI = .987, RMSEA = .046, 95 %
CI [.001; .078], five pairs of indicators’ errors were correlated
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 7 of 11

Table 4 Structure of the final 8 domain Healthcare Team survey as emerged from analyses
Domain Items λ R2
1 Accountability (Acc) α = .844 The PCT (Primary Care Team) team is made up of members with clearly defined roles, such as patient .72 .52
self-management, education, proactive follow up and resource coordination.
The PCT and patients share responsibilities in managing patients’ health. .74 .55
The PCT is characterized by collaboration and trust. .78 .60
The PCT works with patients to help them understand their roles and responsibilities in care. .74 .55
2 IT capacity (IT) α = .874 The PCT uses electronic data to monitor and track patient health indicators and outcomes. .83 .69
N
The PCT team uses electronic data to support the documentation of patient needs. .75 .56
N
The PCT uses electronic data to develop care plans. .79 .63
N
The PCT uses electronic data to determine clinical outcomes. .90 .80
3 Plan of Care (PC) α = .903 The PCP asks for patients’ input when making a plan for their care. N .82 .67
N
The PCT helps make care plans that patients can follow in their daily life. .89 .80
The PCT develops care plans that incorporate plans recommended by other health care providers .91 .83
patients see. N
4 Follow-up Plan of Care (FPC) The PCT team reviews and updates patients’ care plan with them. N .81 .65
α = .886 N
The PCT follows through with the care plan. .74 .55
The PCT uses patients’ care plan to follow progress. N .80 .64
The PCT helps patients plan so they can take care of their health even when things change or when .78 .62
unexpected things happen.
5 Self-Management (SM) α = .803 Someone on the PCT team helps patients set goals for managing their health. .77 .60
Someone on the PCT team checks to see if patients are reaching their goals. .75 .56
The primary care practice/health center has behavior change interventions readily available for .61 .37
patients as part of routine care.
The primary care practice/health center has peer support readily available for patients as part of .68 .46
routine care.
6 Communication (Comm) α = .865 The PCT team informs patients about any diagnosis in a way that patients can understand. .78 .61
The PCT team helps patients understand all of the choices for their care. .78 .61
The PCT team considers and respects patients’ values, beliefs and traditions when recommending .74 .55
treatments.
The PCT team’s care coordination activities are based upon ongoing assessment of patient needs. .75 .56
7 Link to Community Resources Someone on the PCT team offers patients the opportunity to learn more about managing their health, .74 .54
(ComRes) α = .896 such as with group appointments, support groups and patient education.
Someone on the PCT team asks patients about what they need for support, such as care programs, .79 .62
financial services, equipment and transportation.
Someone on the PCT team gives patients information about additional supportive services offered at .86 .75
the practice/health center or in their community, such as counseling programs, support groups or
rehabilitation programs.
Someone on the PCT team encourages patients to attend programs in their community that could .79 .63
help them, such as support groups or exercise classes.
Someone on the PCT team connects patients to needed services, such as transportation or home .83 .69
care.
8 Care Transitions (CT) α = .875 When patients are discharged from the hospital, the PCT team is informed about the care patients .69 .48
received from the hospital.
When patients are discharged from the hospital, the PCT team receives information from the hospital .68 .47
about new prescriptions or if there was a change in medication.
When patients are discharged from the hospital, their primary care medical record includes a .91 .83
discharge summary in a timely manner. N
When patients are discharged from the hospital and there are test results pending, their primary care .85 .72
medical record includes the test results within 2 weeks. N
Indicators in italics were originally hypothesized to belong to a different domain; N: items had never/always response options, while the others had the disagree/
agree options; fit was χ2 (417) = 639.3, p < .001, CFI = .931, RMSEA = .058, 95%CI [.049; .067]; 19 pairs of residual errors were correlated
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 8 of 11

final analyses, with the following four being common be- for primary care. They hold the potential to be of particular
tween CC measure types: Plan of Care (PC), Communica- use to primary care practices seeking a practical tool to help
tion (Comm), Link to Community Resources (ComRes), assess CC in the medical home environment.
and Care Transitions (CT). The provider CC measure had In this study, we used a consensus approach to select ap-
four additional domains: Accountability (Acc), IT capacity propriate CC domains and develop measures specific to the
(IT), Follow-up PC (FPC), and Self-Management (SM); a outpatient community health setting, and then validated
total CC score was computed for each CC measure type as these measures in a real-world practice environment. At the
the average of all domains. Internal consistency was high for time this study was conducted, CHCI was implementing a
all MHCCS-P domains (Cronbach alphas = .893 - .909) and standard CC model as part of its adoption of the PCMH
for all MHCCS-H domains (Cronbach alphas = .803 - .903). model. It thus provided an ideal testing environment for the
All four final domains and the global CC score correlated new measures.
(from .377 to .708, p < .001) with whether patients had a Although some domains were collapsed or discarded
care plan, and with the ratings of the care received and of during the ESEM analyses, most were maintained in the
the care coordination received. Overall, CC domains and final version of the healthcare team survey. Similarly, key
CC scores did not differ by patient education level or ethni- domains that suitably represent some of the most salient
city, with the exception of the Communication domain, sup- features of the PCMH CC Conceptual Model such as Care
porting MHCCS-P’s discriminant validity. The predictive Plan, Communication, Patient Assessment and Support
validity of the MHCCS–P was also confirmed because the and Care Transitions, were retained in the final version of
global CC score and three of the four domains, all except the patient survey. The ESEM analyses, however, excluded
Care Transitions (r = .048, p = .653), correlated with the self- from the patient survey two domains that can be essential
rated health (.235, p < .001 with PC; .272, p < .001 with to the model from a clinical and a PCMH perspective –
Comm; .177, p < .001 with ComRes). the Self-Management and Healthcare Home domains.
The MHCCS-H domains (and the overall CC score) This may suggest that these individual domains may be
showed good convergent validity, all being correlated with less perceived by patients, or less visible to them as stan-
the rating of the care coordination provided (from .402 to dalone domains of care as measured by the survey items.
.628, all p values < .001), and with how the practice coordi- We tried to analytically re-attach items from the discarded
nates care (from .433 to .630, all p values < .001). Moreover, domains to the final patient survey structure, as it was
there were differences by staff role in Accountability (Acc), strongly suggested during the survey development process
IT capacity (IT), Follow-up Plan of Care (FPC), Link to that they were clinically relevant and conceptually consist-
Community Resources (ComRes), and the overall provider ent. The resulting models were rejected purely on statis-
CC score (CC), with average scores ordered as follows: tical grounds, because the items’ removal helped to clarify
Administrator (highest) > Nurse (middle) > PCP (lowest) for the structure of the domains that were retained. We
all domains and the overall CC score. No differences were recognize that the poor performance of the rejected items
seen by gender of respondent, except for females reporting may have been due to the nature of the study sample ra-
higher healthcare team CC average scores for the Plan of ther than the properties of the items. While we think prac-
Care (PC), Follow-up PC (FPC), and Communication titioners and researchers should continue to validate the
(Comm) domains, confirming discriminant validity. The structure of the MHCCS-P as emerged from our analyses
predictive validity of MHCCS-H needs further investigation. with other samples of patients, we also suggest they con-
sider alternative solutions by including some of the items
Discussion that were rejected with the current model.
We developed the MHCCS-P and MHCCS-H for assessing Healthcare reform efforts are shifting the emphasis to ac-
the provision of CC in the primary care safety-net setting countable care. This shift, combined with incentives to im-
from the perspectives of patients and the healthcare team, plement the PCMH model and obtain recognition from
and examined each survey’s construct validity among the agencies such as the National Center for Quality Assurance
patient sample at a large FQHC and among a clinical staff (NCQA) are leading to a growing interest in improving care
sample from 12 FQHCs across the country. The resulting coordination across the healthcare continuum. As primary
models provided a reasonable fit and revealed satisfactory care practices seek to implement CC within the PCMH
levels of internal consistency reliability. The self-report sur- model of care, increased attention and support will be
vey provides a framework for evaluating the coordination needed to assist them with implementation of key fea-
of care for patient populations requiring complex care tures, including a well-functioning team that focuses on
within the primary care setting and in critical transitions. the patient’s needs while using evidence-based practices.
The MHCCS-P and MHCCS-H are, to our knowledge, the Strategies will be needed to enable teams to function
first to incorporate a broad range of CC domains and pro- effectively in this mode and help them establish account-
vide a comprehensive, non-condition-specific assessment ability and negotiate responsibilities for the desired
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 9 of 11

outcomes with their patients. Actively engaging patients and To better capture the process of CC and its quality, prac-
their families as members of the medical home care team tices should consider using the MHCCS-P and MHCCS-H
and the medical neighborhood is critical to the PCMH together. Assessments of CC processes that are more diffi-
model. Similar to the CAHPS Patient-Centered Medical cult to capture in a survey can benefit from advanced analyt-
Home Survey (PCMH CAHPS) [36] the MHCCS-H asks ical approaches to yield additional insight into contextual
about care provided by the entire primary care team, not factors that facilitate or impede CC. Combining MHCCS
just the primary care clinician. Items loading on the Ac- data with quantitative data may provide the most thorough
countability domain in the healthcare team survey reflect and balanced assessment of CC quality in primary care. The
the fact that care coordination involves multiple different exact measures and data collection methods need to be de-
members of the care team, including the patient. The Ac- termined based on the purpose for the CC assessment [40].
countability items pertain to collaboration and working to- A major strength of the MHCCS-P and MHCCS-H is that
gether in new ways, and gauge healthcare team members’ they assess all relevant domains of CC. Given that the indi-
ability to successfully share responsibilities in managing pa- vidual domain measures performed well, individual compo-
tients’ health as a team. nents of the survey may be able to be used on their own.
Patient-centered outcomes are critical for a more balanced It is particularly worth noting that concurrently with this
assessment of healthcare quality [37]. Since CC is essentially work, the NQF released a revised CC framework and prior-
dedicated to identifying patient needs and helping to meet ities for CC measurement in September 2014 [41]. Their
those needs on an individual case-by-case basis, patient final conceptual framework includes eight subdomains:
feedback should be an essential part of any evaluation. Our Comprehensive Assessment, Goal-Setting, Shared Account-
MHCCS-P incorporates such patient feedback in assessing ability, Linkages/Synchronization, Quality of Services, Ex-
the quality of CC. We recruited patients in the study exclu- perience, Progression toward Goals and Efficiency. Each of
sively from one large multi-site FQHC and achieved a re- these subdomains maps onto a corresponding domain or
sponse rate of 33.4 %, which was close to the rates (35-44 %) subdomain in the PCMH CC Conceptual Model used in this
reported for low-income populations in the Consumer As- project. Additionally, the multi-stakeholder CC reviewing
sessment of Health Plans Study (CAHPS) [38]. We used a committee recommended deliberate action to fill perform-
similar low-income, low-literacy patient population to valid- ance gaps in addressing four of these eight domains: Com-
ate the survey. Such patients have higher rates of chronic ill- prehensive Assessment, Shared Accountability, Linkages/
ness, poorer health outcomes overall [14], and are more Synchronization, and Progression toward Goals. The final
likely to require support in the form of CC than patients in domain structure of the MHCCS-P upholds and addresses
the general population. While this is one of the strengths of each of these: Plan of Care, Accountability, Link to Commu-
this study, it is also a limitation in that the results may not nity Resources and Follow-up Plan of Care, respectively.
be generalizable to wider patient populations. In addition, it Further research is needed to assess whether the indi-
is important to note that the MHCCS-P was validated with vidual domain scores and total CC scores improve in re-
data collected in one region of the country and that the sponse to a CC intervention, whether survey scores are
characteristics of participating patients may have differed associated with clinical outcomes, satisfaction in care, and
from those of patients who chose not to participate. Per- healthcare costs and savings, and to explore the feasibility
formance of this survey, including rejected MHCCS-P of a single dyadic patient-provider CC measure [42–44].
items, should be reevaluated in a more diverse primary care
patient population. Conclusions
While the use of risk screening tools is a promising In conclusion, we developed the MHCCS-P and MHCCS-
method [39], there is no one best method for identifying H, with questions mapped to each domain of a broad con-
patients in need of CC within the medical home. Indi- ceptual model of CC. Our findings suggest that the frame-
vidual patients may have a need for different forms of work has both clinical as well as construct validity. The
CC (either simultaneously or at different points in time). MHCCS-P and MHCCS-H were designed to measure qual-
A patient recently discharged from the hospital may ity of CC from the perspectives of the patients and the
need brief transition care support, while a patient with healthcare team. Both instruments demonstrated good reli-
poorly controlled chronic illness may need disease man- ability and discriminant validity in this first field test. They
agement, self-management support and a care plan as can be used separately or together to evaluate CC strengths
well as links to community resources and supports. and areas for improvement within the medical home prac-
These complex needs led us to select a comprehensive tice. Although developed and validated for measuring CC at
set of patient inclusion criteria, which in turn allowed FQHCs, the survey instruments may be relevant for measur-
for variation in the level of received CC (patients re- ing CC among other primary care populations. Further
ceived low level, intermediate and high-level care studies are needed to determine whether the survey can de-
coordination). tect clinically important changes over time.
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 10 of 11

Additional files 12. Rittenhouse DR, Casalino LP, Shortell SM, McClellan SR, Gillies RR, Alexander
JA, et al. Small and medium-size physician practices use few patient-
centered medical home processes. Health Aff (Millwood). 2011;30(8):1–10.
13. Rittenhouse DR, Shortell SM, Gillies RR, Casalino LP, Robinson JC, McCurdy
Additional file 1: PCMH Care Coordination Conceptual Model. RK. Improving chronic illness care: findings from a National Study of Care
Additional file 2: Final MHCCS-P. Management Processes in Large Physician Practices. Med Care Res Rev.
2010;67(3):301–20.
Additional file 3: Final MHCCS-H.
14. Institute of Medicine. Unequal treatment: confronting racial and ethnic
disparities in health care. Washington: The National Academies Press; 2003.
Competing interest 15. Van Houdt S, Heyrman J, Vanhaecht K, Sermeus W, De Lepeleire J. An
The authors declare that they have no competing interests. This research in-depth analysis of theoretical frameworks for the study of care
was supported by Aetna Inc., one of the nation’s leaders in health care, coordination. International Journal of Integrated Care. 2013;13:e024
dental, pharmacy, group life, and disability insurance, and employee benefits. 16. McDonald KM, Schultz E, Albin L, Pineda N, Lonhart J, Sundaram V, et al. Care
The views presented here are those of the authors and not necessarily those Coordination Atlas Version 3 (prepared by Stanford University under subcontract
of Aetna, its directors, officers, or staff. to Battelle on contract no. 290-04-0020). Rockville, MD: Agency for Healthcare Re-
search and Quality; 2010. Available at: http://www.ahrq.gov/professionals/systems/
long-term-care/resources/coordination/atlas/index.html
Authors’ contributions 17. Schultz EM, Pineda N, Lonhart J, Davies SM, McDonald KM. A systematic review of
IZ and DA conceived the study, and significantly contributed to the study the care coordination measurement landscape. BMC Health Serv Res.
design and manuscript draft. TT and KK coordinated all aspects of the study 2013;13(1):119.
including the acquisition of survey data and contributed to the drafting of 18. McDonald KM. Care coordination measures atlas update. 2014.
the manuscript. EC conducted the statistical analyses and helped to draft the 19. National Quality Forum. Preferred practices and performance measures for
manuscript. JF contributed to the design of the project. All authors read and measuring and reporting care coordination: a consensus report.
approved the final manuscript. Washington, DC: National Quality Forum; 2010.
20. Donabedian A. The Criteria and Standards of Quality. Ann Arbor, MI: Health
Acknowledgements Administration Press; 1982.
This research was supported by Aetna Inc., one of the nation’s leaders in 21. Hsu C, Sandford BA. The Delphi Technique: Making Sense of Consensus.
health care, dental, pharmacy, group life, and disability insurance, and PARE. 2007(10):1-8
employee benefits. The views presented here are those of the authors and 22. Keeney S, Hasson F, McKenna HP. A critical review of the Delphi technique
not necessarily those of Aetna, its directors, officers, or staff. as a research methodology for nursing. Int J Nurs Stud. 2001;38(2):195–200.
23. Hasson F, Keeney S. Enhancing rigour in the Delphi technique research.
Author details Technol Forecast Soc Chang. 2011;78(9):1695–704.
1
Weitzman Institute, Community Health Center, Inc., Middletown, CT, USA. 24. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic
2
Ethel Donaghue TRIPP Center, University of Connecticut Health Center, data capture (REDCap)—a metadata-driven methodology and workflow process
Farmington, CT, USA. for providing translational research informatics support. J Biomed Inform.
2009;42(2):377–81.
Received: 6 November 2014 Accepted: 29 May 2015 25. Nie NH, Bent DH, Hull CH. SPSS: Statistical package for the social sciences
(Vol. 227). New York, NY: McGraw-Hill 1975.
26. Muthén L, Muthén B. Mplus user’s guide 5. Los Angeles: Muthén & Muthén;
References 1998–2007.
1. Jackson GL, Powers BJ, Chatterjee R, Bettger JP, Kemper AR, Hasselblad V, 27. John OP, Benet-Martinez V. Measurement: Reliability, construct validation,
et al. The patient-centered medical home: a systematic review. Ann Intern and scale construction. Handbook of Research Methods in Social and
Med. 2013;158(3):169–78. Personality Psychology. Cambridge: Cambridge University Personality; 2000.
2. Gilfillan RJ, Tomcavage J, Rosenthal MB, Davis DE, Graham J, Roy JA. Value 28. Raykov T, Marcoulides GA. Introduction to psychometric theory. New York,
and the medical home: effects of transformed primary care. Am J Manag NY: Taylor & Francis; 2010.
Care. 2010;16:607–14. 29. Gulliksen H. Theory of mental tests. New York, NY: Routledge; 2013
3. Steele GD, Haynes JA, Davis DE, Tomcavage J, Stewart WF, Graf TR, et al. 30. Lord FM, Novick MR, Birnbaum A. Statistical theories of mental test scores.
How Geisinger's advanced medical home model argues the case for Oxford: Addison-Wesley; 1968.
rapid-cycle innovation. Health Aff (Millwood). 2010;29(11):2047–53. 31. Bentler PM, Weeks DG. Linear structural equations with latent variables.
4. Starfield B, Shi L. The medical home, access to care, and insurance: a review Psychometrika. 1980;45(3):289–308.
of evidence. Pediatrics. 2004;113(Supplement 4):1493–8. 32. Bollen KA. A new incremental fit index for general structural equation
5. Jaen DR, Ferrer RL, Miller WL, Palmer RF, Wood R, Davila M. Patient models. Sociol Methods Res. 1989;17(3):303–16.
outcomes at 26 months in the patient-centered medical home national 33. Bollen KA. Latent variables in psychology and the social sciences. Annu Rev
demonstration project. Ann Fam Med. 2010;8(S1):S57–67. Psychol. 2002;53(1):605–34.
6. Reid RJ, Fishman PA, Yu O, Ross TR, Tufano JT, Soman MP. Patient-centered 34. Raines-Eudy R. Using structural equation modeling to test for differential
medical home demonstration: a prospective, quasi-experimental, before and reliability and validity: An empirical demonstration. Struct Equ Model.
after evaluation. Am J Manag Care. 2009;15(9):e71–87. 2000;7(1):124–41.
7. Domino ME, Humble C, Lawrence Jr WW, Wegner S. Enhancing the medical 35. Marsh HW, Liem GAD, Martin AJ, Morin AJ, Nagengast B. Methodological
homes model for children with asthma. Med Care. 2009;47(11):1113–20. measurement fruitfulness of exploratory structural equation modeling
8. Maeng DD, Graham J, Graf TR, Liberman JN, Dermes NB, Tomcavage J, et al. (ESEM): New approaches to key substantive issues in motivation and
Reducing long-term cost by transforming primary care: evidence from engagement. J Psychoeduc Assess. 2011;29(4):322–46.
Geisinger’s medical home model. Am J Manag Care. 2012;18(3):149–55. 36. Scholle SH, Vuong O, Ding L, Fry S, Gallagher P, Brown JA, et al.
9. Reid RJ, Coleman K, Johnson EA, Fishman PA, Hsu C, Soman MP, et al. The Development of and field test results for the CAHPS PCMH Survey. Med
group health medical home at year two: cost savings, higher patient Care. 2012;50(Suppl):S2–S10.
satisfaction, and less burnout for providers. Health Aff. 2010;29(5):835–43. 37. Friedberg MW, Schneider EC, Rosenthal MB, Volpp KG, Werner RM.
10. Pham HH, O’Malley AS, Bach PB, Salontz-Martinez C, Schrag D. Primary care Association between participation in a multipayer medical home
physicians’ links to other physicians through medicare patients: the scope intervention and changes in quality, utilization, and costs of care. JAMA.
of care coordination. Ann Intern Med. 2009;150(4):236–42. 2014;311(8):815–25.
11. O’Malley AS, Reschovsky JD. Referral and consultation communication 38. Fredrickson DD, Jones TL, Molgaard CA, Carman CG, Schukman J, Dismuke
between primary care and specialist physicians: finding common ground. SE, et al. Optimal design features for surveying low-income populations.
Arch Intern Med. 2011;171(1):56–65. J Health Care Poor Underserved. 2005;16(4):677–90.
Zlateva et al. BMC Health Services Research (2015) 15:226 Page 11 of 11

39. Haas LR, Takahashi PY, Shah ND, Stroebel RJ, Bernard ME, Finnie DM, et al.
Risk-stratification methods for identifying patients for care coordination.
Am J Manag Care. 2013;19(9):725–32.
40. McDonald KM, Sundaram V, Bravata DM, Lewis R, Lin N, Kraft S, et al.
Closing the Quality Gap: A Critical Analysis of Quality Improvement
Strategies (Vol. 7: Care Coordination). Technical Review 9. AHRQ Publication
No. 04(07)-0051-7. Rockville, MD: Agency for Healthcare Research and
Quality: 2007. Available from: http://www.ncbi.nlm.nih.gov/books/
NBK44015/
41. National Quality Forum. Priority Setting for Healthcare Performance
Measurement: Addressing Performance Measure Gaps in Care Coordination.
2014.
42. Kenny DA, Kashy DA, Cook WL. Dyadic data analysis. New York, NY: Guilford
Press; 2006.
43. Kenny DA, Veldhuijzen W, Weijden T, LeBlanc A, Lockyer J, Légaré F, et al.
Interpersonal perception in the context of doctor–patient relationships: a
dyadic analysis of doctor–patient communication. Soc Sci Med.
2010;70(5):763–8.
44. Coman E. Proceedings of the CIPCI-TRIPP Roundtable. Hartford: Connecticut
Institute for Primary Care Innovation & Ethel Donaghue Center for
Translating Research into Practice and Policy; 2014.

Submit your next manuscript to BioMed Central

and take full advantage of:

• Convenient online submission

• Thorough peer review
• No space constraints or color figure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution

Submit your manuscript at

www.biomedcentral.com/submit