See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/254336688

Measuring Performance: An Introduction to Data Envelopment Analysis (DEA)

Article in The Journal of Cost Analysis · November 1998

DOI: 10.1080/08823871.1998.10462318

CITATIONS READS
443 9,457

1 author:

William F. Bowlin
North Dakota State University
27 PUBLICATIONS 1,170 CITATIONS

SEE PROFILE

All content following this page was uploaded by William F. Bowlin on 16 October 2014.

The user has requested enhancement of the downloaded file.

 Measuring Performance: An Introduction to Data
Envelopment Analysis (DEA)

WILLIAM F. BOWLIN
Department of Accounting, University of Northern Iowa, Cedar Falls, Ia. 50614-0127

Abstract. Data Envelopment Analyses (DEA) is a methodology that has been used
to evaluate the efficiency of entities (e.g., programs, organizations etc.) which are
responsible for utilizing resources to obtain outputs of interest. It has been used to
evaluate activities as varied as public schools, hospital surgical units, and real-property
maintenance for the U.S. Air Force. DEA is a fractional programming model that
can include multiple outputs and inputs without recourse to a priori weights (as in
index number approaches) and without requiring explicit specification of functional
relations between inputs and outputs (as in regression approaches). It computes a
scalar measure of efficiency and determines efficient levels of inputs and outputs for
the organizations under evaluation.

Keywords: Efficiency, Data envelopment analysis, Performance measurement

INTRODUCTION

Data envelopment analysis (DEA) was first introduced in the literature in 1978 (Charnes
et al. 1978). It is an empirically based methodology that eliminates the need for some of
the assumptions and limitations of traditional efficiency measurement approaches. It was
originally intended for use as a performance measurement tool for organizations that lacked
a profit motivation, e.g., not-for-profit and governmental organizations. However, since its
introduction, it has been developed and expanded for a variety of uses in for-profit as well
as not-for-profit situations.
Charnes et al. (1995), Seiford (1996), and other sources—e.g., internet world wide
web sites discussed later—provide good bibliographies of DEA which include applications
to hospitals, education, military, airlines, and other areas. Some of the applications related
to the military and the defense industry include a financial analysis of the defense industry
(Bowlin 1995); efficiency analysis of Air Force vehicle operations functions (Clarke 1992),
aircraft maintenance activities (Charnes et al. 1985), real-property maintenance activities
(Bowlin 1987), and accounting and finance functions (Bowlin 1989); performance of air-
craft maintenance units in the Israeli Air Force (Roll 1989); efficiency of Army recruiting
districts (Charnes et al. 1986); performance of Navy recruiting activities (Byrnes and Cooke
1988 and Morey 1991); performance of DoD commissary operations (Bowlin 1996); and

3
Bowlin

efficiency-based budgeting (Bowlin, Wallace, and Murphy 1989). Some of the applications
related to health care include prenatal care in England (Thanassoulis et al. 1995), the rela-
tionship between hospital efficiency and type of hospital ownership (Burgess and Wilson
1996), nursing home care in The Netherlands (Kooreman 1994), performance of the public
health service in the United Kingdom (Pina and Jones 1992), and an assessment of the
efficiency of New Jersey hospitals (Borden 1988).
In this issue of the Journal of Cost Analysis there are four new applications of DEA
in different environments. Ruggiero (1998) analyzes the cost efficiency of providing edu-
cational services for a sample of New York State school districts. Thanassoulis (1998) uses
DEA to set marginal costs for use in arriving at a local government’s entitlement to housing
subsidies in England. Kantor and Maital (1998) present research on the efficiency of bank
branches of a Mideast bank. The last article, Bowlin (1998), compares compensation for
defense-industry chief executive officers (CEOs) with non-defense industry CEOs.
This paper is intended as an introduction to DEA and proceeds as follows. The
next section is an introduction to the foundations of the basic DEA models. Details on
the Charnes, Cooper, and Rhodes (CCR) (1978) and Banker, Charnes, and Cooper (BCC)
(1984) models are provided along with a short discussion on some of the extensions of these
models. This is followed by a section which discusses some issues to be considered when
applying DEA. The fourth section covers means now available for testing hypotheses of
differences in performance, as represented by the DEA rating, between samples of organiza-
tions. The final section provides information on other sources of DEA related information.

DATA ENVELOPMENT ANALYSIS MODELS

Charnes, Cooper, Rhodes (CCR) Version

We begin by introducing the Charnes, Cooper, and Rhodes (CCR) (1978, 1979, 1981) ratio
form of DEA:
Ps
u r yr o
maximize: h o = Prm=1
i=1 vi x io
Ps
u r yr j
subject to: Prm=1 ≤ 1; j = 1, 2, . . . , n
i=1 vi x i j
ur
Pm > ε; r = 1, . . . , s (1)
i=1 vi x io
vi
Pm > ε; i = 1, . . . , m
i=1 vi x io

ε>0

4
 Measuring Performance

This model is designed to evaluate the relative performance of some decision

making unit (DMU), designated as DMUo , based on observed performance of j = 1, 2, . . . ,
n DMUs. A DMU is to be regarded as an entity responsible for converting inputs into
outputs.
The yr j , xi j > 0 in the model are constants which represent observed amounts of the
r th output and the i th input of the j th decision making unit which we shall refer to as DMU j
in a collection of j = 1, . . . , n entities which utilize these i = 1, . . . , m inputs and produce
these r = 1, . . . , s outputs. One of the j = i, . . . , n DMUs is singled out for evaluation,
accorded the designation DMUo , and placed in the functional to be maximized in (1) while
also leaving it in the constraints. It then follows that DMUo ’s maximum efficiency score
will be h ∗o ≤ 1 by virtue of the constraints.
The ε > 0 in (1) represents a non-archimedean constant which is smaller than any
positive valued real number. In practice, this non-archimedean concept is handled by the
DEA computer software used. Hence, it need not be specified explicitly. We therefore turn
to other topics as follow.
The numerator in (1) represents a set of desired outputs and the denominator repre-
sents a collection of resources used to obtain these outputs. This ratio results in a scalar
value similar to ratio forms often used in accounting and other types of analyses. The
value h ∗o obtained from this ratio satisfies 0 ≤ h ∗o ≤ 1 and can be interpreted as an
efficiency rating in which h ∗o = 1 represents full efficiency and h ∗o ≤ 1 means ineffi-
ciency is present. The star (*) indicates an optimal value obtained from solving the model.
Furthermore, h ∗o is invariant to the units of measure used for the input and output vari-
ables.
Also, note that no weights need to be specified a priori in order to obtain the scalar
measure of performance. The optimal values u r∗ , vi∗ may be interpreted as weights when
solutions are available from (1). But, they are determined in the solution of the model and not
a priori. To emphasize differences from more customary (a priori) weighting approaches,
the u r∗ , vi∗ values secured by solving the above problem are called P ∗ virtual multipliers and
interpreted in DEA so that they yield P ∗ a virtual output, Yo = u r yr o (summed over r =
1 . . . . s), and a virtual input, X o = vi xio (summed over i = 1 . . . . m), which can allow
us to compute the efficiency ratio h o = Yo / X o .
As can be observed from (1), this h ∗o is the highest rating that the data allow for a
DMU. No other choice of u r∗ and vi∗ can yield a higher h ∗o and satisfy the constraints. These
constraints make this a relative evaluation with:

Ps
u r yr j
Prm=1 =1
i=1 vi x i j

for some j as a condition of optimality.

5
Bowlin

Similar efficiency evaluations can be obtained for each of the j = 1, . . . , n DMUs

listed in the constraints of (1) by according them the same treatment, i.e., positioning them
in the functional as DMUo , one by one, while also leaving them in the constraints. Computer
codes for accomplishing this process are available and are presented in Table 1 later in this
paper.
These efficiency ratings are more than just index numbers which indicate a ranking
of DMUs based on their efficiency. The value of h ∗o has operational significance in that
1 − h ∗o provides an estimate of the inefficiency for each DMUo being evaluated. Carried
further, as shown below, this characterization makes it possible to identify the sources
and amounts of inefficiency in each input and output for every one of the DMUs being
evaluated.
It should be emphasized that the orientation of DEA is generally toward relative
efficiency as determined by the above optimization applied to the data. Thus, for any DMUo
being evaluated, the optimization implies that the evaluation will be effected by reference
to the subset of j = 1, . . . , n DMUs for which:
Ps
u ∗ yr k
1 = Prm=1 r∗ , kεK (2)
i=1 vi x ik

where the stars (*) indicate that these u r and vi values are optimal, and hence, make h o
maximal for DMUo , the DMU being evaluated. In addition, kεK indicates the subset of
DMUs that have attained the value of unity (an efficient DMU), which is the maximum
value allowed by the constraints. Note that these kεK DMUs have attained the value of
unity with the same u r∗ and vi∗ that are best for DMUo .
Using U ∗ , V ∗ to represent vectors with components u i∗ , vr∗ which are optimal for
DMUo in (1), we observe that such a choice will not give h ∗o = 1 unless DMUo is also in
the set kεK . If h ∗o < 1, then DMUo will be characterized as inefficient relative to the set of
DMUs in (2) which attain 100% efficiency with these same U ∗ , V ∗ values. In other words,
DMUo is rated relative to an efficient subset of DMUs with, in general, different efficient
subsets being utilized for the wanted efficiency ratings as different DMUs are brought into
the functional of (1) for evaluation.

Transformation to Linear Programming Form

The problem in (1) would be computationally intractable if addressed directly. Fortunately,

the theory of fractional programming, as first given in Charnes and Cooper (1962), makes it
possible to replace (1) with an equivalent linear programming problem. The transformation
to accomplish this can be found in Charnes, Cooper, and Rhodes (1978). Here we simply

6
 Measuring Performance

present the results of the transformation in the following form.

X
s
maximize: u r yr o
r =1

X
s X
m
subject to: u r yr j − vi xi j ≤ 0
r =1 i=1
(3)
X
m
vi xio = 1
i=1

−u r ≤ −ε
−vi ≤ −ε

The first set of j = 1, . . . , n constraints in (3) come from the less-than-or-equal-to-

unity requirements in (1) Pwhile u r , vi ≥ ε > 0, ∀r, i, come from the non-Archimedean
conditions in (1). Also, vi xio = 1 guarantees that it is possible to move from (3) to (1),
as well as from (1) to (3). Finally, the theory of fractional programming as given in Charnes
and Cooper (1962) insures that

X
s
h ∗o = u r∗ yr o (4)
r =1

where the stars indicate optimal values in (1) and (3), respectively.
The formulation in (1) makes it easy to relate DEA to the one output to one input ratio
measures of efficiency utilized in engineering and the natural sciences. Indeed, as shown in
Charnes, Cooper, and Rhodes (1978), the model in (1) generalizes the usual single output
to single input efficiency measures used in these disciplines in a way that accommodates
the case of multiple outputs and multiple inputs.
The formulation in (3) provides contact with economics. This is accomplished
by interpreting (3) so that the objective is to maximize virtual output (as defined pre-
viously) subject to unit virtual input while maintaining the condition that virtual output
cannot exceed virtual input for any DMU. As noted in Charnes et al. (1985), this implies
that the conditions for Pareto (or Pareto-Koopmans) optimality, as they are called in eco-
nomics, are fulfilled since further increases in this maximal value can be attained only
if some of the input values xi j are increased or if some of the output values yr j are de-
creased.

7
Bowlin

Since (3) is a linear programming problem, it has a dual which can be represented as:
" m #
X Xs
− +
minimize: θ−ε si + sr
i=1 r =1

X
n
subject to: 0 = θxio − xi j λ j − si− (5)
j =1

X
n
yr o = yr j λ j − sr+
j =1

0 ≤ λ j , si− , sr+ for i = 1, . . . , m; r = 1, . . . , s; j = 1, . . . , n

and θ is unrestricted in sign.

It is from (5) that the name data envelopment analysis is derived. Note that any
admissible choice of λ j provides an upper limit for the outputs and a lower limit for the
inputs of DMUo and against these limits θ is tightened with λ∗j , si−∗ , sr+∗ ≥ 0 representing
optimizing choices associated with minimize θ = θ ∗ . The collection of such solutions then
provides an upper bound which envelops all of the observations, and hence, leads to the
name Data Envelopment Analysis.
Recalling that xio , yr o are represented in the constraints as well as the functional in (1),
it is clear that (5) always has at least the solution θ = 1, λo = 1 and all other λ j , si−∗ , sr+∗ = 0
when DMUo is the DMU under evaluation. It follows that an optimum will be attained with
0 ≤ θ ∗ ≤ 1. Because (5) has a finite optimum, the duality theory of linear programming gives:
à m !
X Xs X s
∗ ∗ −∗ +∗
ho = θ − ε si + sr = u r∗ yr o . (6)
i=1 r =1 r =1

Note that θ ∗ = 1 does not imply h ∗o = 1 unless also sr+∗ , si−∗ = 0 for all r and i. That
is, all slack must also be zero in (6). Conversely, sr+∗ , si−∗ = 0 for all r and i does not imply
h ∗o = 1 unless θ ∗ = 1 also. In other words, it is necessary to have both θ ∗ = 1 and zero slack
for efficiency or, conversely, h ∗o = 1 implies θ ∗ = 1 and all slack equal to zero in an optimum
solution to (6) in order for DMUo to be characterized as fully (100%) efficient via DEA.
Therefore, h ∗o = 1 if and only if DMUo is efficient.

Banker, Charnes, and Cooper (BCC) Version

Another version of DEA that is in common use is the Banker, Charnes, and Cooper (BCC)
(1984) model. The primary difference between this model and the CCR model is the

8
 Measuring Performance

treatment of returns to scale. The CCR version bases the evaluation on constant returns
to scale. The BCC version is more flexible and allows variable returns to scale.
Following is the equivalent of (5) for the BCC formulation.
" #
X
m X
s
minimize: θ−ε si− + sr+
i=1 r =1

X
n
subject to: 0 = θxio − xi j λ j − si−
j =1

X
n
yr o = yr j λ j − sr+ (7)
j =1

X
1= λj

0 ≤ λ j , si− , sr+ for i = 1, . . . , m; r = 1, . . . , s; j = 1, . . . , n

The difference between the CCR model (model (5)) and the BCC model (model (7)) is that
the λ j s are now restricted to summing to one. This has the effect of removing the constraint
in the CCR model that DMUs must be scale efficient. Consequently, the BCC model allows
variable returns to scale and measures only technical efficiency for each DMU. That is, for
a DMU to be considered as CCR efficient, it must be both scale and technical efficient. For
a DMU to be considered BCC efficient, it only need be technically efficient.
The separate evaluation of returns to scale in the BCC model is more evident in the
dual to (7) which we write as follows.

X
s
maximize: u r yr o − u o
r =1

X
s X
m
subject to: u r yr j − vi xi j − u o ≤ 0
r =1 i=1
(8)
X
m
vi xio = 1
i=1

− u r ≤ −ε
− vi ≤ −ε

9
Bowlin

In this model, the u ∗o (the ∗ indicates an optimal value determined via model (8)) indicates the
return to scale possibilities. An u ∗o < 0 implies local increasing returns to scale. If u ∗o = 0, this
implies local constant returns to scale. Finally, an u ∗o > 0 implies local decreasing returns
to scale. Note that the CCR model previously discussed simultaneously evaluates both
technical and scale efficiency in the aggregate. The BCC model, however, separates the two
types of inefficiencies in order to evaluate only technical inefficiencies in the envelopment
model (model 7) and scale inefficiencies in the dual to (7) (model (8)).

Illustration

Figure 1 is a simple single output, single input illustration of the CCR and BCC versions
of the DEA model. In Figure 1, the y axis is the values for an output, and the x axis is
the values for an input. Points P1, P2, P3, P4, and P5 represent observed performance of
organizations. The numbers in parentheses following the point designation are the x and y
values for the point.
The solid line, BCC, connecting P1, P2, P3, and P4 represents the frontier developed
using the BCC DEA model. These organizations had the best observed ratio of outputs to
inputs. In the BCC version, P1, P2, P3, and P4 would be considered efficient and receive an

Figure 1.

10
 Measuring Performance

efficiency rating of one (1). These organizations are considered to be technically efficient.
However, as can be seen, this frontier reflects differing returns to scale. The segment P1-P2
reflects locally increasing returns to scale, i.e., an increase in the inputs would result in a
greater than proportionate increase in output. That is, organization P1 is technically efficient
and scale inefficient (not operating at constant returns to scale). P2 represents frontier
operations at constant returns to scale and is, therefore, both technically and scale efficient.
Segments P2-P3 and P3-P4 reflect locally decreasing returns to scale, i.e., an increase in
inputs would result in a less than proportionate increase in output. Organizations P3 and P4
would also be technically efficient but scale inefficient.
The ray (dashed line) extending from the origin through P2 represents the efficiency
frontier as determined by the CCR model. It reflects constant returns to scale. In the CCR
version of the model, only P2 would be rated efficient since it is the only organization
operating at constant returns to scale.
Differences between the CCR and BCC models can be further illustrated using orga-
nization (point) P5 in Figure 1. Using the BCC model, P5’s rating would be based on its
distance from P5BCC , a linear combination of P1 and P2. P5 would receive an efficiency
rating of .47, indicating that the organization’s input should be reduced by 53% to 2.333 in
order for it to be considered efficient.
Based on the CCR model, P5’s rating would be determined by its distance from P5CCR .
P5 would receive an efficiency rating of .36 which indicates the organization would need
to reduce its input by 64% to 1.8 in order to become efficient (both scale and technically
efficient). Note that the rating provided via the BCC model will always be higher than the
one provided by the CCR model (except in the case where an organization is rated efficient
by both the CCR and BCC versions of the model as would be the case with P2).
Also, in the CCR case, organization P5 is moved to a frontier which has constant
returns to scale. This is what Banker (1984) refers to as most productive scale size (MPSS).
In Figure 1, this occurs at P2 (constant returns to scale). However, more generally MPSS
will be a segment where the efficiency frontiers of the CCR and BCC models are tangential.

Validation

DEA has been validated through a variety of means such as observation, simulations, and
hypothetical data sets with known efficiencies and inefficiencies. Below are summaries of
a sample of the studies that have assisted in validating DEA.
Bowlin et al. (1985) developed a hypothetical data set for hospital units with known
efficiencies and inefficiencies. They used this data set to test DEA against ratio and regres-
sion analyses. They found that DEA outperformed both ratio analysis and least-squares
regression in identifying sources and amounts of inefficiencies. Moreover, the performance
of the regression approaches worsened when confidence intervals were used to discriminate

11
Bowlin

between efficient and inefficient behavior, and the inefficient behavior was identified with
observations beyond a piece-wise confidence limit.
Banker et al. (1988) completed a simulation study comparing DEA estimates for a
known production frontier with estimates from a translog regression. Based on results for
randomly generated observations from a known piece-wise log-linear technology, they con-
cluded that the DEA model outperformed the translog method in estimating inefficiencies
and returns to scale. This occurred even though the translog regression is supposedly flexi-
ble enough to reflect nonlinear functions such as the ones used to generate the observations
in their study.
Finally, Banker, Conrad, and Strauss (1986) used empirical data from a sample of
North Carolina hospitals to compare efficiency characterizations obtained from DEA and
econometric models. The DEA model was able to identify inefficiencies and uncover return
to scale possibilities in individual hospitals that were not evident in the translog model. In
addition, Banker, Conrad, and Strauss reported that DEA’s efficiency estimates appeared to
be more closely related to the degree of capacity utilization than were the translog estimates.

Comparison to Regression Analysis

In the past, regression approaches have been commonly used for measuring efficiency.
Consequently, in this section we present some of the differences between regression and
DEA in order to highlight DEA’s characteristics.
Unlike traditional regression approaches, DEA does not require explicit specification
of the functional forms relating inputs to outputs. More than one function (e.g., more than
one production function) is admitted, and the DEA solution can be interpreted as providing a
local approximation to whatever function is applicable in the neighborhood of the coordinate
values formed from the outputs and inputs of the DMUo being evaluated. Thus, DEA is
more flexible in recognizing differences in production functions between DMUs.
Second, DEA is oriented toward individual decision making units which are regarded
as responsible for utilizing inputs to produce the outputs of interest. It therefore utilizes
n optimizations, one for each DMU, in place of the single optimization that is usually
associated with the regressions used in traditional efficiency analyses. Hence, the DEA
solution is unique for each DMU under evaluation.
Third, a deficiency of all of the regression approaches is their inability to identify
sources and estimate the inefficiency amounts associated with these sources. Hence, no
clue as to corrective action is provided even when the dependent variable shows that ineffi-
ciencies are present. DEA provides both the sources (input and output) and amounts of any
inefficiencies. See Chaffai (1997) for additional detailed discussion and further reference.
Fourth, as discussed earlier, the coefficients or weights developed via DEA are unique
to the individual organization under evaluation. They are not like regression coefficients

12
 Measuring Performance

which are applicable to a class or group of organizations. Thus, DEA does not provide a
model for predicting the performance of an organization for years that are not included in
the evaluation as might be possible with regression. Nor can DEA be used, as regression
can, to prepare a model to be extended outside the data base used to develop the model.
Consequently, regression analysis may be more appropriate if the analyst is to use the results
to estimate or predict future performance for a class of entities instead of an individual entity
or if the results of the analysis are to be used to predict performance outside the relevant
range of data.
Turning in another direction, we may identify other advantages of the regression ap-
proaches such as the following. No statistical significance can be determined and attached to
the DEA weights as can be done with regression coefficients. Also, there is no consideration
of random error or an “e” term in the model as there is in regression. Thus, DEA may tend
to confuse random fluctuations with inefficiencies represented in the data.
Furthermore, the use of regressions may be advantageous simply because this ap-
proach is more familiar and thus, better understood or accepted. In this case, however, it
may also be prudent to consider the possibility of accompanying methodological biases
which can affect the outcome of an analysis along lines like those indicated in Charnes,
Cooper, and Sueyoshi (1986).
Finally, we turn to ways in which the two methodologies–regression and DEA—may
be used in combination. One possibility is to use the two approaches to cross-check results.
This can provide protection against what Charnes, Cooper, and Sueyoshi (1986) refer to as
methodological bias. For an application to the analysis of financial institutions, see Fried,
Lovell, and Van den Eeckaut (1993).
In yet another combining approach, DEA and statistical regressions are joined together
in a two-stage complementary manner as follows. Stage one uses DEA to identify efficient
and inefficient observations. Stage two then uses this information by incorporating it in the
form of dummy (i.e., classification) variables in regression formulations using the same
data for an analysis. For an application of this approach to secondary schools see Arnold
et al. (1996). A subsequent simulation study using this approach which also gave very
satisfactory results is reported in Bardhan, Cooper, and Kumbhakar (1997).

Other Model Variations

Since the initial development of DEA by Charnes, Cooper, and Rhodes in 1978, there
have been several variations of the model developed in response to new and varied needs.
The intent here is to describe some of these models and what they were designed to do.
Additional details on the models can be found in the references. Also, see Ahn, Charnes, and
Cooper (1988) for analyses that relate results for these different DEA models in a rigorous
mathematical manner.

13
Bowlin

Additive Model. The concepts of the additive model were first introduced in Charnes
et al. (1985) and further elaborated on in Banker et al. (1989). The additive model is
formulated as follows:
X
m X
s
maximize: si− + sr+
i=1 r =1

X
n
subject to: 0 = xio − xi j λ j − si−
j =1

X
n
yr o = yr j λ j − sr+ (9)
j =1
X
1= λj

0 ≤ λ j , si− , sr+ for i = 1, . . . , m; r = 1, . . . , s; j = 1, . . . , n.

This formulation corresponds to the BCC envelopment version of DEA. Another

version of the additive model,P which corresponds to the CCR formulation in model (5),
can be created by omitting the λ j condition in (9). The difference between this additive
model and the BCC version is that the θ (proportional amount of inefficiency) is dropped
from the model and all inefficiencies are captured in the slack values (sr+ and si− ). Thus, the
only test for efficiency is whether all slacks are zero.
A DMU which is efficient for the BCC model will also be efficient for the additive
model and vice versa (Ahn et al. 1988). However, when a DMU is inefficient, the sources
and amounts of inefficiency may differ for the two versions of the DEA model because of
different metrics used in accomplishing the efficiency evaluation. Returning to Figure 1,
the additive model would evaluate P5 relative to P5BCC for the version which corresponds
to the BCC model. The amounts of inefficiency indicated by the additive model would be
2.667 input units and zero output units.

Multiplicative Model. The models previously discussed (e.g., CCR, BCC, and addi-
tive) prescribed an additive combination of outputs and inputs. Charnes et al. (1982, 1983)
present a multiplicative combinational method. The primary difference between this model
and the other models is that the virtual outputs and P
virtual inputs are formed multiplicatively
instead ofQadditively. That is, the summation sign ( ) in the equations is replaced by a prod-
uct sign ( ). The Y and X vectors of outputs and inputs are logarithms. The result is that
the frontier is piece-wise log-linear (Cobb-Douglas type) rather than piece-wise linear as is
the case for the CCR, BCC, and other models. Like the additive model, the multiplicative

14
 Measuring Performance

model identifies inefficiencies only through slack values and no intensity or proportional
variable (i.e., θ) is involved.

Measures of Efficiency Dominance Model. The CCR, BCC, and other DEA models
allow the comparison of an organization (not on the frontier) to a linear combination of
entities that are on the frontier. For example, in Figure 1, P5’s BCC efficiency valuation is
based on a non-negative, linear combination of P1 and P2. In some cases, however, it may
be desirable to restrict comparisons to actual organizations instead of a linear combination
of organizations. As shown in Bardhan et al. (1996), one way of doing this is to modify the
continuity assumption in the BCC and CCR models. The result is a bi-valency variant of the
DEA additive model which Bardhan et al. refer to as a Measure of Efficiency Dominance
(MED). In Figure 1, P5’s efficiency would be based solely on P2 with amounts of inefficiency
of being two input units and two output units.

Assurance Region and Polyhedral Cone-Ratio Models. Charnes et al. (1990) point
out that the construction of an empirical production function may be flawed. It may inad-
equately represent what is actually possible, or the results can reflect efficient input and
output values that DMU managers might consider to be outside the range of realistic pos-
sibilities. To overcome this problem, two models have been developed which restrict the
values that the virtual weights may attain and thereby, limit the range of acceptable efficient
input and output levels.
One approach is the assurance region approach which was first presented in Thompson
et al. (1986) and further defined in Thompson et al. (1990). The assurance region model
provides lower and upper bounds on the admissible values of variables. These bounds take
the following form.

αr ≤ vr /vr o ≤ βr , r = 1, . . . , s
(10)
δi ≤ u i /u io ≤ γi , i = 1, . . . , m

Here, vr o and u io represent dual variables which serve to establish the upper and lower
bounds represented by αr , βr and by δi , γi for the dual variables associated with each
output and input and αr o , βr o , δio , γio = 1.
Assurance regions are a special case of polyhedral cone-ratios as described in Charnes
et al. (1990). The polyhedral cone-ratio variation of the DEA model also restricts the
ranges of relative valuation of inputs or outputs based on the preferences of managers
about the relative importance of different factors and what determines best practice. The
virtual weights are controlled so as to favor desired patterns of input usage and/or output
production, and thereby, make possible the use and incorporation of expert knowledge in
the evaluation.

15
Bowlin

Non-discretionary Variable Model. The DEA model provides information about

the extent to which an input should be decreased or an output augmented by an inefficient
organization in order for it to become efficient. However, managers sometimes deal with
non-discretionary variables—variables that have values determined by forces exogenous to
the organization under evaluation. In other words, the values of these inputs or outputs are
beyond the control of the organization’s management. Consequently, management or an
analyst might want to treat these types of inputs and outputs differently in a DEA analysis
of the performance of an organization because they are outside the discretionary control of
the managers, and the managers can take no action to affect these inputs and outputs.
The amount a non-discretionary, fixed input variable can be reduced can be determined
but may not be meaningful because the input cannot be reduced. However, in many cases
it is important to take these non-discretionary amounts into account in an evaluation of
an organization. For instance, in an evaluation of Air Force flight operations, it may be
important to take weather (a non-discretionary variable) into account since it would allow
for advantages and disadvantages of airfield and flight operations in vastly different weather
conditions. See Charnes et al. (1985). Banker and Morey (1986b) provide a modification to
the basic DEA model that permits the DEA solution to indicate the amount a controllable
input can be reduced while keeping the non-controllable input fixed at its current level.

Categorical Variable Model. Banker and Morey (1986a) offer an adaptation to the
DEA model that permits the inclusion of variables that are categorical. This extension of
DEA relaxes the need for the variables to be measured on a continuous scale and allows the
incorporation of on-off or present-not present variables in the analysis. For example, in a
study of fast-food restaurants by Banker and Morey (1986a), restaurants were evaluated by
categorizing them according to whether they had drive-through windows so that customers
did not have to leave their cars. Kamakura (1988) provides a further modification which
allows comparing organizations at multiple levels.

IMPLEMENTATION OF DEA

This is as far as the discussion on the foundations of DEA is carried in this paper. In this
section, we present some issues which are important in using DEA.

Model Specification

Positivity Property. Generally, the DEA formulation requires that the input and
output variables be positive (greater than zero). Charnes, Cooper, and Thrall (1991) have
shown how this restriction can be relaxed, but in practice this approach has generally not
been implemented, and therefore, is not discussed here.

16
 Measuring Performance

If a variable is not positive, there are two possible treatments. First, Ali and Seiford
(1990) and Pastor (1996) have shown that an affine displacement does not alter the efficient
frontier, and hence, certain DEA formulations (e.g., the additive model for both inputs and
outputs and the BCC model for outputs) are translation invariant. Consequently, absolute
constants can be added to any input and output in the additive model and any output in the
BCC model in order to solve the non-positivity problem. In this approach, a positive amount
is added to the negative value so that the value of that particular input or output variable
becomes positive. This same adjustment must be made to the same input or output value for
all decision making units included in the data set in order to not alter the efficiency frontier.
For example, assume that one of the output variables in a data set is net income, and
one of the organizations included in the data set had a net income (loss) of −$100. An
appropriate treatment would be to increase the net income value for all organizations in the
data set by $101. All data would then be positive. Since the solution is not affected by this
change, the solution to this adjusted problem is the same as the solution to the problem with
the negative data.
Note that the key to translation invariance in DEA lies with the convexity constraint.
Thus, the CCR models are not translation invariant.
A second approach that might be considered is to substitute a very small positive value
for the negative value if the variable is an output. This approach is suggested based on the
characteristic that the DEA model puts each DMU in the best light possible and therefore,
emphasizes (weights highest) those outputs on which the DMU performs best. Because
of this characteristic, an output variable with a very small value would not be expected to
contribute to a high efficiency rating which would also be true of a negative net income
value. Thus, the theta value would generally not be inappropriately affected by this type of
translation.

Isotonicity Property. As shown in Charnes et al. (1985), it is required that the func-
tions relating inputs to outputs have the mathematical property called isotonicity. Referring
back to (1), this means that an increase in any input should result in some output increase
and not a decrease in any output.
Some analysts have used correlation analysis to determine if this isotonicity property
exists between the selected input and output variables. If the input variable coefficient
obtained from the correlation analysis is positive and significant, then there is support that
the isotonicity assumption is not violated. However, there is the danger that a correlation
analysis will not indicate the presence of the isotonicity property because of inefficiencies
reflected in the data. Consequently, others have relied on the assumption that this relationship
logically should exist between the inputs and outputs.
If it is apparent that the isotonicity property is violated, the isotonicity requirement
may be accommodated by using reciprocals, complements, etc. For example, a particular
output, such as the number of defective items, may be expected to decrease with an increase

17
Bowlin

in inputs. The isotonicity property would be violated because it is desirable and expected that
an increase in an input would result in an increase in an output. This would not be the case
with the defective items since an increase in output would be expected to result in a decrease
in the number of defective items. In this case, an analyst might want to use the reciprocal of
the number of defective items as the output measure with the resulting relationship being
one where the output value would be expected to increase as the inputs increased.

Number of Decision Making Units. A general rule of thumb is that three decision
making units are needed for each input and output variable used in the model in order to
insure sufficient degrees of freedom for a meaningful analysis. If less than three DMUs per
input and output variable are included in the data set, there is a danger that an excessive
number of the DMUs will be considered efficient (receive a rating of one) because of an
inadequate number of degrees of freedom. In this case, there are relatively few DMUs in
the data set, and the probability is improved that the model will find a single variable or
combination of variables at which an organization is the best and therefore, award that
organization a rating of efficient.
One way to obtain additional DMUs is through disaggregating data, e.g., breaking
annual data into quarterly data. In this case, each quarterly operation would then be consid-
ered a separate DMU. Thus, if there were originally seven entities, there would now be 28
DMUs available.

Window Analysis. The data disaggregation presented above also lends itself to what
is commonly referred to as window analysis. A window analysis consists of a series of
analyses with time-dependent DMUs that change for each analysis to reflect a sort of
moving average approach. For example, Analysis #1 could consist of data for first, second,
and third quarter operations. Analysis #2 could consist of second, third, and fourth quarter
operations. Analysis #3 could include third and fourth quarter data from one year and first
quarter data for the next year. Thus, each analysis has a new and different set of DMUs
resulting in differing efficiency ratings.
Using quarterly data in this manner not only increases the number of DMUs, but also
creates a window for studying the stability of the efficiency ratings for each of the DMUs as
the collection of DMUs is altered. Other information is also available which can be studied
for trends and/or other types of time-dependent behaviors. Bowlin (1987) illustrates such a
window analysis procedure.

Control of Weights. The weights, u r and vi , are determined by solving the DEA
model. These weights are computed such that the organization under evaluation is placed
in the best light possible relative to the other units in the data set. The weights developed
via DEA may not represent the same relative subjective weights that management might
apply as to the relative importance of the variables (especially the output variables) used in

18
 Measuring Performance

the DEA model. Consequently, a particular input or output could receive an inappropriate
weight. That is, a particular output which the organization produces a significant amount
of might receive most of the weight while other outputs on which the unit doesn’t do quite
as well might receive no or very little weight. Likewise, the model may not provide any
or very little weight to a particular input even though the unit is using more of it than are
other units. Management may find this weighting scheme unacceptable and want to ensure
that no particular output or input is ignored by the model. The concepts of assurance region
developed by Thompson et al. (1990) and cone ratio developed by Charnes et al. (1990)
that were briefly discussed before can be used to modify the DEA model so as to constrain
the multipliers (u r and vi ) if they are judged to be unreasonable.

Homogeneity of DMUs. DEA requires a relatively homogeneous set of entities.

That is all entities included in the evaluation set should have the same inputs and outputs
in positive amounts.

Specification of Input and Output Measures

Specification of input and output variables is a key consideration in using DEA. Choosing
correct inputs and outputs is important for the effective interpretation, use, and acceptance
of the results of the DEA analysis by management or other affected parties. The following
guidelines might be useful in identifying appropriate input and output variables.
First, recall the previous discussion on DEA’s positivity and isotonicity conditions.
There should be some basis for believing that relationships exist between inputs and outputs
such that an increase in an input can reasonably be expected to increase one or more of the
outputs. In addition, all input and output measures should be regularly reported by each
organization and available in positive amounts in every period for each DMU.
Another consideration is whether the variables should be based on currently available
data or new measures developed. It is generally desirable to stay close to the kinds of input
and output measures currently used by management for performance evaluation. Manage-
ment is already familiar with these measures and has accepted them as being informative.
In addition, data on these variables are already being collected and therefore, new data
collection systems are not needed.
Management should be involved in the selection of inputs and outputs. Discuss with
management what they believe are important outputs and inputs. The omission of pertinent
variables can limit the managerial usefulness of the DEA analysis. Also, remember that the
inputs and outputs do not have to be reduced to a common unit of measure.
Third, the inputs and outputs should be comprehensive. That is, they should fully
measure the activities of the organization under evaluation and should also be operationally
meaningful in the sense that they should be commonly used, and hence, familiar to officials
concerned with the evaluation and control of these activities.

19
Bowlin

Finally, the values of the variables should be controlled (e.g., by audit and review
processes) so that they cannot be easily manipulated or carelessly reported without some
significant chance of detection and correction. DEA results and the interpretation of these
results can be significantly affected by missing data or misreported data.

Solving the Model

DEA Software. There is software available specifically for solving the DEA model.
The author is aware of three such software packages. Table 1 lists the name of the software
and an address for obtaining information on the software. This listing is presented here
solely for the convenience of the reader and is not intended as an endorsement of any of
these software packages.

TABLE 1. DEA software packages

1. Name: IDEAS (Integrated Data Envelopment Analysis System)

Address: 1 Consulting, Inc.
P.O. Box 2453
Amherst, MA 01004-2453

2. Name: Frontier Analyst

Address: Banxia Software Ltd.
141 St James Road
Glasgow G4 0LT
United Kingdom

Email: info@banxia.scotnet.co.uk
Web Site: http://www.scotnet.co.uk/banxia

3. Name: Warwick-DEA
Address: Professor Emmanuel Thanassoulis
Operations Research and Systems
Warwick Business School
The University of Warwick
Coventry CV4 7AL
United Kingdom

Telephone: 44-1203-523523 ext. 22145

Fax: 44-1203-523179
Email: orset@wbs.warwick.ac.uk

20
 Measuring Performance

Linear Programming. The DEA problem can also be solved with linear program-
ming software if appropriate DEA software is not available. Formulations (5) (CCR ver-
sion), (7) (BCC version), and (9) (additive version) are linear programming formulations
and should be used if the DEA model is to be solved via linear programming. If linear pro-
gramming software is used, a computer run would need to be made for each organization
in the data set. So, if there are 500 entities in the data set, then 500 computer runs would be
necessary. However, only the objective function needs to be changed for each run to reflect
the organization under evaluation. The constraints remain unchanged. In addition, if an
organization is identified in the reference set for a DMU under evaluation (e.g., kεK from
(2)), then the referenced DMU must be efficient (i.e., on the efficiency frontier). Therefore,
a DEA linear programming run would be unnecessary for this unit. Thus, an analyst can
reduce the number of DEA runs required to less than the number in the data set.

HYPOTHESIS TESTING

At times, an analyst may want to test the hypothesis that the DEA efficiency ratings of two
groups of DMUs are different in order to determine if one group of organizations is more
efficient than another group. See, for example, Bowlin (1989, 1995). In this section, we
present a means of hypothesis testing proposed by Banker (1989, 1993). He has proposed
test statistics for determining whether the difference in DEA ratings for two groups of
DMUs (N1 and N2 ) is statistically significant. Different test statistics are used depending on
the assumption that is made regarding the distribution of the inefficiency rating (1/ h ∗o ). Two
assumptions commonly used with stochastic frontier estimation and followed by Banker for
DEA are that inefficiency ratings follow either an exponential or half-normal distribution.
If the distribution of the inefficiency rating is assumed to be exponential with means
1+σ1 for group N1 and 1+σ2 for group N2 , then the null hypothesis for no difference in the
inefficiency ratings of the two groups is given by Ho : σ1 = σ2 . The alternative hypothesis is
Ha : σ1 > σ2 if it is believed that group N1 has a lower average efficiency rating than group
N2 . That is, group N1 is less efficient than group N2 . In this case, the test statistic (TSe ) is
given by:
" #," #
X¡ ± ¢± X¡ ± ¢±
TSe = 1 h ∗j − 1 n 1 1 h ∗j − 1 n 2 (11)
jεN1 jεN2

where:

h ∗j = The DEA efficiency rating computed via models (3), (5), or (7) for each
organization included in the data set.

21
Bowlin

N1 = The organizations included in Group 1.

N2 = The organizations included in Group 2.
n 1 = The number of organizations included in Group 1.
n 2 = The number of organizations included in Group 2.

TSe follows an F-distribution with 2n 1 and 2n 2 degrees of freedom respectively.

If it is assumed the inefficiency ratings follow a half-normal distribution with param-
eters σ1 and σ2 , then the test statistic (TShn ) is:
" #," #
X¡ ± ¢2 ± X¡ ± ¢2 ±
TShn = 1 h ∗j − 1 n 1 1 h ∗j − 1 n 2 (12)
jεN1 jεN2

In this case, the test statistic follows an F-distribution with n 1 and n 2 degrees of freedom.
Note that since DEA provides a relative measure of efficiency that is dependent upon
the DMUs included in the analysis data set, the inefficiency ratings must come from a DEA
analysis that includes all of the organizations in groups N1 and N2 . Also, Banker (1993)
notes that the tests in (11) and (12) are applicable only with large samples (undefined
by Banker but assumed to be defined as in statistical analysis). That is, the total sample
(n 1 + n 2 ) must be large. However, each subgroup (n 1 or n 2 ) need not be large. Finally, if an
analyst does not want to assume a probability distribution for the DEA inefficiency ratings,
then Kolmogrov-Smirnov or other non-parametric tests could be used to determine if there
is a statistically significant difference in the ratings of two samples.

OTHER ISSUES

Presentation of Results

Converting DEA results into information that management and others can readily understand
and use can be troublesome. The DEA literature offers a few articles that specifically
address presentation issues. Desai and Walters (1991) provide a parallel axis approach to
representing the multiple variables used in DEA and how this approach can be used for
displaying the results of DEA. Mahgary and Lahdelma (1995) also offer suggestions for
visualizing DEA results using two-dimensional charts.

Web Sites and Bibliographies

There are a few internet sites that interested individuals can access that provide additional
information on DEA. Table 2 provides the site addresses.

22
 Measuring Performance

TABLE 2. DEA information sources

Emrouznejad’s DEA Home Page

http://www.csv.warwick.ac.uk/˜bsrlu/

Anderson’s DEA WWW Page

http://www.emp.pdx.edu/dea/homedea.html/

Productivity Analysis Research Network (PARN)

Web Page: http://www.busieco.ou.dk/parn/

Email: moa@busieco.ou.dk

One site is maintained by Ali Emrouznejad at Warwick University, United Kingdom.

His DEA Home Page is comprehensive and includes a description of various DEA models,
an extensive bibliography of DEA research, frequently asked questions and discussion
sections, and a DEA tutorial.
Another web site on DEA is maintained by Tim Anderson at Portland State University,
Oregon. It also provides a basic introduction to DEA, a bibliography, and a link to other
world wide web DEA resources.
Finally, another source of DEA information that might be of interest to DEA re-
searchers is the Productivity Analysis Research Network (PARN). PARN was established
to provide an avenue for cooperation and information exchange among researchers involved
with productivity analysis (including DEA). PARN has established two sites with test data,
information, and news on productivity analysis. PARN also maintains a library of produc-
tivity analysis articles that are available to be borrowed. Membership in PARN is required
for access to the PARN sites and library. There is a free membership available.

CONCLUSION

Data envelopment analysis (DEA) first appeared in the literature in 1978 with the CCR
model (Charnes, Cooper, and Rhodes 1978). Since then the methodology has been refined
and further developed to become recognized as an effective tool for measuring performance.
DEA has been applied in a variety of ways and contexts. The Emrouznejad DEA Home
Page (see Table 2) lists over 1,000 references on DEA.
In this issue of the Journal of Cost Analysis, we introduce the readers to the foundations
of the methodology, some of the aspects of the model that an analyst should consider when
using DEA, references for additional information on DEA, and examples of how DEA

23
Bowlin

can be applied. DEA differs from other performance evaluation methodologies in that
DEA can handle multiple inputs and multiple outputs simultaneously without resorting to
some a priori specification of the functional relationship between the inputs and outputs
or subjective weighting scheme. Consequently, the main attention of the analyst can be
directed towards choices of inputs and outputs, how the inputs and outputs are measured,
and the choices of DMUs that are regarded as responsible for converting inputs into outputs.
Additional research relative to applying DEA is needed. Field studies on how man-
agers actually use DEA information are needed in order to extend DEA’s contribution to the
management of real-world problems. Another area of potential research includes studies
on how DEA might be used in conjunction with other methodologies such as regression
analysis or cluster analysis in order to take advantage of each methodology’s specific char-
acteristics in identifying and solving a problem. This is just a sampling of potential research
opportunities relative to DEA.

ACKNOWLEDGEMENT

I would like to thank Professor William W. Cooper, one of the original developers of data
envelopment analysis, for his helpful comments and suggestions in preparing this article.
However, any errors are solely my responsibility.

REFERENCES

Ahn, T.; Charnes, A.; and Cooper, W. W. (1988) “Efficiency Characterizations in Different DEA
Models,” Socio-Economic Planning Sciences 22(6), (pp. 253–257).
Ali, A. I. and Seiford, L. M. (1990) “Translation Invariance in Data Envelopment Analysis,” Opera-
tions Research Letters 9, (pp. 403–405).
Arnold, V. L.; Bardhan, I. R.; Cooper, W. W.; and Kumbhakar, S. C. (1996) “New Uses of DEA
and Statistical Regressions for Efficiency Evaluation and Estimation—with an Application to
Public Secondary Schools in Texas,” Annals of Operations Research 66.
Banker, R. D. (1984) “Estimating Most Productive Scale Size Using Data Envelopment Analysis,”
European Journal of Operational Research 17, (pp. 35–44).
Banker, R. D. (1989) “Econometric Estimation and Data Envelopment Analysis,” in J. L. Chan and
J. M. Patton, eds. Research in Governmental and Nonprofit Accounting 5, (pp. 231–243).
Banker, R. D. (1993) “Maximum Likelihood, Consistency and Data Envelopment Analysis: A Sta-
tistical Foundation,” Management Science 39(10), (pp. 1265–1273).
Banker, R. D.; Charnes, A.; and Cooper, W. W. (1984) “Models for Estimating Technical and Scale
Efficiencies in Data Envelopment Analysis,” Management Science 30(9).
Banker, R. D.; Charnes, A.; Cooper, W. W.; and Maindiratta, A. (1988) “A Comparison of DEA and
Translog Estimates of Production Frontiers,” in A. Dogramaci and R. Fare, eds. Applications
of Modern Production Theory: Efficiency and Productivity (Norwell, Mass: Kluwer Academic
Publishers, Inc.).

24
 Measuring Performance

Banker, R. D.; Charnes, A.; Cooper, W. W.; Swarts, J.; and Thomas, D. A. (1989) “An Introduction
to Data Envelopment Analysis with Some of Its Models and Their Uses,” in J. L. Chan and
J. M. Patton, eds. Research in Governmental and Nonprofit Accounting 5, (pp. 125–163).
Banker, R. D.; Conrad, R. F.; and Strauss, R. P. (1986) “A Comparative Application of Data En-
velopment Analysis and Translog Methods: An Illustrative Study of Hospital Production,”
Management Science 32(1), (pp. 30–44).
Banker, R. D. and Morey, R. C. (1986a) “The Use of Categorical Variables in Data Envelopment
Analysis,” Management Science 32(12), (pp. 1613–1627).
Banker, R. D. and Morey, R. C. (1986b) “Efficiency Analysis for Exogenously Fixed Inputs and
Outputs,” Operations Research 34(4), (pp. 513–521).
Bardhan, I.; Bowlin, W. F.; Cooper, W. W.; and Sueyoshi, T (1996) “Models and Measures for
Efficiency Dominance in DEA, Part I: Additive Models and MED Measures,” Journal of the
Operations Research Society of Japan 39(3), (pp. 322–332).
Bardhan, I.; Cooper, W. W.; and Kumbhakar, S. C. (1997) “A Simulation Study of Joint Uses of DEA
and Statistical Regressions for Efficiency Evaluation,” Journal of Productivity Analysis.
Borden, J. P. (1988) “An Assessment of the Impact of Diagnostic Related Group (DRG)-Based
Reimbursement on the Technical Efficiency of New Jersey Hospitals Using Data Envelopment
Analysis,” Journal of Accounting and Public Policy 7(2), (pp.77–96).
Bowlin, W. F. (1987) “Evaluating the Efficiency of US Air Force Real-Property Maintenance Activi-
ties,” Journal of Operational Research Society 38(2), (pp. 127–135).
Bowlin, W. F. (1989) “An Intertemporal Assessment of the Efficiency of Air Force Accounting
and Finance Offices,” Research in Governmental and Nonprofit Accounting 5, (pp. 293–
310).
Bowlin, W. F. (1995) “A Characterization of the Financial Condition of the United States’ Aerospace-
Defense Industrial Base,” Omega, International Journal of Management Science 23(5), (pp.
539–555).
Bowlin, W. F. (1996) “Benchmarking and Efficient Program Delivery for the Department of Defense’s
Business-Like Activities,” Military Operations Research 2 (1), (pp. 19–30).
Bowlin, W. F. (1998) “Executive Compensation in the U.S. Defense Industry,” Journal of Cost Analysis
(this issue).
Bowlin, W. F.; Charnes, A.; Cooper, W. W.; and Sherman, H. D. (1985) “Data Envelopment Analysis
and Regression Approaches to Efficiency Evaluation and Estimation,” Annals of Operations
Research 2, (pp. 113–138).
Bowlin, W. F.; Wallace, J. R. II; and Murphy, R. L. (1989) “Efficiency-Based Budgeting,” Journal of
Cost Analysis 8, (pp. 33–54).
Burgess, J. F. Jr. and Wilson, P. W. (1996) “Hospital Ownership and Technical Inefficiency,” Man-
agement Science 41(1), (pp. 110–123).
Byrnes, P. E. and Cooke, T. W. (1988) Recruiting Efficiency and Enlistment Objectives: An Empirical
Analysis (Alexandria, VA.: Center for Naval Analyses, P.O. Box 16268).
Chaffai, M. E. (1997) “Estimating Input Specific Technical Inefficiency: The Case of the Tunisian
Banking Industry,” European Journal of Operational Research 98(2).
Charnes, A.; Clark, T.; Cooper, W. W.; and Golany, B. (1985) “A Developmental Study of Data
Envelopment Analysis for Measuring the Efficiency of Maintenance Units in the U.S. Air
Force,” Annals of Operations Research 2, (pp. 95–112).

25
Bowlin

Charnes, A. and Cooper, W. W. (1961) Management Models and Industrial Applications of Linear
Programming (New York: John Wiley & Sons, Inc.).
Charnes, A. and Cooper, W. W. (1962) “Programming with Linear Fractional Functionals,” Naval
Research Logistics Quarterly 15, (pp. 517–522).
Charnes, A. and Cooper, W. W. (1985) “Preface to Topics in Data Envelopment Analysis,” Annals of
Operations Research 2, (pp. 59–94).
Charnes, A.; Cooper, W. W.; Divine, D.; Klopp, G.; and Stutz, J. (1986) “An Application of Data En-
velopment Analysis to the U.S. Army Recruitment Districts,” in R. L. Schultz, ed., Applications
of Management Science (Greenwich, Conn. JAI Press, Inc.,).
Charnes, A.; Cooper, W. W.; Golany, B.; Seiford, L.; and Stutz, J. (1983) “Invariant Multiplicative
Efficiency and Piecewise Cobb-Douglas Envelopments,” Operations Research Letters 2, (pp.
101–103).
Charnes, A.; Cooper, W. W.; Golany, B.; Seiford, L.; and Stutz, J. (1985) “Foundations of Data En-
velopment Analysis for Pareto-Koopmans Efficient Empirical Production Functions,” Journal
of Econometrics 20, (pp. 91–107).
Charnes, A.; Cooper, W. W.; Huang, Z. M. and Sun, D. B. (1990) “Polyhedral Cone-Ratio DEA Models
with an Illustrative Application to Large Commercial Banks,” Journal of Econometrics, 46 (pp.
73–91).
Charnes, A.; Cooper, W. W.; Lewin, A. Y.; and Seiford, L. M. eds. (1995) Data Envelopment Analysis:
Theory, Methodology and Applications. (Kluwer Academic Publishers: Hingham, MA)
Charnes, A.; Cooper, W. W.; and Rhodes, E. (1978) “Measuring Efficiency of Decision Making Units,”
European Journal of Operational Research 3, (pp. 429–444).
Charnes, A.; Cooper, W. W.; and Rhodes, E. (1979) “Short Communication: Measuring Efficiency of
Decision Making Units,” European Journal of Operational Research 4, (pp. 339).
Charnes, A.; Cooper, W. W.; and Rhodes, E. (1980) “Management Science Relations for Evaluation
and Management Accountability,” Journal of Enterprise Management 2, (pp. 143–162).
Charnes, A.; Cooper, W. W.; and Rhodes, E. (1981) “Data Envelopment Analysis as an Approach
for Evaluating Program and Managerial Efficiency—with an Illustrative Application to the
Program Follow Through Experiment in U.S. Public School Education,” Management Science
27, (pp. 668–697).
Charnes, A.; Cooper, W. W.; Seiford, L.; and Stutz, J. (1982) “A Multiplicative Model for Efficiency
Analysis,” Socio-Economic Planning Sciences 16(5), (pp. 223–224).
Charnes, A.; Cooper, W. W.; Seiford, L.; and Stutz, J. (1983) “Invariant Multiplicative Efficiency and
Piecewise Cobb-Douglas Envelopments,” Operations Research Letters 2(3), (pp. 101–103).
Charnes, A.; Cooper, W. W.; and Sueyoshi, T. (1986) “A Goal Programming/Constrained Regression
Review of the Bell System Breakup,” Management Science.
Charnes, A.; Cooper, W. W.; and Thrall, R. M. (1991) “A Structure for Characterizing and Classifying
Efficiency and Inefficiency in Data Envelopment Analysis,” Journal of Productivity Analysis
2, (pp. 197–237).
Clarke, R. L. (1992) “Evaluating USAF Vehicle Maintenance Productivity Over Time: An Application
of Data Envelopment Analysis,” Decision Sciences 23(2).
Cooper, W. W. and Ijiri, Y. eds. (1984) Kohler’s Dictionary for Accountants, 6th ed. (Englewood
Cliffs, N.J. Prentice-Hall, Inc.)

26
 Measuring Performance

Desai, A. and Walters, L. C. (1991) “Graphical Presentations of Data Envelopment Analyses:

Management Implications from Parallel Axes Representations,” Decision Sciences, 22 (pp.
335–353).
El-Mahgary, S. and Lahdelma, R. (1995) “Data Envelopment Analysis: Visualizing the Results,”
European Journal of Operational Research, 85 (pp. 700–710).
Fried, J. O.; Lovell, C. A. K.; and Van den Eeckaut, P. (1993) “Evaluating the Performance of United
States Credit Unions,” Journal of Banking and Finance 17, (pp. 251–265).
Kamakura, W. A. (1988) “A Note on ‘The Use of Categorical Variables in Data Envelopment Analy-
sis’,” Management Science 34(10), (pp. 1273–1276).
Kantor, J. and Maital, S. (1998) “Using Data Envelopment Analysis to Improve the Efficiency of a
Major Bank,” Journal of Cost Analysis (this issue).
Kooreman, P. (1994) “Nursing Home Care in The Netherlands: A Nonparametric Efficiency Analysis,”
Journal of Health Economics 13(3), (pp. 301–316).
Morey, R. C. (1991) “The Impact of Changes in the Delayed-Entry Program on Navy Recruiting
Cost,” Interfaces 21(4), (pp. 79–91).
Pastor, J. T. (1996) “Translation Invariance in DEA: A Generalization,” Annals of Operations Research
66.
Pina, V. and Torres, L. (1992) “Evaluating the Efficiency of Nonprofit Organizations: An Application
of Data Envelopment Analysis to the Public Health Service,” Financial Accountability &
Management 8(3), (pp. 213–224).
Roll, Y.; Golany, B.; and Seroussy, D. (1989) “Measuring the Efficiency of Maintenance Units in the
Israeli Air Force,” European Journal of Operational Research 43(2), (pp. 136–142).
Ruggiero, J. (1998) “Cost Efficiency in the Provision of Educational Services: An Application of Data
Envelopment Analysis,” Journal of Cost Analysis (this issue).
Seiford, L. M. (1996) “Data Envelopment Analysis: The Evolution of the State of the Art (1978–
1995),” The Journal of Productivity Analysis 7, (pp. 99–137).
Thanassoulis, E. (1998) “Estimating Efficient Marginal Resource Levels Using Data Envelopment
Analysis,” Journal of Cost Analysis (this issue).
Thanassoulis, E.; Boussofiane, A.; and Dyson, R. G. (1995) “Exploring Output Quality Targets in the
Provision of Perinatal Care in England Using Data Envelopment Analysis,” European Journal
of Operational Research 80 (3), (pp. 588–607).
Thompson, R. G.; Langemeier, L. N.; Lee, C. T.; and Thrall, R. M. (1990) “The Role of Multiple
Bounds in Efficiency of Analysis with Application to Kansas Farming,” Journal of Economet-
rics 46 (pp. 93–108).
Thompson, R. G.; Singleton, R. D.; Thrall, R. M.; and Smith, B. A. (1986) “Comparative Site Eval-
uations for Locating a High-Energy Physics Laboratory in Texas,” Interfaces 16, (pp. 16–26).

27

View publication stats