PPA 670 POLICY ANALYSIS
ESTABLISHING ANALYSIS CRITERIA
What Are Criteria
Reliability and Validity
WHAT ARE CRITERIA
Every time a policy problem is identified, some statement
of goals is adopted. The goals are what the adopted policy alternative
should accomplish. Goals are broad, formal, long-term problem-solving achievements
that are desired. An example might be to make sure that all rivers are
safe, clean, and usable.
Goals are translated into objectives. Objectives
are more concrete statements about desired end states, with time tables,
target populations, and resource limits. An objective might be to make
the Colorado River safe for swimming and fishing.
Criteria are the measurable dimensions of objectives.
Criteria are used to compare how close different proposed policy alternatives
will come to meeting the goals of solving the problem. Criteria set the
rules to follow in analyzing and comparing different proposed policy alternatives
Sample criteria for improving river water quality might be:
effectiveness--how much of an improvement in water quality will
this alternative produce?
cost--how much will it cost to improve the quality of the river
using this alternative?
technical--do we have the equipment and know-how to use this alternative?
political--is this alternative politically acceptable?
Measures are the actual measurements that will be
taken of each proposed policy alternative. For example, measures such as
the following might be employed:
effectiveness--how many milligrams of pollutants per liter of water
will this alternative clean up?
cost--how many dollars will be required to implement this alternative?
technical--is the necessary equipment for this alternative available
and are people trained to use it?
political--what percentage of the voting-age population will favor
this alternative in a statewide poll?
One difficulty in specifying criteria and measures
is that many problem statements have vague, fuzzy, or even conflicting
goals. This is often necessary in order to get consensus on taking some
action about the problem. But this complicates the selection of criteria.
If dirty rivers are a problem, and the goal is to
have clean rivers, what is the most important considerations in choosing
between different ways of cleaning up the rivers? Is it cost? Is it effectiveness?
Is it equity?
What do we mean by "clean"? It is impossible to get
rivers 100% clean. Do we use Federal, State, or local standards on admissible
levels of toxicity? How will we measure the level of cleanliness that different
policy alternatives are likely to produce?
RELIABILITY AND VALIDITY
The criteria and their measures must be unambiguous.
They should be relatively straightforward and simple to measure. Their
application should produce uniform results, no matter who does the measuring
of different alternatives. And repeated measurements of the same alternative
should produce the same results, again, no matter who does the measuring.
Criteria and measures should be appropriate to the
unit of analysis. That is, if the goal of a proposed policy alternative
is to change the investment strategies of cities, the unit of measurement
is cities, not individuals. Be sure to specify whether the unit of measurement
is households or families, census tracts or neighborhoods, school children
or school districts, etc.
Most policy analysis involves at least one economic
criterion. These include impacts on the economy, on expected public sector
revenues, on government spending, etc.
Costs need to be counted. One cannot assume that the
money was going to be spent anyway. Costs should be identified as completely
as possible, eliminating unpleasant surprises down the road.
The most common economic criteria are costs. These may include:
borrowing costs--the costs of borrowing funds
decreases in net worth--decreases in assets and/or liabilities
direct costs--directly attributable to the policy alternative
indirect costs--additional impacts not included in the goals
intangible costs--costs that cannot be counted or quantified
monetarizable costs--can be expressed in dollars
one-time fixed costs--new capital expenditures, equipment, training,
operations and maintenance costs--ongoing costs of the alternative
opportunity costs--other things that could have been done with the
same resources instead
tangible costs--can be counted and quantified
Another type of cost criterion that is often employed
is marginal cost. That is, if some good or service is already being produced,
how much more will it cost to produce one additional unit of output?
Another type of economic criterion is benefits. Benefits
are the opposite of costs. Benefits are ways in which the policy actors
will be better off. Benefits can be measured in many of the same ways as
The types of costs that are considered in marginal analysis are:
fixed costs--these do not vary in the short run, no matter how many
units are produced
variable costs--these vary directly with the volume of output of
goods or services
average costs--the total of units of output divided by the total costs
marginal costs--the costs of producing one additional unit of output
sunk costs--these are costs that can be ignored as they have already
been spent in the past
Benefits are often more difficult to quantify than costs.
One alternative is to use "shadow prices," or the value of the benefits
in a perfectly competitive market, for example, free recreation facilities,
wilderness areas, parks, etc.
direct benefits--directly attributable to the policy alternative
increases in net worth--increases in assets and/or liabilities
indirect benefits--additional benefits not included in the goals
interest earned--interest that will accrue or be paid
intangible benefits--benefits that cannot be counted or quantified
monetarizable benefits--can be expressed in dollars
one-time benefits--one-time reduction in the problem
ongoing benefits--continuing decreases in the problem
tangible benefits--can be counted and quantified
Efficiency and effectiveness are technical and economic
questions, but equity is a public question. Equity asks about the social
allocation of burdens and benefits. Equity asks the questions of "who pays?"
and "who benefits?"
A proposed policy alternative may impact equity if
it will change the distribution of burdens and benefits in society. There
is no universally approved optimal or right answer for how benefits and
burdens should be distributed in society. That is a continuing area of
contention, and essentially a political decision.
However, there are guidelines for equity, such non-discrimination,
and the same treatment for those equally situated and different treatment
for those unequally situated.
Horizontal equity asks whether burdens and benefits
are being shifted among groups in society which are relatively equal.
Vertical equity asks whether burdens and benefits
are being shifted among groups in society which are relatively unequal.
Inter-generational equity asks whether burdens or
benefits are being shifted from one time period to another, whether younger
generations will have to pay more and receive less than older ones, or
Groups are often identified on the basis of:
race or ethnicity
Problems in assessing equity include:
how should the population be sub-divided?
how should groups be defined?
should historical criteria, the status quo, or desired states be used?
what is a burden?
what is a benefit?
what is a degree of need?
what is an ability to pay?
Effectiveness is often used as a criterion by which
to judge policy proposals. Effectiveness is the extent to which the proposed
policy will attain the goals set forth in the problem statement. For example,
if the goal is to decrease the current teenage driver accident rate, how
much will each policy alternative decrease the rate below current levels?
Another technical criterion is technical feasibility.
This asks whether the technology exists or is readily available to implement
a proposed alternative. For example, one proposed policy alternative may
be to install in all cars a breath analyzing device that would not let
a car start if the driver has been drinking. However, this technology is
not widely or cheaply available.
Other technical criteria may question whether the
measurement of criteria can be conducted at the desired level of reliability
and validity. For example, are there tests that can adequately measure
whether students in bilingual education programs have the same level of
literacy as students in non-bilingual education programs?
Many times the client for the policy analysis will hold
a political office. In that case, the policy analyst must often include
political criteria in the assessment of proposed policy alternatives.
Political viability asks whether or to what extent
a proposed policy alternative will be acceptable to relevant powerful groups,
decision makers, legislators, administrators, citizens, neighborhoods,
unions, or others.
Other ways of assessing political viability include:
acceptability--is the proposed alternative acceptable to policy
makers, policy targets, the general public, voters, etc.?
appropriateness--is the proposed alternative appropriate to the
values of the community, society, the legislature, etc.?
legal--is the proposed alternative legal under current law, or will
statutes have to be amended or enacted?
responsive--will the proposed alternative meet the real or perceived
needs of the target group, the public, etc.?
Many public policies are implemented by public agencies.
Therefore, administrative operability or administrative ease are often
used as criteria for judging proposed public policies.
Questions that may be addressed include:
authority--does the agency have the authority to implement the proposed
commitment--does the proposed policy have the commitment of top
managers, field staff, and support staff?
capacity--does the agency have the resources to implement the proposed
policy, in terms of staff, skills, money, training, expertise, etc.?
support--are the facilities, equipment, and other support available
for the proposed policy?