Translating Financial Education into Behavior Change

for Low-Income Populations

Angela C. Lyons, Yunhee Chang, and Erik M. Scherpf

The impact that financial education had on the financial behaviors of (a) the agency staff who were trained to
deliver the program and (b) the low-income individuals who participated in the program was investigated. Spe-
cifically, the researchers examined the relationship between total number of financial education lessons com-
pleted, prior financial experience, and improvement in individuals’ financial behaviors. The results provide
some evidence that financial education may result in improved financial behaviors. However, the findings sug-
gest that prior level of financial experience may matter more than the number of lessons completed. Researchers
may want to re-examine the indicators currently being used to show program impact and whether financial
knowledge is the appropriate catalyst to foster behavior change.

Key Words: financial education, financial socialization, low-income populations, program evaluation

Introduction tion needs of low-income populations. However, research

High levels of consumer debt, low personal saving rates, measuring the effectiveness of these programs has not kept
and increases in personal bankruptcy filings have gener- pace. There are a number of reasons why limited research
ated concern that consumers lack adequate financial skills is available. First, researchers face challenges in collecting
(Bell & Lerman, 2005; Lyons, Palmer, Jayaratne, & data from program participants with low financial literacy
Scherpf, 2006; National Endowment for Financial Educa- levels. Low-income participants are often difficult to track
tion, 2004). As the financial system has grown rapidly and have high program drop out rates and low survey
more complex, consumers have had to become more response rates (Anderson, Zhan, & Scott, 2004; Lyons,
actively involved in managing their finances. Yet, many 2005; Lyons et al., 2006; Lyons & Scherpf, 2004). These
consumers, even those who would describe themselves as factors, coupled with their reluctance to divulge personal
“financially savvy,” are having difficulty assessing their information, limit the amount and type of information that
options and making sound financial decisions. The burden can be collected. The result is that survey instruments are
for low-income and disadvantaged individuals can be often kept short and simple to increase response rates and
particularly overwhelming. In the current financial envi- reduce measurement error.
ronment, it is easy for low-income and disadvantaged
populations to fall victim to predatory lenders and finan- Second, researchers face challenges collecting impact data
cial scams, especially because many lack adequate finan- because of the nature of the organizations that deliver
cial education (Lyons & Scherpf, 2004). Basic financial financial education programs. Most programs that target
management skills are important for all households, but low-income populations are operated by small non-profit
they are particularly critical for low-income households to organizations with limited staff and financial resources.
ensure long-term financial security. Relative to their operating expenses, program evaluations
can be expensive to conduct, especially rigorous evalua-
A number of financial education programs have been tions that use control groups and have a longitudinal com-
developed in recent years to address the financial educa- ponent. In addition, many of the agency staff and volun-

Angela C. Lyons, Assistant Professor, University of Illinois at Urbana-Champaign, Department of Agricultural and Consumer Economics, 421 Mumford
Hall, 1301 West Gregory Drive, Urbana, IL 61801, anglyons@uiuc.edu, (217) 244-2612
Yunhee Chang, Visiting Assistant Professor, University of Mississippi, 202 Lenoir Hall, P.O. Box 1848, University, MS 38677, chang@olemiss.edu,
(662) 915-1352
Erik M. Scherpf, Research Associate, University of Illinois at Urbana-Champaign, Department of Economics, 330 Wohlers Hall, 1206 S. Sixth Street,
Champaign, IL 61820, scherpf@uiuc.edu, (217) 333-0120

© 2006 Association for Financial Counseling and Planning Education. All rights of reproduction in any form reserved. 27
teers do not have expertise in evaluation and lack the changes. The results also provide insight into the impor-
understanding and knowledge about how to measure tance of controlling for previous financial experience.
program impact to show that their programs are working Finally, the findings show how RPTs can provide useful
(Lyons, 2005; Lyons et al., 2006). In the end, there are few insight into the effectiveness of financial education pro-
incentives for instructors to collect data and for partici- grams for low-income populations, underscoring the value
pants to provide information. of more traditional types of evaluation methods.

Most organizations currently conducting program evalua- Review of Literature

tions use only a post-test or a pre- and post-test to collect Recent literature has provided general insight into the link
impact data (Lyons et al., 2006). However, given the between financial education and behavior change. Re-
nature of low-income populations and the constraints searchers typically have concluded that financial education
facing many organizations, it may be more efficient and results in positive behavior change. The majority of these
effective to collect impact data using a retrospective pre- studies have focused on collecting data from target popula-
test (RPT). The RPT is administered in the same way as a tions that are readily available and willing to participate in
post-test in that participants are asked to answer questions formal evaluations, such as employees, students, and
about their level of knowledge and behavior after the financial counseling clients. For an overview of the litera-
program. They are then asked to think back to their level ture, see Bell and Lerman (2005), Braunstein and Welch
of knowledge and behavior prior to the program. Despite (2002), Fox, Bartholomae, and Lee (2005), Hilgert,
the potential limitations of this method, RPT can reduce Hogarth, and Beverly (2003), Hogarth (2002), Hogarth,
response-shift biases and provide a feasible and efficient Beverly, and Hilgert (2003), Lyons (2005), Lyons et al.
evaluation method for collecting impact data (Lamb, 2005; (2006), and National Endowment for Financial Education
Lamb & Tschillard, 2005).1 (2004).

For the current study, a RPT was used to collect 4 years of Studies that have concentrated on the effect of financial
repeated cross-sectional evaluation data from a nationally education in the workplace have focused on increasing
recognized financial education program. The data were employees’ savings and enhancing worker productivity.
used to investigate the impact that education had on the Bayer, Bernheim, and Scholz (1996) and Bernheim and
financial behaviors of the agency staff that were trained to Garrett (2003) found that employer-provided financial
deliver the program and of the low-income individuals education increased employee participation in retirement
who participated in the program. Controlling for prior plans and the amount saved for retirement and for general
financial behaviors, we estimated a series of probit models purposes. They also found that the effect of workplace
to determine if the amount of financial education received financial education tended to be strongest for employees
had an impact on (a) overall financial behavior and (b) who saved little before the program. In another study,
anticipated changes in five specific financial behaviors. Garman, Kim, Kratzer, Brunson, and Joo (1999) found that
The “treatment effect” in our models was defined by the 75% of individuals who chose to participate in employer-
variation in the intensity of the treatment (i.e., the number sponsored financial education workshops felt more confi-
of lessons completed) instead of by whether an individual dent in their ability to make investment decisions, and in
received the treatment or did not receive the treatment (i.e., turn, made better financial decisions following the work-
participated in the program or did not participate). Al- shops. Kim and Garman (2003) also found that employer-
though this was not a traditional control group study, we provided financial education increased the financial confi-
were able to examine the impact of the education condi- dence of program participants and resulted in improved
tional on some level of program participation by compar- financial practices and worker productivity.
ing the behavior changes of participants who received
more lessons to those who received fewer lessons. Studies that have focused on youth provide evidence that
formal courses in personal finance increase financial
The findings presented provide insight into how financial knowledge and result in more positive financial behaviors.
education programs for low-income populations can be For example, Boyce and Danes (2004) found that a formal
improved, especially with respect to program length and financial planning program had a significant and positive
the indicators currently being used to show how financial impact on high school students’ spending habits, savings
education can translate into actual and anticipated behavior behaviors, and confidence levels in managing money, even

28 Financial Counseling and Planning Volume 17, Issue 2 2006

3 months after they completed the program. Bernheim, ished and leveled off. A follow-up report by Schreiner,
Garrett, and Maki (2001) found that mandated financial Clancy, and Sherraden (2002) showed that savings in-
education during high school resulted in higher savings creased by only a small amount initially and then the effect
rates and higher net worth when students reached adult- leveled off after 8 to 10 hours of education.
hood.
In a more recent study, Shockey and Seiling (2004) used
Other studies have shown that financial counseling results the Transtheoretical Model of Change (TTM) to specifi-
in improved financial behaviors. Staten, Elliehausen, and cally assess change in six financial behaviors over a 4-
Lundquist (2002) tracked credit counseling clients for 3 week period for individuals enrolled in an IDA financial
years and found that those who received counseling were education program. The TTM framework integrates major
able to reduce their debt, improve their credit card man- psychological theories into a theory of behavior change
agement, and lower their delinquency rates more than (i.e., Prochaska, 1979; Prochaska & DiClemente, 1983).
those who did not receive counseling. Hirad and Zorn The six financial behaviors examined were setting finan-
(2001) found that borrowers who participated in pre- cial goals, using a spending plan, tracking spending, reduc-
purchase homeownership counseling had a 19% lower 90- ing debt, setting aside money for unplanned expenses, and
day delinquency rate than those who did not receive coun- saving money. Prior to the program, Shockey and Seiling
seling. Kim, Garman, and Sorhaindo (2003) provided found that program participants were, on average, at the
evidence that suggested that credit counseling may have stage of action with respect to reducing debt and at the
direct effects on financial stressor events and indirect stage of preparation for all other behaviors. Following the
effects on individuals’ perceived financial well-being program, they found that participants experienced the
and health. smallest change for the behavior that was associated with
reducing debt and the largest change for setting aside
Research that has measured the effectiveness of financial money for unplanned spending.
education for low-income populations has been more
limited. Existing literature has been tied frequently to Another study of low-income individuals, not tied to an
participation in Individual Development Account (IDA) IDA program, investigated the impact that financial educa-
programs. The goal of IDA programs has been to increase tion had on the decision of unbanked individuals to open a
savings rates for low-income families by providing match- bank account and move into mainstream financial markets.
ing funds for savings toward a specific purpose such as Lyons and Scherpf (2004) showed that financial education
homeownership, higher education, or to start a small was successful in encouraging the unbanked to open a
business. Many of these programs have included a finan- bank account. However, they also found that, even after
cial education component. Research from one IDA pro- the program, financial constraints prevented a significant
gram focused on knowledge gain and examined the levels proportion of unbanked participants from opening an
of pre-training financial knowledge of program partici- account. According to Lyons and Scherpf, no matter how
pants (Anderson et al., 2004; Zhan, Anderson, & Scott, in much financial education some low-income individuals
press). The goal of the current research was to identify received, they were still unable to change some financial
gaps in financial knowledge and to determine the financial behaviors, because their overall financial position had not
education needs of low-income populations. However, this changed. They argued that the best measure of program
research spent little time investigating whether financial “success” may be related to whether the participants re-
education resulted in positive behavior change for program ceive the financial skills needed to make decisions that are
participants. applicable to their specific financial circumstances.

Other studies have focused more on behavior change by To summarize, the literature has provided evidence that
examining how financial education affects spending, financial education can improve the financial well-being of
savings, and debt management outcomes. Clancy, low-income individuals and their families. Yet, studies that
Gristein-Weiss, and Schreiner (2001) used data collected have focused on low-income populations have been lim-
from the American Dream Demonstration and found that ited in the following respects. First, most of these studies
saving deposits and saving frequency in IDAs increased as have relied on data collected solely from program partici-
hours of financial education increased from 0 to 12 hours. pants and not from any comparison groups. They com-
However, after 12 hours, they found that the effect dimin- pared only the pre- and post-program knowledge and

Financial Counseling and Planning Volume 17, Issue 2 2006 29

behavior of program participants. However, in most cases, likely to have a more meaningful learning experience and
there was significant variation in the amount of financial to report improvement.
education each participant received even within the same
program, which offered an alternative to comparison group This study builds upon prior research and addresses these
data. In addition, the administration of these programs was critical gaps in the literature. Using a RPT, we controlled
often varied, especially with respect to how participants for prior financial behaviors and investigated the impact
were recruited and what portions of the curriculum were that the program had on both program participants and
delivered by the instructors. Some programs offered “one- agency staff. We also used the total number of financial
shot” workshops and seminars that focused on one or two education lessons completed to provide additional insight
lessons from the program, whereas others offered a more into the relationship between the amount of financial
comprehensive program with multiple lessons. Only re- education received and self-reported improvement in
cently have researchers and financial professionals ques- individuals’ actual and anticipated financial behaviors.
tioned whether the amount of financial education matters
(Clancy et al., 2001; Schreiner et al., 2002). In other Data Collection
words, is more better? A few studies have used hours of Description of the Program
education to account for the amount of financial education All My Money is a financial education program developed
received by program participants. However, there are other by University of Illinois Extension that focuses on provid-
measures that could be used, such as the number of lessons ing financial management and consumer skills to low-
completed. Using alternative measures allows us to com- income households. The curriculum consists of eight
plement the absence of control groups by comparing the instructor-led lessons that cover a number of personal
participants who completed more lessons to those who finance topics including (a) making spending choices, (b)
completed less. It also adds to our understanding of envelope budgeting, (c) planning expenditures, (d) under-
whether more really is better. Determining the optimal standing credit, (e) handling credit problems, (f) building
amount of education can have important implications for consumer skills, (g) taking consumer action, and (h) man-
program delivery. aging a checking account. Each lesson consists of hands-
on activities and handouts as well as lesson plans and
A second limitation of previous literature is that it has instruction guides. Also, each lesson is designed to take
focused on the impact that financial education has on the approximately 60 minutes to deliver.
knowledge and behaviors of only the program participants.
Most financial education programs that have targeted low- The program was primarily designed to target two audi-
income populations, however, have had a train-the-trainer ences: (a) staff of social service organizations and govern-
component, where financial educators trained agency staff ment agencies that worked directly with low-income
from non-profit organizations so that they, in turn, could audiences and who were trained to deliver the program to
effectively deliver the program to their clients. During their clients and (b) low-income clientele who may have
these training programs, the agency staff actually com- had limited financial literacy. Extension educators trained
pleted the program. They also learned about effective agency staff members using a series of workshops that
delivery methods and received guidance on how to re- totaled 16-20 hours of hands-on instruction in basic finan-
spond to clients’ needs and how to motivate them to cial management depending on the number of lessons
change their behaviors. Little, if any, research has exam- offered by the instructors. Roughly 95% of the agency
ined the impact that financial education has on the agency staff were trained in the entire program and received all
staff trained to deliver the programs (Baron-Donovan, eight lessons. Trainings occurred over the span of a few
Wiener, Gross, & Block-Lieb, 2005; Shelton & Moss, days to a few weeks.
2002). These instructors play a critical role with respect to
the quality of the program and whether program partici- The agency staff, in turn, offered the program to their
pants are motivated to positively change their behaviors. clientele. Agency staff, however, had considerable discre-
Instructors who go through the program themselves are tion over how clients were recruited and which lessons
likely to be more confident in their own financial manage- were delivered to their clients. Therefore, the number and
ment skills and in their ability to respond to participants’ types of lessons offered to the clientele varied significantly
questions. The end result is that low-income audiences are by instructor and location. For example, the data showed

30 Financial Counseling and Planning Volume 17, Issue 2 2006

that more than half of the clients who participated in the participants are asked to make assessments. Also, if the
program were taught the budgeting, planning, and credit time lapse between the “before” and “after” is only a few
lessons, whereas the banking and problem-handling les- days or a couple of weeks, the likelihood of recall bias will
sons were taught less frequently. Thus, the program lasted be lower, as was the case in the current study.
anywhere from a few days to a few weeks depending on
how many lessons were administered to the clientele. The RPT included a self-assessment of how overall finan-
cial management performance changed after the program.
To date, the program has primarily been offered in the Information was also collected on anticipated changes in
state of Illinois. Over 100 agencies and organizations in specific financial behaviors. The overall impact of the
Illinois have participated in the All My Money program program was measured using the question, “After partici-
including, but not limited to welfare-to-work and other pating in the program, how much would you say your
social service programs, IDA programs, consumer credit ability to manage money has changed?” Participants re-
counseling services, homebuyer education programs, ported their changes on a 5-point scale ranging from much
community and faith-based organizations, and financial worse to much better. The questionnaire also asked about
institutions. respondents’ self-assessment with respect to the following
five behavior categories: budgeting, intra-household com-
How the Evaluation Was Conducted munication, bill payment, ability to handle consumer
Between 1998 and 2002, University of Illinois Extension problems, and comparison shopping. Participants indicated
educators collected data in Illinois from both the agency on a 4-point scale how often they engaged in each behav-
staff and clientele. The data were gathered using a RPT, ior prior to the program and how often they planned to
where participants were asked at the end of the program to engage in each behavior now that they have completed the
retrospectively report their financial behaviors prior to the program. Responses ranged from almost always to almost
program and how they would change as a result of the never. Additional information was obtained on each par-
program. Given the recent movement towards more rigor- ticipant’s age, gender, family size and composition, per-
ous program evaluations that include control groups and a sonal and household income, and educational attainment.
longitudinal component (Lyons, 2005; Lyons et al., 2006), The location and dates of the program, including number
the RPT may be perceived as less rigorous, and therefore of lessons completed, were also recorded for each partici-
less convincing, than other approaches. One concern in pant.
using the RPT is that program participants may be more
inclined to show a learning effect, especially if the partici- The process for collecting the survey information was as
pants develop a good rapport with their instructor. There follows. Extension educators administered the RPT to
are additional concerns that using a RPT could affect the agency personnel at the end of the train-the-trainer pro-
validity of the data by introducing self-reporting and recall gram. When the agents delivered the program to their
bias. clients, they had the option of administering the same RPT
to those who completed the program. Agents who adminis-
However, given the realities of program evaluation for tered the evaluation to their clients were asked to return the
organizations that target low-income populations, the RPT surveys to the extension educator who had trained them.
is a common and useful evaluation tool. One clear advan- Of the more than 100 agencies who participated in the
tage is that it only has to be administered once, reducing training, only 19 agencies chose to return surveys. Because
the amount of time the participants spend as research it was optional, the agencies that returned the surveys may
subjects. Another advantage is that it can reduce response- not have been random. In addition, their clientele may
shift effects, which occur when a participant’s frame of have differed in a fundamental way from the clientele from
reference changes significantly during the program the agencies that did not return their surveys. We com-
(Howard, 1980; Lamb, 2005; Lamb & Tschillard, 2005). pared the characteristics of the agencies that returned
For example, participants may not be familiar with basic surveys with those that did not and found that the agencies
terms and concepts prior to the program. Therefore, a that had returned clientele surveys were fairly representa-
traditional pre-test may yield biased results if the test tive of the agencies that had not. Key characteristics that
introduces terms and concepts before the participants are were compared included size of the organization, target
ready for them. The RPT reduces the likelihood of re- audiences, types of programs offered, and services pro-
sponse-shift effects by clearing up misconceptions before

Financial Counseling and Planning Volume 17, Issue 2 2006 31

vided to clientele. The composition of the clientele also (47.4%) of the pooled sample reported personal monthly
tended to be fairly representative. earnings of $1,500 or more (the top income bracket in the
survey), and 45.8% claimed household income from other
In the end, a total of 763 evaluations were collected, 546 sources besides their own earnings. Most (60.3%) of the
from agency staff and 217 from clientele. Due to incom- trainings and programs were delivered in the Chicago area,
plete and missing information on key survey questions, the largest metropolitan area in the state.
174 observations were unusable. The final sample con-
sisted of 589 observations (77.2% of the 763 collected On average, clientele participants completed 3.7 lessons.
surveys), where 428 were agency personnel (72.7%) and The average age was 34.5 years, with the majority (58.4%)
161 were clientele (27.3%). under 35 years of age. More than a quarter did not com-
plete high school, whereas only 11.8% had a college
Sample Description degree. In addition, the majority (59.1%) of clientele
Table 1 presents demographic information on the agency participants reported monthly incomes below $1,000, and
personnel who participated in the training and the clientele nearly three-quarters (73.9%) of them did not have any
who participated in the program. The first set of columns other source of household income. Approximately 88%
presents data for the pooled sample of agency personnel were female, and 79.5% reported having children living in
and clientele. The second set of columns is restricted to the household. Only 58.4% reported living with another
clientele participants only. One might question why the adult. Thus, the client sample consisted of a large number
sample of agents and clientele were pooled. There were of single mothers. With respect to location, 82% were
several reasons for this. First, recall that roughly 95% of located in the Chicago metropolitan and surrounding areas.
the agents were trained in all eight lessons. This provided
insufficient variation to separately measure the impact of Table 1 also provides demographic information for partici-
the number of lessons on agency staff. In addition, the pants who reported and did not report a positive improve-
pooled regression findings showed that the effects of the ment in their overall financial behaviors following the
program on participants’ behaviors did not vary signifi- program. Approximately 47.0% of the pooled sample
cantly between the agency staff and the clientele. Addi- reported that as a result of the program their ability to
tional qualitative evidence suggested that the financial manage their money was “much better,” 43.0% reported
behaviors of the agents may not have been all that different that it was “a little better,” 9.9% indicated that it was
from those of the clients. A number of the agents may have “about the same,” and only 0.1% indicated that it was “a
been struggling financially themselves, even though they little worse” or “much worse.” Participants who reported
may have been more financially knowledgeable and may that their ability to manage their money was “a little bet-
have had more education and income than their clients. ter” or “much better” were classified as having experi-
However, given that there was substantial variation in the enced a positive improvement in their overall financial
number of lessons completed by the clientele, which is the behavior.
ultimate target audience, we also include information in
the descriptive tables on the clientele-only sample because The majority of program participants reported an improve-
this population may be of greater policy interest. Addi- ment in their financial management practices: 90.0% in the
tional explanation for why we pooled the data is provided pooled sample (n = 530) and 85.0% in the clientele-only
later in the paper. sample (n = 137). Note, however, that although clientele
participants accounted for only 27.3% of the overall sam-
On average, the pooled sample completed 6.7 lessons. ple, they made up 40.7% of those reporting no improve-
Participants were comprised predominantly of female ment. This perhaps suggests that either the program was
(86.9%) and middle-aged participants. The average age of not as successful for the clientele participants or they could
the participants was 39 years, with 83.1% of the sample not accurately assess their level of improvement. Due to
falling into the 25-54 age bracket. Only 8.2% of the entire the small number of participants reporting no improve-
sample did not complete high school, whereas 44.3% had ment, however, one must be cautious in drawing infer-
received a college degree. With respect to family size and ences from these numbers.
composition, 59.8% reported having at least one child
living in the household, and 70.6% reported having Two-tailed t tests were used to identify characteristics that
spouses or other adult household members. Nearly half were significantly different between those who reported an

32 Financial Counseling and Planning Volume 17, Issue 2 2006

Table 1. Description of the Sample by Overall Improvement in Financial Behavior

Pooled sample Clientele-only sample

Financial behavior Financial behavior
after the program after the program
Variables Total Total
(N = 589) No (N =161) No
Improve Improve
improve |t| improve |t|
(n = 530) (n = 137)
(n = 59) (n = 24)

Total number of lessons 6.7 5.6 6.8 4.08** 3.7 2.9 3.9 2.16**
Age 39.0 39.1 39.0 0.06 34.5 34.1 34.6 0.17
24 or less 8.7 11.9 8.3 0.92 20.5 25.0 19.7 0.59
25-34 31.1 23.7 31.9 1.28 37.9 33.3 38.7 0.50
35-44 28.2 32.2 27.7 0.72 23.6 25.0 23.4 0.17
45-54 23.9 23.7 24.0 0.04 13.7 8.3 14.6 0.82
55 or more 8.2 8.5 8.1 0.10 4.3 8.3 3.6 1.03
Female 86.9 91.5 86.4 1.10 88.2 87.5 88.3 0.11
Education
Less than high school 8.2 10.2 7.9 0.60 26.1 25.0 26.3 0.13
High school or GED 20.5 28.8 19.6 1.66* 35.4 37.5 35.0 0.23
Some college 27.0 18.6 27.9 1.52 26.7 25.0 27.0 0.20
Bachelor’s degree 27.7 23.7 28.1 0.71 8.7 8.3 8.8 0.07
Graduate degree 16.6 18.6 16.4 0.44 3.1 4.2 2.9 0.32
Number of children
None 40.2 45.8 39.6 0.91 20.5 29.2 19.0 1.14
1 21.7 13.6 22.6 1.61 18.6 12.5 19.7 0.83
2 21.4 22.0 21.3 0.13 31.1 33.3 30.7 0.26
3 or more 16.6 18.6 16.4 0.44 29.8 25.0 30.7 0.56
Other adult members 70.6 83.1 69.2 2.21** 58.4 83.3 54.0 2.73**
Personal income
$249 or less 3.2 1.7 3.4 0.70 9.9 4.2 10.9 1.02
$250-499 5.8 13.6 4.9 2.72** 19.3 25.0 18.2 0.77
$500-749 7.8 10.2 7.5 0.71 19.3 25.0 18.2 0.77
$750-999 8.0 6.8 8.1 0.36 10.6 16.7 9.5 1.05
$1,000-1,249 12.6 15.3 12.3 0.66 10.6 12.5 10.2 0.33
$1,250-1,499 15.3 11.9 15.7 0.77 11.2 4.2 12.4 1.18
$1,500 or more 47.4 40.7 48.1 1.08 19.3 12.5 20.4 0.91
Other income sources 45.8 57.6 44.5 1.92* 26.1 41.7 23.4 1.89*
Chicago area 60.3 62.7 60.0 0.40 82.0 79.2 82.5 0.39
Clientele 27.3 40.7 25.8 2.43** 100.0 . .

Note. Mean values are reported for the variables that represent total number of lessons and age.
*p < .10. **p < .05.

Financial Counseling and Planning Volume 17, Issue 2 2006 33

improvement and those who did not. Note that participants spouse or parent (p ≤ .05). In addition, they were less
who reported an improvement completed more lessons on likely to report having other income sources besides per-
average than those who showed no improvement. In the sonal income (p ≤ .10). Further evidence from the pooled
pooled sample, those who reported improvement attended sample revealed that those who reported an improvement
6.8 lessons, compared to 5.6 for those who reported no were less likely to be a client than those who reported no
improvement. For the clientele, the average number of improvement (p ≤ .05).
lessons completed was 3.9 and 2.9, respectively. The
differences between the two groups were significant at Table 2 presents evidence of program impact according to
the .05 level. anticipated changes in specific financial behaviors. Over-
all, the program appears to have had a positive impact on
Program participants who reported an improvement were each of the five behaviors. The most noticeable improve-
also less likely than those who reported no improvement to ment was for participants who reported that they antici-
live in a household with other adult members including a pated running out of money less frequently after the pro-

Table 2. Anticipated Changes in Specific Financial Behaviors

Pooled sample (%) Clientele-only sample (%)

Financial behaviors
pre post pre post
Do not run out of money (n = 549)
1 = Almost never 10.9 1.5 26.8 1.4
2 = Sometimes 16.2 3.5 26.8 6.3
3 = Often 39.3 30.6 32.4 48.6
4 = Almost always 33.5 64.5 14.1 43.7
Talk with family about money (n = 555)
1 = Almost never 19.1 7.2 28.7 12.6
2 = Sometimes 45.2 24.7 38.5 30.8
3 = Often 22.5 40.4 18.2 28.0
4 = Almost always 13.2 27.8 14.7 28.7
Do not pay bills late (n = 551)
1 = Almost never 8.0 1.8 20.0 3.6
2 = Sometimes 9.6 3.3 16.4 6.4
3 = Often 35.4 17.1 34.3 29.3
4 = Almost always 47.0 77.9 29.3 60.7
Complain when having a consumer problem (n = 553)
1 = Almost never 25.5 13.9 24.1 24.1
2 = Sometimes 44.7 28.0 41.8 39.7
3 = Often 16.1 28.9 20.6 20.6
4 = Almost always 13.7 29.1 13.5 15.6
Compare prices and quality before buying (n = 556)
1 = Almost never 7.9 2.2 12.7 4.9
2 = Sometimes 29.5 8.3 29.6 10.6
3 = Often 25.4 23.7 20.4 23.9
4 = Almost always 37.2 65.8 37.3 60.6
Note. The number of observations varies slightly for each behavior because a few participants chose not to respond to all
financial behavior questions.

34 Financial Counseling and Planning Volume 17, Issue 2 2006

gram, and this improvement was more pronounced for the Empirical Framework
clientele. Over half of the clientele reported not running Although we did not have access to a traditional control
out of money “almost never” or “sometimes” prior to the group (i.e., a sample of non-participants), we were able to
program, compared to only 7.8%, who anticipated not assess the treatment effect of the program by looking at
running out of money following the program. whether the program impact was larger for participants
who completed a greater number of lessons. The relation-
Participants, especially clientele participants, reported ship was expressed as follows:
similar improvement in paying bills on time. For the
clientele, 29.3% reported that they “almost always” did not Yi* = α · Lessonsi + Xiβ + Tδ + ei, (1)
pay bills late prior to the program, compared to 60.7% who where Yi = 1 iff Yi* > 0 and 0 otherwise.
anticipated “almost always” paying bills on time after the
program. Participants also experienced improvements in The subscript i indexed individual participants, for i = 1,
two other behavior categories—financial communication 2, ..., N. In this model, Yi* was the improvement in finan-
within the family and comparison shopping. cial management behaviors, that indicated the degree to
which participants anticipated that they would engage in
Table 3 examines the relationship between the total num- more desirable financial behaviors as a result of the pro-
ber of lessons completed (or the amount of financial edu- gram. Yi* was a latent measure that was not directly ob-
cation received) and the self-reported improvement for servable. Instead, a binary index, Yi, was observed in the
each of the five financial behaviors. Information is also data such that Yi was equal to one if the i t h participant
included on pre-program behavior, where pre-program reported any positive change in his/her financial behavior
behavior is defined to be the self-reported degree to which following the program and zero otherwise.
the participant engaged in a particular financial behavior
prior to the program. For each behavior, the sample ex- Yi* was modeled as a function of the total number of
cludes those who responded that they “almost always” lessons completed by each program participant (Lessons),
engaged in that particular financial behavior prior to the a vector of demographic and economic characteristics of
program. For the pooled sample, improvement in each of the participant (X), and a vector of fiscal year dummies
the behaviors was associated with a higher average number (T). Included in X were control variables such as age,
of lessons completed, with the exception of the financial gender, education, household size and composition, per-
behavior “do not run out of money.” A similar, but gener- sonal income of the participant, and whether the household
ally weaker trend was found for three of the five financial had other income sources. We also included in the model
behaviors for the clientele sample. For the clientele-only an indicator for whether the participant resided in the
sample, improvement in talking with the family about Chicago metropolitan or surrounding areas, because we
money, handling consumer problems, and comparison suspected that the overall economic and financial environ-
shopping were associated with more lessons on average. ment that households faced in large metropolitan areas was
considerably different from that found in more rural areas.
Table 3 also shows that the greatest improvement occurred One might question why the model did not control for
among participants who reported the poorest financial race/ethnicity, marital status, employment patterns, and
behaviors prior to the program. In both samples, over some measure of asset holdings or net worth. Unfortu-
80.0% (88.5% in the pooled sample and 83.9% in the nately, the data set did not include this information. Our
clientele sample) of those reporting no improvement in the task was to gain as much information as possible from the
“do not run out of money” category indicated that they available data that had been collected.
“often” did not run out of money prior to the program. In
addition, it should not be surprising that, for the clientele To determine if the base level of financial knowledge and
sample, 78.1% of those who reported an improvement in experience varied between the clientele and agency staff,
the “do not run out of money” category indicated that their we also included an indicator variable that identified
pre-program behavior in this area was poor (i.e., they whether the participant was a client or agent. The signifi-
“almost never” or “sometimes” did not run out of money). cance of this dummy variable determined whether there
It should be noted, however, that some of these descriptive were differences in the parameters (α, β, and δ) between
differences disappeared when we controlled for other those who showed improvement and those who did not,
factors in the multivariate analysis. which in turn indicated whether separate models should be

Financial Counseling and Planning Volume 17, Issue 2 2006 35

Table 3. Anticipated Changes in Specific Financial Behaviors by Total Number of Lessons and
Pre-Program Financial Behaviors, Reduced Sample

Do not run out Talk with family Do not pay Complain when Compare prices
of money about money bills late problem and quality
No No No No No
Improve Improve Improve Improve Improve
improve improve improve improve improve
Pooled sample
n 104 261 221 261 70 222 216 261 90 259
Total number of 6.6 6.5 6.7 7.1* 6.1 6.6* 6.3 7.4** 6.6 7.1*
lessons completed
Pre-program
behavior (%)
1 = Almost never 2.9 21.8** 13.6 29.1** 4.3 18.5** 19.9 37.5** 3.3 15.8**
2 = Sometimes 8.7 30.7** 46.2 57.1* 10.0 20.7* 52.8 51.0 40.0 49.4
3 = Often 88.5 47.5** 40.3 13.8** 85.7 60.8** 27.3 11.5** 56.7 34.8**
Clientele-only sample
n 31 91 63 59 29 70 84 38 26 63
Total number of 3.8 3.7 3.7 3.8 4.1 3.6 3.8 4.1 3.6 4.2
lessons completed
Pre-program
behavior (%)
1 = Almost never 6.5 39.6** 20.6 47.5** 10.3 35.7* 22.6 39.5* 7.7 25.4*
2 = Sometimes 9.7 38.5** 46.0 44.1 13.8 27.1 47.6 50.0 38.5 50.8
3 = Often 83.9 22.0** 33.3 8.5** 75.9 37.1** 29.8 10.5** 53.8 23.8**
Note. The sample was reduced to those who did not respond “almost always” for their pre-program behavior, and thus the
category “almost always” is excluded from the table. The breakdown of other sample characteristics by behavior and level
of improvement is available upon request.
*p < .10. **p < .05.

estimated for the agency staff and the clientele. The vector The remainder of this section presents the specifics for the
T controlled for economic conditions specific to each two models.
survey year, as well as for yearly variation in audience
makeup, program budgets, resources available for training, Model 1: Overall Program Impact
and the ease of participants’ behavioral adjustments. Recall that the overall program effect was measured by
the question, “After participating in the program, how
For each model, the unknown parameters (α, β, and δ) much would you say your ability to manage money has
were obtained using the probit method. The error terms, ei, changed?” Out of the five ordered categories ranging from
were assumed to be random and normally distributed with much worse to much better, the responses “a little better”
a mean of zero. The coefficient of interest, α, was expected and “much better” were considered to demonstrate positive
to be positive and significant, which meant that the more latent effects. The probability that the program had a
lessons participants completed, the more likely they were positive effect overall was modeled as follows:
to engage in more desirable financial behaviors. Two
probit models were estimated to determine the effect that Pr(Yi = 1 | Lessonsi, Xi, T) = Φ(α·Lessonsi + Xiβ + Tδ), (2)
financial education had on (a) overall financial behavior where Φ(.) was the standard normal distribution func-
and (b) anticipated change in specific financial behaviors. tion.

36 Financial Counseling and Planning Volume 17, Issue 2 2006

Note that this model did not control for an individual’s In this model, j was equal to 1 for “poor improvement,” 2
prior financial knowledge and habits. Although demo- for “moderately poor improvement,” 3 for “moderately
graphic and socio-economic control variables were in- good improvement,” and 4 for “good improvement.” I[.]
cluded in the model to account for sample heterogeneity, was an index function that took the value of one if its
prior differences in overall financial education across argument was true and zero otherwise. The parameters θj
participants were not observed. It was important, however, for j = 1, 2, 3 represented how the likelihood of improve-
to assess the program effect holding constant the partici- ment depended on the participant’s level of financial
pants’ financial habits and practices prior to the program. behavior prior to the program. The parameters were ex-
We addressed this issue by estimating a second model that pected to decrease in j, which implied that the program
looked at anticipated changes in specific financial behav- was more likely to benefit those who started out at lower
iors. levels of financial knowledge (see the Appendix for
mathematical proof). Because no improvement was possi-
In addition to running the probit models, we estimated a ble for those who reported the highest level of financial
series of ordered probit models. Using the participants’ knowledge prior to the program (prei = 4), Equation 3 was
responses to the 5-point scale, we created a three-category estimated for the sample restricted to those with prei < 4.
dependent variable that was equal to 1 if “much better,” 2 A significant and positive α would suggest that, for a given
if “a little better,” and 3 if “about the same or worse.” The level of prior financial skill, an additional lesson would
coefficients obtained from the ordered probit were similar increase the probability that participants anticipated an
to those for the probit. The probit, however, was found to improvement in their financial behaviors upon completing
be a better fit for the distribution of the data. The findings the program.
from the probit models are presented in the results section;
the findings from the ordered probits are available from Results
the authors upon request. Tables 4 and 5 present the estimation results for (a) the
likelihood that the program had a positive impact on over-
Model 2: Changes in Specific Financial Behaviors all behavior and (b) the likelihood that the program re-
As previously mentioned, participants were asked to sulted in improvement in specific financial behaviors.
evaluate their financial practices in five behavioral cate- Coefficients, standard errors, and marginal effects are
gories using a 4-point scale to indicate how often they presented in both tables. Marginal effects were calculated
engaged in each behavior prior to the program and how at the sample means. Recall that we were unable to run
often they planned to engage in each behavior following separate regressions for the agents and clientele due to data
the program. Responses ranged from almost always to constraints related to the limited variation in the number of
almost never. Using this information, we constructed a lessons completed by the agents. For this reason, models
binary dependant variable to indicate improvement for were estimated for (a) the pooled sample of agency staff
each behavioral category. Specifically, Yi equaled one if and clientele and (b) the clientele-only sample. Note that
posti > prei and 0 otherwise. Prei and posti were the self- for the pooled regression in Table 4 the dummy variable
reported levels of financial practice of the i t h participant for clientele was insignificant. This finding suggested that
before the program and the anticipated level of financial the impact of the program on the behaviors of the clientele
practice after the program, with the higher value denoting was not significantly different from that of the agents,
more desirable financial behaviors.2 Because the partici- providing empirical support for the pooled model. Also,
pant’s prior skill levels could affect the program’s impact, recall that for Table 5 the sample was reduced to the par-
dummy variables that controlled for financial behaviors ticipants who had less-than-perfect financial management
prior to the program were included in the model such skills prior to the program (prei < 4).
that
Overall Program Impact
Pr(Yi = 1 | Lessonsi, Xi, T, prei) = (3) Table 4 shows that, although the coefficient on the total
Φ(α · Lessonsi + ∑ θjI [prei = j] + Xiβ + Tδ), number of lessons completed was positive for both the
j pooled and clientele-only samples, it was only significant
where Φ(.) was the standard normal distribution at conventional levels for the pooled sample. The marginal
function. effect of an additional lesson at the mean for the pooled
sample was 1.5 percentage points. This suggested that, in

Financial Counseling and Planning Volume 17, Issue 2 2006 37

Table 4. Probit Regressions for the Positive Overall Program Effect

Pooled sample Clientele-only sample

Variables (n = 589) (n = 161)

Coeff (SE) dY/dX Coeff (SE) dY/dX

Total number of lessons 0.135 (0.053)** 0.015 0.191 (0.188) 0.011
Age
25-34 0.430 (0.315) 0.044 0.786 (0.566) 0.039
35-44 -0.048 (0.310) -0.006 0.182 (0.554) 0.009
45-54 0.232 (0.332) 0.024 0.932 (0.771) 0.029
55 or more 0.199 (0.413) 0.020 -0.062 (0.846) -0.004
Female -0.486 (0.291)* -0.042 -0.913 (0.687) -0.027
Education
Less than high school 0.100 (0.326) 0.011 -0.002 (0.469) -0.000
Some college 0.513 (0.251)** 0.049 0.656 (0.549) 0.028
Bachelor’s degree 0.160 (0.250) 0.017 0.239 (0.891) 0.011
Graduate degree -0.208 (0.281) -0.026 -0.354 (1.025) -0.027
Number of children
1 0.456 (0.251)* 0.043 0.960 (0.643) 0.032
2 0.250 (0.235) 0.026 1.132 (0.578)* 0.048
3 or more 0.044 (0.253) 0.005 0.916 (0.563) 0.039
Other adult members -0.355 (0.228) -0.036 -0.553 (0.490) -0.029
Personal monthly income
$500-749 0.159 (0.363) 0.016 -0.174 (0.510) -0.011
$750-999 0.631 (0.405) 0.048 0.059 (0.650) 0.003
$1,000-1,249 0.120 (0.355) 0.013 0.110 (0.714) 0.006
$1,250-1,499 0.707 (0.371)* 0.056 2.354 (0.964)** 0.042
$1,500 or more 0.306 (0.331) 0.035 1.097 (0.693) 0.036
Other income sources -0.405 (0.193)** -0.048 -0.851 (0.449)* -0.073
Chicago area -0.419 (0.206)** -0.045 -1.462 (0.718)** -0.042
Clientele 0.176 (0.341) 0.019 … … …
Year dummies (1998-2000) Yes Yes
Constant 1.228 (0.686)* 2.980 (1.320)**
Predicted probability at X-bar 0.943 0.977
Pseudo R-squared .182 .434
Log likelihood -156.800 -38.357

Note. We also estimated ordered probits using the dependent variable in three categorical responses, and the coefficients
were very similar to the findings for the probits. Base categories include age (24 or less), high school or GED, no child-
ren, and personal income ($499 or less). In the clientele-only regression, a dummy variable for 1998 is dropped from the
regression to avoid perfect collinearity. Robust standard errors are reported, and marginal effects are calculated at the mean
values.
*p < .10. **p < .05.

38 Financial Counseling and Planning Volume 17, Issue 2 2006

terms of overall program impact, an additional lesson may ment in their financial behaviors, suggesting difficulties in
have resulted in a more positive improvement in overall financial education for the working poor in urban settings.
financial behavior for agency personnel than for clientele.
However, the magnitude of the effect of the lessons for the Changes in Specific Financial Behaviors
pooled sample was not large. An additional lesson at the Table 5 presents the regression results for the five financial
sample mean (6.7 lessons) increased the likelihood that behaviors. The coefficients and marginal effects are pre-
participants experienced an improvement in their overall sented for the number of lessons (α) and the indicator
financial behavior by 1.6%, off a baseline probability of variables that control for levels of pre-program behavior
94.3%. (θ1 and θ2). θ3 was the omitted base category and is there-
fore not included in the table. The findings showed that the
We considered the possibility that the effect of the number effect of the number of lessons was insignificant for all of
of lessons may not have been linear and explored alterna- the behaviors except comparison shopping. Controlling for
tive specifications by including a quadratic term for the demographic and economic characteristics, as well as for
number of lessons. The probit coefficient for the quadratic the level of comparison shopping before the program,
term was insignificant. To determine if the specific lessons additional lessons significantly increased the probability
that the participant completed had an impact on his/her that a participant reported an improvement in comparison
behaviors, we also estimated the model using dummy shopping by 4.7 percentage points. Thus, the number of
variables for individual lessons instead of a single variable lessons did not appear to play a significant role in improv-
for the total number of lessons. The results showed that the ing specific financial management behaviors. We suspect
quantity of lessons received was more important than the this was because the number of lessons completed may
specific type of lesson. In fact, there was little evidence have been determined endogenously with the unobserved
that specific types of lesson resulted in improved financial traits underlying participants’ ability to improve behaviors.
behaviors. That is, the number of lessons may have been negatively
correlated with self-perceived pre-program behaviors.
With respect to factors other than the number of lessons, Also, once prior level was held constant, those who were
we found that for the pooled sample females were signifi- the least likely to be able to improve their financial behav-
cantly less likely than males to report an improvement in iors may have been more likely to be taught more lessons.
their overall financial behavior (p ≤ .10). Given the small If this was the case, the program impact (as measured by
number of male participants in the sample, however, this the number of lessons completed) may have been absorbed
finding may not have been representative of the low- in θ and, therefore, may not have appeared in α.
income male population as a whole. With respect to educa-
tion, only the category “some college” was positive and The significance of the coefficients for pre-program be-
significant (p ≤ .05), suggesting the program effect was havior levels (θ1 and θ2) showed that the probability of
greatest for those with some college education. improvement was larger for participants who started out
with poorer financial practices. This result was consistent
Participants with children were somewhat more likely than with existing findings from evaluations of employer-
participants without children to experience an overall provided financial education programs (Bayer et al., 1996;
improvement (p ≤ .10), which implied that financial educa- Bernheim & Garrett, 2003). For instance, participants who
tion program for adults may indirectly help children in almost always ran out of money before the program were
poor families. Compared to the lowest income category 44.1 percentage points more likely to improve their budg-
(less than $499 per month), higher levels of income in- eting after they attended the lessons than those who had
creased the probability of a positive program impact. similar demographic and economic characteristics but
However, the income bracket ($1,250-1,499 per month) seldom ran out of money. Similar results were found for
had a significant coefficient (p ≤ .10 for the pooled sample communicating with family members about money, paying
and p ≤ .05 for the clientele-only sample). For both the bills on time, filing consumer complaints, and comparison
pooled and clientele-only samples, access to other house- shopping, which suggests that participants who lacked
hold income sources as well as residing in the Chicago financial skills prior to the program may have been more
metropolitan or surrounding areas significantly decreased likely to gain from the program. As with the pooled sam-
the likelihood that participants reported overall improve- ple, the coefficients for low pre-program status (θ1 and θ2)

Financial Counseling and Planning Volume 17, Issue 2 2006 39

Table 5. Improvements in Specific Financial Behaviors, Reduced Sample

Pooled sample Clientele-only sample

Behaviors
Coeff (SE) dF/dX Coeff (SE) dF/dX
Do not run out of money
α -0.014 (0.062) -0.004 -0.081 (0.187) -0.010
θ1 1.483 (0.340)*** 0.441 2.462 (0.794)*** 0.288
θ2 1.201 (0.222)*** 0.357 2.434 (0.714)*** 0.285
n 365 122
lnL -171.929 -28.742
Talk with family about money
α 0.010 (0.054) 0.004 -0.172 (0.122) -0.069
θ1 1.430 (0.199)*** 0.566 1.632 (0.445)*** 0.650
θ2 0.861 (0.153)*** 0.341 0.869 (0.395)** 0.346
n 482 122
lnL -283.685 -63.074
Do not pay bills late
α 0.028 (0.070) 0.008 -0.003 (0.147) -0.001
θ1 1.296 (0.363)*** 0.358 0.649 (0.497) 0.183
θ2 0.880 (0.276)*** 0.243 0.745 (0.494) 0.210
a
n 292 99
lnL -133.379 -38.831
Complain when having
consumer problems
α 0.094 (0.058) 0.037 -0.001 (0.109) -0.000
θ1 1.095 (0.197)*** 0.434 0.942 (0.444)** 0.311
θ2 0.551 (0.174)*** 0.218 0.453 (0.390) 0.149
n 477 122
lnL -278.658 -58.478
Compare prices and quality
before buying
α 0.160 (0.077)** 0.047 0.171 (0.150) 0.044
θ1 1.451 (0.361)*** 0.426 1.663 (0.589)*** 0.423
θ2 0.399 (0.169)** 0.117 0.811 (0.448)* 0.207
b
n 349 89
lnL -170.208 -34.721
Note. The sample was reduced to those who did not respond “almost always” for their pre-program behavior. The omitted
category for θ’s was prei = 3 (θ3). Coefficients were obtained by controlling for age, gender, education, number of children,
presence of other adult members, personal income, Chicago area indicator, clientele indicator, and the year fixed effects.
Estimated coefficients and marginal effects for the control variables are available upon request.
a
Two control variables (personal income $1,000-$1,249 and the year dummy for 1998) were dropped from the regression to
avoid perfect collinearity. bTwo control variables (age 55 or more and the year dummy for 1998) were dropped from the
regression to avoid perfect collinearity.
*p < .10. **p < .05. ***p < .01.

40 Financial Counseling and Planning Volume 17, Issue 2 2006

were positive for the clientele-only sample, but the results payments from the program, they were less likely than
tended to be somewhat less significant. agency staff to be in a financial position that would allow
them to alter their ability to pay their bills on time. In other
Discussion words, agency staff that missed bill payments may have
The results of this study have useful implications for just been lacking financial management skills, whereas
financial professionals and educators who are evaluating clients who missed payments may have been more likely
financial education programs that target low-income to be financially destitute.
populations. The results of this study provide some evi-
dence that the amount of financial education received (i.e., It is also interesting to note that running out of money is
number of lessons) may result in an overall improvement a behavior for which the pre-program level was highly
in financial behavior. However, the findings seem to significant for both the pooled and clientele-only samples.
indicate that the prior level of financial experience may However, where one might expect to find a larger marginal
matter more than the number of lessons completed. Sig- effect, the marginal effect was lower for the clientele-only
nificantly greater impacts on behavior were observed for sample because they were more likely to start with lower
participants with lower levels of financial behaviors prior levels of financial knowledge and thus had more room to
to the program, suggesting that the program was effective improve. However, like paying bills on time, an individ-
in reaching participants who were most in need of finan- ual’s ability to not run out of money is also related to the
cial education, including agency staff who reported poor financial situation of the participant. In this case, the
pre-program behaviors. Note, however, that the same self- agency staff were likely to be better off financially than the
reported characterization of pre-program behavior by clientele, and thus in a better position to change behaviors
agency staff and clientele (i.e., the same prei) may have related to their financial holdings.
represented different levels of actual behavior, especially
if agents had higher levels of pre-program skills and Overall, these findings suggest that financial education
knowledge than clients. programs may want to distinguish between behaviors that
can more easily be changed in the short run and behaviors
The results of this study also suggest that the program had that require more fundamental changes in other aspects of
the largest impact on those financial behaviors that could participants’ lives before they can be realized. Financial
most readily be altered in the short run. For instance, the education programs that focus solely on behavioral goals
impact of the number of lessons was significant for com- that participants have little chance of implementing in the
parison shopping—a behavior participants could immedi- short term may run the risk of becoming irrelevant to their
ately improve regardless of their current financial circum- target audience. Participants may view the goals of the
stances. Moreover, the largest marginal effects for pre- program as unattainable; some may even become discour-
program behavior of the clients were found for those aged and not take any action to change their behaviors.
behaviors that could most readily be changed after the
program (i.e., comparison shopping and talking with Given our findings, financial professionals and educators
family about money), instead of those behaviors that were need to more carefully assess how knowledge translates
dependent on the participant’s financial situation (running into behavior change for low-income populations and the
out of money and paying bills on time) or personal circum- instructors who deliver the programs. It is critical that
stances (dealing with consumer problems). researchers select outcomes and indicators that are appro-
priate to the financial capabilities of their target audiences.
Interestingly, neither the number of lessons nor the level of It may be that some individuals, because of their particular
pre-program behavior had a significant impact on clientele financial situation, are unable to change certain financial
participants paying their bills on time. However, for the behaviors no matter how much financial education they
pooled sample, poor pre-program behavior did result in a receive. Thus, if financial education does not result in
significant increase in paying bills on time. A plausible behavior change, it may not be that the program is ineffec-
explanation is that, on average, the clientele may have tive. It may be that inappropriate outcomes and indicators
been worse off financially than the agency staff because were selected. Researchers may want to focus less on
their actual financial position probably did not change as outcomes tied to individuals’ financial situations and more
an immediate result of the program. Thus, even though on whether individuals are able to make sound financial
clientele may have learned of the costs associated with late decisions regardless of their financial situation. They may

Financial Counseling and Planning Volume 17, Issue 2 2006 41

also want to use a wider set of program outcomes to ensure Extension educators also noted that there were agents who
that the positive effects of these programs are not underes- were experiencing financial problems and having difficulty
timated. managing their own finances. One extension educator
commented:
In addition, financial professionals and educators may
want to re-examine the link between financial knowledge A number of the agency staff who deal with low-
and behavior change, especially for program instructors. income clients are low paid and money is tight for
One might assume that instructors delivering programs to them, and many are struggling financially themselves.
low-income audiences are knowledgeable and experienced
financial educators. However, informal interviews con- Thus, the agents and clientele in our sample were more
ducted with extension educators, who trained the agents similar than one might initially think. In light of these
for this particular program, seem to reveal that this may informal interviews, our finding of no significant differ-
not exactly be the case. It was reported that levels of ence in overall financial behavior between agents and
financial knowledge and experience varied significantly clients for the pooled model may not be that surprising
for the agents—from having no financial education experi- (i.e., recall that in Table 4 the dummy variable for clientele
ence to several years of experience. Extension educators was found to be statistically insignificant). Although the
reported that for some agents the training sessions served agents may have been more financially knowledgeable
as a “refresher course” or as a “reinforcement” of basic than the clients on average, a significant number of the
financial concepts. However, others were seeing some of agents may have been struggling financially, even though
the information for the first time. One extension educator they had more education and income than their clients.
commented: Thus, one must consider the possibility that providing
more educational knowledge (“number of lessons”) may
We thought they knew this stuff when they came in, but not necessarily be the appropriate catalyst to motivate
we got agents who wrote in the comments that they individuals to change their financial behaviors. It may be
felt they learned something from the program. They that the level of financial experience matters more, as
said they could use this information for themselves as participants with the poorest pre-program behaviors gener-
much as for their clients. ally reported the greatest improvement.

At the end of the program, agents were asked to comment As with most program evaluations, one needs to be cau-
on what they learned from the program and how they tious in regarding these findings as conclusive. First, our
planned to use the information and materials they received. clientele-only sample is somewhat small for making infer-
In general, several reported that they felt “more familiar” ences about the program’s impact on the financial behav-
and “more comfortable” with the information, especially iors of the low-income population as a whole. Recall,
related to budgeting, savings, and credit management. One however, that the clientele sample is fairly representative
agent commented: of the clientele as a whole for all the agencies that partici-
pated in a train-the-trainer program. Second, the treatment
I learned new avenues to save money personally and effect in our model is identified not by variation in the
also new techniques and strategies to help my clients outcomes of those who underwent the treatment versus
budget and learn new skills to save money and start those who did not (i.e., participants versus non-parti-
savings or checking accounts. cipants), but rather by the variation in the outcomes of
participants who underwent varying treatment intensities
Another stated: (measured by the number of lessons) and who started out
with different pre-program behaviors. As a result, the
I learned principles of effective money management effects we found may not reflect true treatment effects but
that will help me to become more organized in devel- rather marginal effects conditional on some level of ex-
oping a system that I am comfortable with (a) in perience and participation in the program.
tracking my expenditures and (b) investing wisely,
consistently, and enthusiastically. The info on credit Ideally, we would have liked to have had a control group
card smarts. I will use this with clients as well as in and a larger sample size so we would not have had to pool
my personal life. the data. It would have been of particular interest to exam-

42 Financial Counseling and Planning Volume 17, Issue 2 2006

ine whether changes in financial behavior would have survey of households. Journal of Public Economics,
occurred even without the program. Unfortunately, be- 87(7/8), 1487-1519.
cause of resource constraints, this level of “rigor” is not Bernheim, B. D., Garrett, D. M., & Maki, D. M. (2001).
feasible for most non-profit organizations. As researchers Education and saving: The long-term effects of high
and funders push for more rigorous evaluations, it is im- school financial curriculum mandates. Journal of
portant that the profession continue to recognize the value Public Economics, 80(3), 435-465.
of more traditional evaluation methods such as RPTs, pre- Boyce, L., & Danes, S. M. (2004). Evaluation of the NEFE
and post-tests, and qualitative surveys that collect best High School Financial Planning Program, 2003-2004.
practices and success stories. The value of these types of Retrieved November 1, 2005, from http://nefe.org/
methods should not be overlooked. As this study has hsfppportal/includes/main/home.asp?portal=4
shown, RPTs can still provide useful insight into the Braunstein, S., & Welch, C. (2002, November). Financial
effectiveness of financial education. The self-reported, literacy: An overview of practice, research, and
retrospective measures used in this study, though imperfect policy. Federal Reserve Bulletin, 445-457.
indicators of actual and anticipated changes in behavior, Clancy, M., Grinstein-Weiss, M., & Schreiner, M. (2001).
may still serve as good indicators of the program’s impact. Financial education and savings outcomes in individ-
These indicators reflect changes not only in participants’ ual development accounts (Working Paper No. 01-2).
level of knowledge but also in participants’ confidence in St. Louis, MO: Washington University, Center for
their skills and in their ability to shape their future behav- Social Development.
iors. If participants are feeling better about their financial Fox, J., Bartholomae, S., & Lee, J. (2005). Building the
situation at the end of the program, this is a positive reflec- case for financial education. The Journal of Consumer
tion of what the program is trying to achieve. Affairs, 39(1), 195-214.
Garman, E. T., Kim, J., Kratzer, C. Y., Brunson, B. H., &
From a program delivery standpoint, programs that focus Joo, S. H. (1999). Workplace financial education
on more basic and fundamental decision-making skills improves personal financial wellness. Financial
may give participants the confidence they need to take the Counseling and Planning, 10(1), 79-88.
first step towards behavior change. These types of pro- Hilgert, M. A., Hogarth, J. M., & Beverly, S. G. (2003,
grams may not need to have numerous and complicated July). Household financial management: The connec-
financial lessons. A few general lessons in basic financial tion between knowledge and behavior. Federal
education are likely to result in positive outcomes. Reserve Bulletin, 309-322.
Hirad, A., & Zorn, P. M. (2001). A little knowledge is a
References good thing: Empirical evidence of the effectiveness of
Anderson, S. G., Zhan, M., & Scott, J. (2004). Targeting pre-purchase homeownership counseling (Low-
financial management training at low-income audi- Income Homeownership Working Paper Series,
ences. The Journal of Consumer Affairs, 38(1), 167- LIHO-01.4). Cambridge, MA: Harvard University,
177. Joint Center for Housing Studies. Retrieved
Baron-Donovan, C., Wiener, R. L., Gross, K., & Block- November 1, 2005, from http://www.jchs.harvard.edu/
Lieb, S. (2005). Financial literacy teacher training: A publications/homeown/liho01-4.pdf/
multiple-measure evaluation. Financial Counseling Hogarth, J. M. (2002). Financial literacy and family &
and Planning, 16(2), 63-75. consumer sciences. Journal of Family and Consumer
Bayer, J. P., Bernheim, B. D., & Scholz, J. K. (1996). The Sciences, 94(1), 14-28.
effects of financial education in the workplace: Hogarth, J. M., Beverly, S. G., & Hilgert, M. (2003).
Evidence from a survey of employers (NBER Working Patterns of financial behaviors: Implications for com-
Paper No. 5655). Cambridge, MA: National Bureau of munity educators and policy makers. 2003 Federal
Economic Research. Retrieved September 15, 2005, Reserve System Community Affairs Research
from http://www.nber.org/papers/w5655 Conference. Retrieved November 1, 2005, from http://
Bell, E., & Lerman, R. I. (2005). Can financial literacy www.federalreserve.gov/communityaffairs/national/
enhance asset building? Opportunity and Ownership CA_Conf_SusCommDev/pdf/hogarthjeanne.pdf
Project, The Urban Institute, 9(6), 1-7. Howard, G. S. (1980). Response-shift bias: A problem in
Bernheim, B. D., & Garrett, D. M. (2003). The effects of evaluating interventions with pre/post self reports.
financial education in the workplace: Evidence from a Evaluation Review, 4(1), 93-106.

Financial Counseling and Planning Volume 17, Issue 2 2006 43

Kim, J., & Garman, E. T. (2003). Financial education and Schreiner, M., Clancy, M., & Sherraden, M. (2002).
advice changes worker attitudes and behaviors. Saving performance in the American dream demon-
Journal of Compensation and Benefits, 19(5), 7-13. stration: A national demonstration of individual
Kim, J., Garman, E. T., & Sorhaindo, B. (2003). development accounts. St. Louis, MO: Washington
Relationships among credit counseling clients’ finan- University, Center for Social Development.
cial well-being, financial behaviors, financial stressor Shelton, G. S., & Moss, J. K. (2002). Development of a
events, and health. Financial Counseling and Plan- national certification examination for homeowner
ning, 14(2), 75-87. educators and housing counselors. Financial
Lamb, T. (2005). The retrospective pretest: An imperfect Counseling and Planning, 13(2), 83-92.
but useful tool. The Evaluation Exchange, XI(2), 18. Shockey, S. S., & Seiling, S. B. (2004). Moving into
Retrieved October 1, 2005, from http://www.gse. action: Application of the transtheoretical model of
harvard.edu/hfrp/eval/archives.html behavior change to financial education. Financial
Lamb, T. A., & Tschillard, R. (2005, Fall). Evaluating Counseling and Planning, 15(1), 41-52.
learning in professional development workshops: Staten, M. E., Elliehausen, G., & Linquist, E. C. (2002).
Using the retrospective pretest. Journal of Research in The impact of credit counseling on subsequent bor-
Professional Learning. Retrieved November 28, 2005, rower credit usage and payment behavior
from http://www.nsdc.org/library/publications/ (Monograph #36). Washington, DC: Georgetown
research/index.cfm University, Credit Research Center. Retrieved No-
Lyons, A. C. (2005). Financial education and program vember 1, 2005, from http://www.msb.georgetown.
evaluation: Challenges and potentials for financial edu/prog/crc/pdf/M36.pdf
professionals. Journal of Personal Finance, 4(4), 56- Zhan, M., Anderson, S. G., & Scott, J. (in press). Financial
68. management knowledge of low-income populations:
Lyons, A. C., & Scherpf, E. (2004). Moving from un- Characteristics and needs. Journal of Social Service
banked to banked: Evidence from the Money Smart Research.
Program. Financial Services Review, 13(3), 215-231.
Lyons, A. C., Palmer, L., Jayaratne, K. S. U., & Scherpf, Endnotes
1
E. (2006). Are we making the grade? A national The advantages and disadvantages of using a retrospec-
overview of financial education and program evalua- tive pre-test are discussed in the section on data collection.
2
tion. The Journal of Consumer Affairs, 40(2), 208- The incidences of negative changes were very few and
235. treated as no improvement.
National Endowment for Financial Education. (2004).
Motivating Americans to develop constructive finan- Acknowledgements
cial behaviors. A White Paper from the Think Tank The authors graciously acknowledge assistance from the
Sponsored by the National Endowment for Financial University of Illinois Extension Consumer and Family
Education. Financial Counseling and Planning, 15(2), Economics Team. Specifically, we thank Debra Bartman,
39-50. Ellen Burton, Karen Chan, Mary Ann Fugate, Patricia
Prochaska, J. O. (1979). Systems of psychotherapy: A Hildebrand, Jennifer Hunt, Linda Crawl Jackson, Evelyn
transtheoretical analysis. Homewood, IL: Dorsey. Prasse, Kathy Reuter, Lois Smith, Kathy Sweedler, and
Prochaska, J. O., & DiClemente, C. C. (1983). Stages and Susan Taylor. All views expressed in this paper are those
processes of self-change in smoking: Toward an of the authors and do not reflect the views or policies of
integrative model of change. Journal of Consulting the University of Illinois Extension or any members of its
and Clinical Psychology, 51(3), 390-395. staff.

44 Financial Counseling and Planning Volume 17, Issue 2 2006

Appendix
Let z*, a random variable signifying the latent financial
effectiveness of an individual, be distributed with the
cumulative distribution function G(z). Although z* is
unobservable, we observe its ordered ratings, pre and post,
which are defined as

pre (or post) = 1, if z* < λ1

= 2, if λ1 ≤ z* < λ2
= 3, if λ2 ≤ z* < λ3
= 4, if z* ≥ λ3
where λ1 < λ2 < λ3. For simplicity, assume pre ⊥ post (it
can also be shown that the implication does not change
when the two are correlated). The probability of improve-
ment, conditional on the financial effectiveness, can be
shown as

Pr( post − pre > 0 | pre = j ) = 1 − g ( λ )

j

for j = 1, 2, 3, and 0 for j = 4. Since G(.) increases in j,

∂ Pr( post − pre > 0)

<0
∂pre

In other words, the chances of improvement decline as pre-

program status increases, other things being equal.

Financial Counseling and Planning Volume 17, Issue 2 2006 45