Title III and Title V Performance
Measurement System
Agencies
and organizations increasingly
rely on performance measurement
strategies to organize, manage,
and evaluate programs. For this
multiyear project, ICF's education
research experts developed a Web-based
performance measurement system for
the Title III and Title V programs
on behalf of the U.S. Department
of Education. Title III and Title
V programs make available financial
support to colleges and universities
that serve substantial proportions
of minority and low-income students.
Working with directors
of the Department of Education's
Institutional Development and Undergraduate
Education Service (IDUES), IDUES
program staff, and consultants, we
designed and implemented
a fully Web-based annual performance
measurement (APR) system for Title
III and Title V programs. This very
successful system provides information
for program monitoring, performance
measurement, and communication about
the IDUES program to Congress and
other stakeholders. Current phases
of the project include funding for
three separate tasks: continuation
of the APR system for the data collection
and reporting cycle, analysis and
reporting of recently collected data,
and development and initiation of
the first Web-based final performance
report to be completed electronically
by grantees in the final year of
their funding cycle.
Evaluation of the Impact
of State and Federal Support of
Advanced Placement Programs
The College Board established the
Advanced Placement (AP) Program to
offer rigorous college-level academic
courses to high school students.
Beginning in 1983, states developed
legislative initiatives to increase
participation in AP programs, particularly
among low-income and minority students.
Beginning in the 1998-1999 school
year, federal support to 34 states
further encouraged the participation
of low-income and minority students
by awarding grants to provide funds
to pay exam fees or provide support
for other types of activities that
foster participation. The evaluation
assessed the effects of state and
federal expenditures on AP program
participation and performance. The primary concern was whether the gaps in
participation and performance for
low-income and minority students
were reduced over time in relation
to federal and state funding. Analyses
of AP data for the 1990-1991 through
1999-2000 school years determined
changes in patterns of participation
and student performance. Case studies of AP programs in five states where financial support has been in place for varying amounts of time provided in-depth information
about strategies for recruiting and supporting low-income and minority students.
The findings informed the development of College Board and AP policy, influenced
advocacy efforts for increasing the level of federal and state support for
the AP program, and augmented current research on best practices.
Measures of Officer Accession
Quality at Colleges and Universities
For the U.S. Army
Accessions Command (USAAC), ICF
is developing a college
marketing model to assist the Department
of Defense (DoD) and the Services
(Army, Navy, Air Force, Marines)
in allocating officer recruiting
resources and in determining the
best national colleges and universities
in which to place Reserve Officers'
Training Corps (ROTC) programs. The
college marketing model will use
individual- and institutional-level
variables to predict officer performance
and retention, and to examine the
capability of various ROTC incentive
packages to recruit and enroll quality
officer candidates. ICF will collect
data from students at 81 colleges
and universities on their perceptions
of and likelihood to enroll in ROTC.
The college marketing model will
be used by DoD to answer important
policy and resource decisions.
Support for Adult Literacy
Services and Assessment
The U.S. Department of Education,
Office of Vocational and Adult Education
(OVAE), has embarked upon an initiative
aimed at increasing capacity among
faith- and community-based organizations
(FCBO) that deliver adult literacy
services, particularly as it concerns
FCBOs that do not receive federal funds
and have limited access to technical
support. Working collaboratively with
OVAE and FCBO experts, ICF (under
subcontract to MPR Associates, Inc.),
has been funded by the Department
of Education to lead this initiative.
The initiative will
emphasize supporting the efforts of
local programs to satisfy
important instructional, educational,
and assessment needs in adult literacy
programs. In particular, the
initiative will produce an electronic
toolkit for adult literacy program
instructors and administrators. The
toolkit, which will be made available
via the Internet, will identify high-quality
assessment training materials available to FCBOs. It will provide
easy-to-understand guidance on important
service delivery issues, such as
identification and selection of appropriate
curriculum, good instructional delivery,
and how to judge effectiveness of
the curriculum and instruction through
student assessment. The information
will be designed, organized, and
presented in a manner that makes
it suitable for self-directed learning
without the aid or support of on-site
trainers.
Lead Safety and Lead Awareness
Education for Women, Infants, and
Children (WIC), and Head Start Programs
For the U.S. Environmental Protection
Agency (EPA), ICF researched,
wrote, designed, and produced a suite
of educational materials on lead
safety and lead awareness targeting
recipients of the Women, Infants,
and Children (WIC) program. The goal
of this outreach effort is to educate
WIC recipients on the importance
of a healthy diet and its ability
to reduce the effects of lead poisoning.
ICF also worked closely
with EPA and Head Start to create
a suite of materials targeted toward
parents of children in Head Start
programs across the country. We designed
a brochure to appeal to parents and
developed fact sheets for staff members
to use in reinforcing the message
with parents and to encourage blood
tests to check lead levels in at-risk
children. All materials were co-branded
by EPA and the National Head Start
Association and were provided on
the Web, as well as in hard copy.
Virginia Community College System (VCCS) Professional Development Program Assessment
Since the early 1990s, the Virginia Community College System (VCCS) has been committed to building a system-wide comprehensive professional development program to enhance the educational services and programs available to full-time and adjunct faculty, administrative faculty, and classified staff from its 23 community colleges on 40 campuses across the Commonwealth. This professional development program has provided, and continues to provide, support for all areas of professional development, including academic/functional discipline, instructional, career, and organizational development. ICF's assessment helped to articulate and describe the processes currently being used to accomplish the program mission, identify and assess the current professional development needs of the VCCS, define the desired program outcomes and develop strategies for measuring these outcomes, identify promising practices of selected program components, and assess the impact of the program on various outcomes.
Our professional development program assessment used a mixed-method approach to data collection, including a review of extant materials, interviews, focus groups, and a survey to provide rich and systematic data about program processes, outcomes, and participant perceptions. The development and revision of a logic model guided the articulation of the program. ICF assessed participant perceptions of and satisfaction with the system-level professional development program, the kind of learning that has occurred, the extent to which this learning has changed participant behaviors, and program outcomes. We also provided VCCS with a set of measures and strategies to conduct a more intensive, longitudinal evaluation to assess the impact of the program on teaching and student outcomes. Recommendations for program improvements were derived from the results of the assessment, and the VCCS leaders used the results from this assessment to guide program improvements and to justify resource allocations and requests for increased funding.
Support to the National Community
Technology Centers (CTCs)
In collaboration with seven partner
organizations and led by the Education
Development Center, ICF
provided technical assistance, training,
regional development, and other services
to the nation's National Community
Technology Centers (CTC) through
the America Connects Consortium (ACC).
ACC is funded under a contract with
U.S. Department of Education's
Office of Vocational and Adult Education,
with additional matching funding
and in-kind support from a variety
of other sources. ICF's
primary role in this collaboration
was to provide technical assistance
and training to CTC staff on workforce
development topics, as well as
marketing and design services on
collateral outreach materials.
IRS Education and Training Services
ICF International provided education
and training services
to help the U.S. International Revenue
Service (IRS) retool its workforce
and improve performance. ICF offered
the IRS options to educate employees
in key competencies through both
college-level courses and custom-developed
training programs available on-line,
in the classroom, or through computer-based
training. In addition, ICF provided
the IRS with many other services,
including the creation of educational
needs assessments, curriculum design,
train-the-trainer programs, course
tracking, evaluation, and research.
To meet IRS needs, ICF assembled
the IRS Consortium for Learning and
Workforce Development by forging
partnerships with an initial consortium
of 16 top-tier institutions of higher
learning and by leveraging its alliances
with leading e-learning technology
and content providers. The consortium
offered myriad education and training-related
services in 13 identified competency
areas, ranging from taxation and
compliance to information technology,
communications, labor relations,
and customer service.
OPM Training and Management Assistance
(TMA) Program
Through our OPM
federal contract vehicle,
ICF is helping federal
agencies with their training needs.
ICF's adult education
services include analysis, design,
development, implementation, and
evaluation of instructional products,
regardless of subject area. This
includes instructor-led materials,
print-based instructional materials,
technology-based training products,
knowledge management systems, learning
strategies and plans, consulting,
learning systems development and
integration, and learning management
systems. ICF provides
distance learning, instructor-led
training, and blended learning
solutions, ranging from multimedia
and satellite-based training, and
learning management and electronic
performance support systems, to
job performance aids, management
coaching, and senior leadership
seminars.
Our solutions incorporate cutting-edge
multimedia technologies to deliver
high-quality learning interactions.
Clients to date include U.S. Office
of Personnel Management (OPM), National
Science Foundation (NSF), Department
of Homeland Security Immigration
and Naturalization Service (DHS/INS),
Department of Defense (DOD), Secret
Service (USSS), Agency for International
Development (USAID), Department of
Energy (DOE), Department of the Interior
(DOI), Department of Homeland Security
Forest Service (DHS/USFS), Securities
and Exchange Commission (SEC), Department
of Health and Human Services (HHS),
Program Support Center (PSC), and
Substance Abuse and Mental Health
Services Administration (SAMHSA). |
