WORKING emerged in a number of stages. The
first was recognition that, although there is a growing clamor
to insure that the workforce has a "good work-ethic"
and related traits, there is a notable lack of simple, inexpensive
ways to measure many of the key competencies. The second stage
for WORKING was to assess what specific competencies were
being demanded. Many broad studies (such as SCANS) were analyzed
and compared in terms of how often they called for (or implied
a need for) each of 24 different competencies. This data,
combined with discussions with businessmen and educators,
led to a focus on nine of these competencies which seemed
to rate high on three criteria: particularly important (in
the workplace, and often in the classroom), infrequently or
never covered in available assessment instruments, and/or
likely to be validly assessable in a self-rating format.
These nine competencies were then translated
into specific items through an extensive series of steps.
First, focus groups brainstormed some 412 items covering the
nine areas. These were progressively sorted, reviewed, and
refined (by psychometricians, teachers, employers, employees,
and students) through three rounds into 254, 149, and then
108 items. These were informally piloted with several groups,
reducing the number of possible items to 85.
The 85 items were then combined into a structured
instrument (a forerunner of the final WORKING format) and
formally pilot tested. Sixteen colleges and universities in
eleven states were recruited and were sent field-test packets.
Ultimately, thirteen institutions (rural, suburban, and urban,
two-year and four-year) participated, representing 640 useable
returns. Results were tallied and were compared with teacher
ratings. Employers (for working students) were queried, but
the data, though interesting, was too scant for statistical
use.
Statistical analysis was conducted to determine
which items of WORKING were the strongest predictors, in combination,
in each of the nine areas. This analysis led to reduction
in the number of items from 85 to the final 50-item scales
of WORKING. The statistical analyses revealed several interesting
relationships: high scores correlate with years of work experience
on all but one scale (not teamwork); GPA correlates with scores
on six of the scales (not teamwork, systems thinking, or adapting
to change); and teacher ratings and student self-ratings were
very similar on eight of nine scales (not adapting to change).
All of these results suggest strong, useful information.
WORKING is a statistically valid and reliable
assessment, independently verified by Public
Policy Associates. See the WORKING Technical Manual for
greater detail on the nature of each scale, the evolution
and validation of the instrument, and on the need for the
instrument.
