case study

  Case Studies of Teachers Learning and Applying

Opportunity to Learn Assessment Strategies

In Two Urban Elementary Schools

by

Floraline I. Stevens

1999

Publication Series No. 8
This report is disseminated in part by the Office of Educational Research and
Improvement (OERI) of the U.S. Department of Education through a grant to the Laboratory for
Student Success (LSS) at the Temple University Center for Research in Human Development and
Education (CRHDE). The opinions expressed do not necessarily reflect the position of the
supporting agencies, and no official endorsement should be inferred.


INTRODUCTION
   For several years, opportunity to learn (OTL) research was limited to determining whether
there was content coverage (students covered the core curriculum). Questions about content
coverage arose from such large-scale surveys as NAEP, SIMS, and when researchers were
planning the Third International Mathematics and Science Study (TIMSS). Therefore, in 1993,
OTL was expanded into a conceptual framework that included content coverage and three
additional variables: (1) content exposure—enough time is allocated for in-depth teaching and for
students’ time-on task; (2) content emphasis—topics are selected that are part of core curriculum
and taught to all students; and (3) quality of instructional delivery—lessons presented are coherent
so students are able to understand and use the information earned. All of these variables are closely
related to what teachers do in their classrooms when teaching (Stevens, 1993a), and classroom
activities are connected logically and sequentially with a beginning, middle, and end.
The next step following the development of the OTL conceptual framework was to
transform the research to procedural knowledge. In other words, through a nationwide survey of
teachers, we investigated which OTL assessment strategies were teacher friendly (e.g., used on a
daily basis) and then taught teachers through professional development how to use these strategies
in their schools and classrooms. Results from the survey indicated that the teachers found most of
the OTL assessment strategies to be teacher-friendly or sustainable over time. They did not find the
strategies burdensome to implement. Also, the teachers indicated that they would implement or
were already implementing some of the strategies in their classrooms (Stevens, et al., 1998).
The strategy for OTL workshops was to avoid the replication of traditional professional
development, described as relatively short-term and involving teachers in several hours or days of
workshops that later had a very low teacher implementation level of 15% (Goldenberg &
Gallimore, 1991; Meyer, 1988). Instead, the workshops attempted to adopt the qualities that
several researchers advocated as needed in educational professional development processes,
mainly:
· to be school-wide and context-specific
· to have supportive school principals who endorse the process and encourage change
· to be long-term with adequate support and follow-up, and
· to encourage collegiality (Fullan, 1990; Griffin, 1986; Loucks-Horsley, et al., 1987;
McLaughlin, 1991).
In addition, professional development should focus on ways of thinking and teacher action
rather than behaviors (Gallagher, Goudvis, & Pearson, 1988). Lieberman (1996) states that if
professional development is to enable teachers to really change the way they work, then teachers
must have opportunities to talk, think, try, and hone new practices.
PURPOSE OF THE STUDY
The research seeks to examine the proposition that if professional development focuses on
ways of thinking and teacher action, it will facilitate the ongoing implementation of those OTL
assessment strategies rated as teacher-friendly and very teacher-friendly.
Methods and Procedures
Case study methodology was used to investigate a contemporary phenomenon (teachers
learning and applying OTL assessment strategies) within its real-life context (in two urban
elementary schools) when the boundaries between the phenomenon and the context were not clearly
evident. The research questions were basically how and why (Lin, 1989). For this case study, the
researcher attempted to respond to four questions through observations, on-going dialogue,
interviews with principals, document analysis, and evaluative questionnaires. The questions
included:
1. How did teachers view the OTL assessment strategies after they practiced and
implemented them in their classrooms?
2. How did teachers view the level of teacher-friendliness of the strategies after practicing
and implementing them?
3. Were there obstacles to implementing the strategies in the schools and classrooms?
4. What factors encouraged long-term, ongoing implementation of the OTL assessment
strategies in classrooms; and more specifically, what role did the school principal play in
the implementation and ongoing execution of the strategies?
The researcher acted as the leader for the OTL assessment strategies workshops in the
schools once a month from January through May and continued with follow-up visits in June to
check the level of strategy implementation after the workshops had ended. The schools being
studied were two low-achieving elementary schools in Washington, D.C. The study takes the
research perspective that through case study research, it can include quantitative data (i.e., normreferenced
test scores) and qualitative data (i.e., questionnaires, interviews, etc.). Therefore, data
were collected from the following sources:
· workshop participants who completed end-of-workshop session evaluation forms
· workshop discussions with participants
· end-of-workshop feedback sessions with participants
· interviews with teachers, teacher-facilitators, and school principals
· informal conversations with school staff
· observation of the school environments, and
· examination of school documents.
Background
The two schools chosen were part of the original five schools identified for intervention
because of student low achievement in reading and mathematics. At the behest of the then
Washington, D.C., superintendent of schools, the schools were implementing an intervention model
from Temple University’s Mid-Atlantic Educational Regional Laboratory for Student Success
(LSS). The Community for Learning (CFL) model with its Adaptive Learning Environments
Model (ALEM) component stressed individual learning plans for students, interactive teaching,
arranging space and facilities (e.g., learning centers), on-site professional development, and a fulltime
instruction-focused teacher-facilitator assigned to each school (Wang, 1992).
The investigator was invited to lead workshops about Opportunity to Learn (OTL)
assessment strategies to supplement the work of the schools’ reform efforts. There were six OTL
assessment strategies presented:
1. Using networking and collaborating to improve instructional practices
2. Keeping journals
3. Assessing students’ mastery of skills and concepts
4. Conducting observations for constructive feedback
5. Conducting surveys about teaching practices, and
6. Conducting surveys about school resources needed for effective teaching.
These strategies grew from research about opportunity to learn (Stevens, 1993b; Bailey,
1996; Stevens, et al., 1998). Each teacher and the school administrators were provided with their
own OTL assessment strategies handbook to use at each workshop session and for follow-up work.
Describing the School Environment
School #1: One school was located in the Northwest section of Washington, D.C., within walking
distance of Union Station. It was surrounded on all sides by housing projects. Inside, the school
was very clean and the walls were decorated with students’ work. This school’s students totaled
631 and were principally African American/Black. Most of the 33 teachers were African
American. Sixteen of the teachers were permanent. Auxiliary positions included three physical
education teachers, a counselor, a librarian, and an English as a Second Language (ESL) teacher
for students who spoke Spanish and French. In addition to the school principal, there was an
assistant principal. Many students were part of several generations of families who lived in the
housing projects and attended the elementary school.
The principal described the staff as being very complacent when she had arrived three
years ago. She had perceived the teachers as not wanting to change and some were rebellious
against her push for change. The principal said that she did a lot of talking about change, retooling,
and creating options. Her evaluation is that the majority of the staff has grown and
matured together while others have left. She and her staff have since developed positive working
relationships and understand each other’s philosophies and goals.
However, the principal described the staff as being hurt and angry when the school was
identified for intervention. As part of the intervention process, a teacher-facilitator was assigned to
the school to assist teachers to do a better job through implementing the ALEM component in their
classrooms. After the initial feelings of anger and resentment toward a person assigned to “help”
them, the faculty began to see the positive side of having a teacher-facilitator work with them and
having on-site training provided for them.
The teacher-facilitator described her role in implementing the ALEM component of the
Community for Learning (CFL) school reform intervention as providing a lot of training and
mentoring. She said that the principal viewed her as in charge of instruction, although she doubted
that the principal would verbally say this. She felt that the teachers viewed her as being their
advocate and as being very supportive and helpful to them. The researcher observed that the
teacher-facilitator was very positive about her role and enthusiastic about the help she was
providing to the teachers.
School # 2: The second school was located in the Southeast section of Washington, D.C.,
surrounded on three sides by housing projects. The buildings were over 50 years old but they
appeared to be newer from the outside because of the white brick architecture. It was a very clean
school, no trash or graffiti was visible. The floors of the buildings were shiny from wax and the
walls had attractive displays of children’s school work. Bulletin boards told of the school’s motto
and vision, students were recognized for perfect attendance, and for being student scholars. There
were themes dealing with Africa and African Americans. There were messages of encouragement
and pride for the students and their parents displayed throughout the school. A parents’ learning
center was on the second floor of one of the buildings. The students numbered 525 and most were
African American/Black. There were 24 classroom teachers who were also principally African
American and eight other educators/resource teachers—librarian, counselor, art teacher, science
teacher, physical education teacher, mathematics teacher, music teacher and reading teacher. There
was an assistant principal. Half of the classroom teachers were permanent. Three teachers were
first-year teachers.
The principal came to the school three years ago. She described some of the teachers as
initially temperamental and immature. She felt there were two major instructional problems:
teachers were limited to whole group instruction; and there was a lack of academic progress for
students completing the third grade.
The teacher-facilitator felt that when she first came to the school, the teachers were
suspicious of her, but she is now accepted by over 90% of them. However, a few continue to
periodically influence other teachers to react negatively to her. She stated that she is not sure what
to expect from day to day and must be careful what she says because it is sometimes misconstrued.
She recalled that her first year was “rough” getting to know the principal and the teachers since she
was initially viewed as an “outsider” sent by the school district. The teacher-facilitator stated that
at the end of the first year of installing the ALEM model, the teachers began to feel pride in their
work. They acknowledged that the past year had been good and they were now ready to move on.
The researcher found that the teacher-facilitator at school #2 was very concerned that her efforts as
a facilitator would result in improved academic achievement of the students. The researcher
observed that the teacher-facilitator was frustrated by her perceptions of lack of support from the
school principal. For example, the school principal would not assign a computer to her when new
ones became available and so she had to share the vice-principal’s computer or use her personal
computer at home.
How OTL Assessment Strategies Workshop Sessions Were Conducted
The workshop leader came to the two schools in January of the second year of the
intervention and made monthly visits until the end of May. Teachers were initially wary of the
leader because the principal of each school had accepted the idea of having the OTL workshops,
but had not discussed the workshops with the teachers. When the workshop leader arrived for the
first workshop sessions at the two schools, the teachers were polite while not wholly receptive to
the idea of attending OTL assessment strategies workshops. However, as the sessions continued,
the teachers’ attitudes became more and more positive. Their positive evaluations of the sessions
mirrored their behavior in the sessions. In the beginning, the leader spent a lot of time cultivating
acceptance by reassuring teachers about their professionalism and listening to the teachers cite
their concerns and complaints about the students, the school district, and, at one school, the
principal. Each month, the investigator gave a workshop on one of the assessment strategies.
The first workshop topic was Using Networking and Collaborating to Improve
Instructional Practices. Participants were asked to read the definitions of networking and
collaborating found in their workbooks. Next, appointed teacher-participants were asked to read
aloud the research about networking and collaborating. Then the teachers were asked to form
groups according to their grade level. The first activity was to meet in their groups and list the
achievement problems in their school. Next, they were assigned to read a case study found in the
workbook about a school district with student achievement problems and to review the opportunity
to learn variables (i.e., content coverage, content emphasis, content exposure, and quality of
instructional delivery). Using the case study information and the OTL information, the teachers in
each grade level group were asked to generate and list possible strategies/solutions that address the
reading and mathematics achievement problems in the case study. A representative from each
group was asked to share the information about the strategies/solutions generated within the entire
workshop group.
The investigator observed that the workshop activities were catalysts in having teachers
meet and discuss instructional problems. As the investigator moved from group to group, the
discussions were lively and focused on problem-solving. One teacher commented that this was a
new experience for her and her teacher colleagues to meet solely to discuss issues of instruction.
The workshop leader then assigned a follow-up activity to move teachers from the
workshop activity to their own school’s educational problems. The assignment was for the teachers
to meet within their grade levels and list the achievement problems in their school. They then chose
from among the problems one in particular that their team would work to address. To prepare them
for their grade level meeting, the entire workshop group reviewed Schmoker’s model for an
efficient meeting and they were asked to use this model when they convened their grade level
meetings (Schmoker, 1996). Each grade level was to select one team member to report their work
at the next professional development session.
The investigator observed that by assigning “homework” to the teachers in each of the
grade levels and allowing at least three weeks for them to meet prior to the next workshop, the
possibility of implementation of networking and collaborating outside of the workshop environment
was greatly increased. Also, the knowledge that they would be expected to report back to their
teacher peers increased the need for accountability from the groups. At the next meeting, before
going to a new topic, the group leaders reported their identified problems. There was discussion
about the overlap of problems identified at each school and a decision to share the list with each
school principal for further planning and addressing.
The professional development procedural format followed for each of the monthly session
topics was that teachers learned the definitions of and research on the topic; teachers discussed the
OTL assessment strategy; teachers performed assigned activities or practices during the workshop;
teachers were assigned homework and were given enough time between sessions to implement the
strategy; and teachers reported the results of the OTL assessment strategy’s
practice/implementation at the next professional development session.
The Workshop Leader’s Role in Encouraging the Implementation
of the OTL Assessment Strategies
It was observed that over time a very positive rapport evolved among the workshop leader,
the school principals, the teacher-facilitators, and the teachers. Part of the success of the
workshops could be attributed to the workshop leader having been a school teacher. She could
relate her own school experiences with those of the teachers in the workshops. When the
discussions or teachers’ opinions were not beneficial to good instructional practices or to building
attitudes that were positive, the workshop leader firmly advocated “good practices” through
reviewing research on the topic and/or soliciting the consensus of the group. One good practice
espoused was instructional planning as a prerequisite to an effective lesson presentation. In this
respect, one workshop teacher-participant expressed her anger at the school principal for requiring
that lesson plans be on hand in her classroom. The teacher complained that she had been a teacher
for over 20 years and her plans were in her head and written lessons plans were unnecessary. When
the workshop leader did not agree with the teacher and supported the principal, several teachers
lent their support by responding to the teacher’s statement by asking, “Did she leave her head at
school when she was absent and the substitute teacher needed the plans?” The workshop
participants seemed to appreciate that they were viewed by the workshop leader as professionals
who were intelligent, knowledgeable, and able to analyze information.
The two-and-one-half-hour workshop sessions were lively because of the interactive
format, which included multiple activities such as reading, discussing, planning, reporting, etc.
Teachers were not passive participants. They were provided with information and were asked to
meet and plan based on the information and their experiences. There were no pre-set behaviors or
outcomes defined/described by the leader. Therefore, many of the participants’ responses were
innovative once they met, developed actions, and developed their own OTL assessment strategies in
the context of the topic presented.
How the Participants Rated/Evaluated the Workshop Sessions
and Identified Obstacles to Ongoing Implementation
At the end of each workshop, the teachers and administrators were asked to rate their
workshop experiences and to rate how teacher-friendly the OTL assessment strategy was after
practicing it in the workshop. The findings about the workshop sessions from the evaluation forms
ranged from some teacher-friendly to much teacher-friendly. No strategy was rated as very much
teacher-friendly. At the final follow-up/feedback sessions, teachers indicated that working in small
groups to learn how to implement the OTL assessment strategies was a valuable experience. Most
teachers indicated that they would use all of the OTL assessment strategies in their classrooms.
After reflecting, they found three strategies very teacher-friendly: Networking and collaborating,
assessing students’ mastery of skills, and conducting observations.
Table 1: Workshop participants’ ratings of OTL Assessment Strategies
________________________________________________________________________
OTL Assessment Strategy Mean Rating
_______________________________________________________________________
Using Networking and Collaborating to Improve Instructional Practices 3.36
Keeping Journals 3.58
Assessing Students’ Mastery of Skills and Concepts 4.07
Conducting Observations for Constructive Feedback 3.48
Conducting Surveys about Teaching Practices 3.89
Conducting Surveys about School Resources Needed for Effective Teaching 3.82
_______________________________________________________________________
Note: Rating Scale: 1= Not At All Teacher-Friendly; 2 = Not Much Teacher-Friendly;
3 = Some Teacher-Friendly; 4 = Much Teacher-Friendly; and 5 = Very Much Teacher-Friendly.
The workshop participants identified the best and worst features of the OTL assessment
strategies. They described the obstacles to the ongoing implementation of the OTL assessment
strategies. Teachers said the greatest obstacle to the ongoing implementation of the strategies was
that they were time-consuming. This was the same finding from the national survey (Stevens, et al.,
1998). Specifically, keeping journals, conducting surveys, and networking and collaborating were
all described as time-consuming. However, the positive analytical comments for supporting the
implementation of networking and collaborating seemed to outweigh the negative comments (see
Table 2).
Table 2. Workshop participants’ citations of the best and
worst features of the OTL Assessment Strategies
________________________________________________________________________
OTL Assessment Strategy Best Feature Worst Feature
________________________________________________________________________
1. Using Networking and Sharing and gaining Teaming with
Collaborating to Improve from each other’s teachers of lesser
Instructional Practices strengths and experience experiences
Finding areas of Finding time to
uniformity in their meet
teaching
Resistance to
Problem-solving change teaching
through finding practices and to
strategies to improve accept new ideas
academic achievement
Working as a team
2. Keeping Journals Opportunity to be Time consuming.
reflective Unwilling to write
each day about
progress
3. Assessing Students’ Better knowledge of Time spent testing
Mastery of Skills and what students have instead of teaching
Concepts and have not learned
4. Conducting Observations Opportunity to discuss Dependent upon
for Constructive Feedback teaching strategies the rapport and
with colleagues trust between the
observed and
Opportunity to see observer
different strategies
5. Conducting Surveys Opportunity to Time needed to
About Teaching collaborate about the conduct surveys
results of survey
Unwillingness of
Learn what is being teachers to listen
done in the school to ideas of others
Evaluate what is and is
not being taught
________________________________________________________________________
Teachers Implementing the Networking and Collaborating OTL Assessment Strategy
At one of the schools, the grade level chair for the third grade reported that her group had
met to solve the problem of a very heavy work load. Apparently, instead of meeting to collaborate
and network, they were working singularly. They solved their work load problem by agreeing to
divide up the work. At their meetings, they agreed on what needed to be taught during a certain
period of time. Then each teacher assumed the responsibility for preparing lessons and materials
for one subject area: Teacher #1 was responsible for reading; teacher #2 for mathematics; and
teacher #3 for English language arts. Also, they agreed to meet regularly to discuss and assess
where they were going with their classes and the progress or problems that they needed to address.
Teachers on several occasions wanted the workshop leader to remind their principals that they
needed additional time for meeting and planning together.
This information was then shared with other workshop participants when the grade level
chair reported it in her workshop session and the workshop leader reported it at other workshop
sessions. The leader observed that the other workshop participants were excited about this way of
using the collaboration and networking assessment strategy proposed by the grade 3 group. The
leader described this division of work to the workshop participants as “working smart.”
Feedback Sessions Provide Information about Ongoing Implementation
of OTL Assessment Strategies
At the final workshop, teachers participated in a feedback session with the co-principal
investigator. The faculty at each school rated the OTL assessment strategies along three
dimensions: not teacher-friendly, teacher friendly, and very teacher-friendly. The results were that
all of the strategies were at a minimum rated teacher-friendly. However, collaboration and
networking were found to be the most important of the assessment strategies, and lack of time or
limited time was viewed as the greatest deterrent to implementing the strategies. Teachers reiterated
this finding in follow-up interviews with grade level chairs and three other teachers at each school.
School principals when interviewed indicated that the workshops should have occurred at the
beginning of the school year to better prepare teachers, rather than mid-year.
At the end of the five months of professional development, the teachers were asked to think
about all of the OTL assessment strategies presented in the workshop sessions and again cite which
ones were not teacher-friendly, teacher-friendly and very teacher-friendly. The researcher
observed that the end-of-the-year results differed from earlier evaluations. On the several occasions
when the OTL assessment strategies were evaluated, teachers found most of the strategies teacherfriendly.
However, the specific results changed.
· Teachers reported that administrators’ observations of teachers teaching, surveying
teachers about their teaching practices, and networking and collaborating as a form of
professional development were very teacher-friendly OTL assessment strategies.
· Teachers said that networking and collaborating was a form of professional development
that was good because they were able to share information about successful teaching
strategies used in classrooms and obtain good ideas about how to improve their teaching.
The teachers noted a common concern about this strategy: There has to be a commitment
of time.
OTL assessment strategies of interval testing/assessment, surveying teachers about
resources needed for effective instruction, and keeping journals received mixed reactions, including
not teacher-friendly, teacher-friendly and very teacher-friendly from the various feedback
discussion groups. Those few strategies that were regarded by some teachers in their feedback
discussion as not teacher-friendly were accompanied with suggestions to modify and make them
more teacher-friendly.
Follow-up Interviews with School Principals, Teacher-Facilitators,
Grade Level Chairs and Selected Teachers about Obstacles
to Ongoing Implementation of the OTL Assessment Strategies
Telephone calls and visits to the schools were made in June 1998 to encourage principals
to continue emphasizing the importance of implementing those OTL assessment strategies that the
teachers viewed and evaluated as being useful/helpful to them. Teachers reported that networking
and collaborating with each other was essential and the time needed for this strategy was a
consistent message repeated by the teachers. When questioned about time for teachers to meet,
principals indicated that teachers already had conference hours for meetings. However, the teachers
at both schools indicated that the principals’ scheduling decisions, sometimes caused by teacher
absences, interrupted their opportunities to meet on a regularly scheduled basis.
Development of an OTL Assessment Strategies Implementation Checklist
The checklist was intended to be used in two ways: (1) for school staffs to do an
implementation self-assessment; and (2) for the researcher to use as an interview protocol with
school staffs. The assessment format covered the workshop activities. There was the expectation
that teachers would follow the suggested OTL assessment strategies that were teacher-friendly
during the school day, when and as needed. The rationale and format for the OTL assessment
implementation checklist followed the checklist rationale and format developed for the Adaptive
Education Project (Wang, 1992).
The implementation checklist delineated the workshop dimensions and performance
indicators were developed for each dimension. The opportunity to learn variables (content
coverage, content exposure, content emphasis, and quality of instructional delivery) were
determined to be the critical dimensions of the professional development workshops. The activities
practiced in the workshop sessions and given as follow-up “homework” assignments became the
performance indicators for each dimension. The responses to the indicators on the implementation
checklist were divided into three categories: yes (implemented), no (not implemented) and in
progress. The checklist was piloted; however, when the researcher attempted to use the checklist
with teachers at the first school, it was discovered that the performance indicators were too detailed
and, thus, too time-consuming. At that time, the researcher eliminated many of the indicators and
selected only those indicators judged to be the most critical for providing information about
ongoing implementation of the OTL assessment strategies. Using the revised implementation
checklist, the teachers who were interviewed indicated that they were attempting to implement the
OTL assessment strategies. This was verified by the school principals and teacher-facilitators when
the researcher conducted follow-up interviews with them in June.
Principals’ Management and Leadership Behaviors were Obstacles
to Implementation and Ongoing Implementation of OTL Assessment Strategies
Although both school principals espoused support of their teachers implementing OTL
assessment strategies, their management and leadership styles sometimes proved to be a hindrance
to effective implementation. One reason was that roles were not clearly defined. One principal
welcomed the teacher-facilitator as an ally in improving instruction while another principal viewed
the teacher-facilitator as a possible threat to her principalship. Mixed messages were being sent
and were not cleared up while the researcher was visiting the schools. It appeared to the researcher
that both principals and teacher-facilitators together needed to address and plan for successful
relationships and to set guidelines about how they were to work together to improve students’
academic achievement.
The teacher-facilitator who felt that she was the acknowledged instructional leader by the
teachers and had the passive acceptance of her role by the principal seemed to be much more
comfortable in her role of providing assistance and facilitating the adoption of OTL assessment
strategies in the school. The school principal had been an elementary teacher and an elementary
school principal before being assigned to her present school. She was not threatened by the teacherfacilitator’s
presence in the school and appeared to value the teacher-facilitator as being
knowledgeable about the elementary curriculum and practices. Thus, the principal relied on the
teacher-facilitator’s assistance. This may have been an obstacle to assessment implementation
because it was the teacher-facilitator who attended all of the workshop sessions while the principal
visited occasionally and did not stay throughout the session. The principal’s lack of consistent
attendance did not model for the teachers the importance of the workshop sessions.
The teacher-facilitator who was unsure of her role because of the principal’s periodic
resistance to her presence in the school felt uncomfortable in being a strong advocate for any kind
of change. Instead, this teacher-facilitator took the safe road of responding only to requests for
assistance but did not initiate assistance. The researcher observed that the principal of this school
missed the opportunity to use a person who was knowledgeable about the elementary curriculum
and practices. It appeared that the principal needed a planned partnership with the teacherfacilitator
because the principal was learning about the elementary curriculum having previously
worked at the secondary level. A partnership or team approach would have been much more
effective for the school and for the teachers within the school who wanted to implement the OTL
assessment strategies and the ALEM model of school reform. In contrast to the first school, this
principal modeled the importance of the workshops by attending all sessions and staying
throughout the sessions. However, there were other underlying relationship problems with some of
the teachers that diminished the impact of the principal’s modeled message of support for the
workshops to her staff.
Norm-referenced Test Scores for the Two Schools:
The Results of the Community for Learning Component—
ALEM and the OTL Assessment Workshops
Assessing students’ mastery of skills and concepts was a popular session with the teachers
because they wanted their schools to show improvement and move them out of a “worst schools”
category. However, they were frustrated and angry with the D.C. school district because no
workshops or materials about the Stanford 9 had been provided when this session was presented.
The workshop leader stepped into the void and geared the sessions on students’ mastery of skills
and concepts to information about the Stanford 9. The workshop leader used the publishing
company’s Stanford 9 materials to show teachers how to plan to address the testing information for
their school and to begin preparing an instruction level for the Stanford 9. Again, collaboration and
networking was stressed as essential to implement the actions needed.
For the school year 1997-98, the two schools did improve in reading on the Stanford 9,
almost reaching the median (50th percentile). One school’s percentile score for Spring 1998 was
44, an increase of 14 points from the Fall 1997 score. The other school’s percentile score was 47,
an increase of 19 points from its Fall 1997 score. It should be noted that the scores for reading
were only for those students who took the Stanford 9 in the fall and spring. In other words, these
were the students who benefited from a complete year of instruction in the same school.
For the school year 1996-97, these same two schools showed improvement in reading but
at lower levels than 1997-98. School #1’s reading percentile score improved from 20 to 31, an
increase of 11 points, while school #2 increased its percentile score from 32 to 42, an increase of
10 points. Another analysis of the Stanford 9 reading achievement test scores looked at the two
different cohort groups’ scores for Spring 1997 and Spring 1998. The two schools’ reading
percentile score increased by 13 and 5 percentile points, respectively.
Table 3. NCE and Percentile Reading Scores for 2 Schools,
Fall 1996 and Spring 1997 and Fall 1997 and Spring 1998.
School Fall 1996 Spring 1997 Diff. Fall 1997 Spring 1998 Diff.__
#1 %ile 20 31 11 30 44 14
NCE 28.4 36.9 8.5 35.8 46.4 10.6
#2 %ile 32 42 10 28 47 19
NCE 36.6 43.4 6.8 34.8 48.1 13.3
______________________________________________________________________________
FINDINGS
Teachers rated all of the OTL assessment strategies as teacher-friendly. However, it was
the networking and collaborating assessment strategy that captured the ongoing support of all
teachers. In fact, teachers urged the workshop leader to remind the principals to keep their
commitment to provide time for the teachers to meet regularly so that they could network and
collaborate about instructional issues and practices in their schools.
Implementation of ALEM and the OTL assessment strategies were complementary to each
other. Nothing in the OTL assessment strategies workshop was contradictory to ALEM model’s
practices and procedures. ALEM stressed that teachers needed to have team meetings while OTL
stressed the need to network and collaborate in small groups. While ALEM was more prescriptive
about the teaching practices and room environment, the OTL assessment strategies encouraged
teachers to think and choose the assessment actions they felt were appropriate, but to also include
planning as a necessary component.
Levels of demonstrated leadership seemed to play an important role. There were higher
achievement scores at the school where the principal and assistant principal were constant
attendees at the workshop sessions in contrast to the school where the assistant principal attended
no workshop sessions and the principal’s attendance was brief and intermittent. Also, the teacherfacilitators
proved to be quite valuable in providing instructional support to the teachers.
Whatever the “chemistry” or mixture of principal type, teacher-facilitators, and teachers,
both schools improved in their reading percentile scores. The schools had improved in 1996-97, the
year prior to the OTL assessment strategies workshops, but the amount of positive change was not
as large as the year of the workshop (1997-98). In fact the increase in percentile scores in 1997-98
was almost double the amount of change in 1996-97 and the Spring 1998 scores for both schools
differed by 5 points.
REFERENCES
Bailey, M. (1996). Assessing opportunity to learn in urban schools: A report on the review of
research documents to identify current OTL practices. Philadelphia: Temple University
Center for Research in Human Development and Education.
Fullan, M. (1990). Staff development, innovation and institutional development. In B. Joyce
(Ed.), Changing school culture through staff development. (pp. 3-25). Alexandria, VA:
Association for Supervision and Curriculum Development.
Gallagher, M., Goudvis, A., & Pearson, P. (1988). Principals of organizational change. In S. J.
Samuels & P. Pearson (Eds.), Changing school reading programs. (pp. 11-39). Newark,
DE: International Reading Association.
Goldenberg, C., & Gallimore, R. (1991). Changing teaching takes more than a one-shot workshop.
Educational Leadership, 49(3), 69-72.
Griffin, G. (1986). Clinical teacher education. In J. Hoffman & S. Edwards (Eds.), Reality and
reform in clinical teacher education. (pp. 1-24). New York: Random House.
Lieberman, A. (1996). Practices that support teacher development: Transforming conceptions of
professional learning. In M. McLaughlin & I. Oberman (Eds.), Teacher Learning: New
Policies, New Practices. The Series on School Reform. New York: Teachers College
Press.
Loucks-Horsley, S., Harding, C., Arbuckle, M., Murray, L., Dubea, C., & Williams, M. (1987).
Continuing to learn: A guidebook for teacher development. Andover, ME: Regional
Laboratory for Educational Improvement of the Northeast and Islands/National Staff
Development.
McLaughlin, M. (1991). Enabling professional development: What we have learned. In A.
Lieberman & L. Miller (Eds.), Staff development for education in the 90’s. (pp. 61-82).
New York: Teachers College Press.
Meyer L. (1988). Research on implementation: What seems to work?. In S. Samuels, & P.
Pearson (Eds.), Changing school reading programs: Principles and case studies.
Newark, DE: International Reading Association.
Schmoker, M. (1996) Results: The key to continuous school improvement. Alexandria, VA:
Association for Supervision and Curriculum Development.
Stevens, F. (1993a). Opportunity to learn: Issues of equity for poor and minority students.
Washington, DC: National Center for Education Statistics.
Stevens, F. (1993b). Applying opportunity to learn conceptual framework to the investigation of
the effects of teaching practices via secondary analyses of multiple-case-study summary
data. Journal of Negro Education, 62(3), 232-248.
Stevens, F., Wiltz, L., & Bailey, M. (April, 1998). Teachers’ evaluations of the sustainability of
opportunity to learn (OTL) assessment strategies: A national survey of classroom teachers in large urban school districts. (Paper). Philadelphia: Temple University Center
for Research in Human Development and Education.
Wang, M. (1992). Adaptive education strategies: Building on diversity. Baltimore: Paul H.
Brookes Publishing Co, Inc.
Appendix
(See “99-8 Appendix” document available on this Web site.)