Evaluation for the 2003 Honolulu District Professional
Development
Program: Teaching
Science Literacy through Inquiry-
The Research
Investigation Process (RIP™)
ANOVA Science
Education Corporation
Honolulu, Hawaii
June 28, 2003
The purpose of this professional development program was to
introduce K-12 teachers to the teaching of science through true scientific
inquiry, using the research investigation process (RIP ä) and to explore the RIP
as a tool for addressing the Hawaii Science Content and Performance Domain I
standards in the classroom.
Specifically, it was designed to guide teachers in the use of the
inquiry process; to have teachers learn how to design and conduct scientific
research studies; to have them become familiar with techniques to assist in
guiding students through the scientific inquiry process; to have them examine,
practice, understand, and become competent in the ability to apply data
analysis techniques to decision-making in science; to increase confidence in
using scientific research in their approach to instructing students in science
and in addressing the scientific inquiry benchmarks and science inquiry content
standards; to have them implement the RIP as a tool for instruction in the
classroom; and to increase student interest in learning science.
Over the course of the initial three-day workshop session,
the research investigation process (RIP) was introduced and teachers were
provided the opportunity to develop an understanding of each of the elements of
the RIP through their participation in and development of actual research
investigations. Teacher participants
were guided through a number of activities related to making observations;
posing research questions; obtaining, examining, and evaluating background
information; constructing hypotheses; and designing the methods for a research
investigation. Techniques in data
summary, analysis and presentation were explored in the context of hypothesis
testing and decision-making in science.
Teachers were then expected to introduce workshop-related concepts and
activities learned into their classroom and guide their students in conducting
their first RIP over the subsequent three months. During the three-month implementation period,
half-day individual teacher/small group follow-up sessions were available to
the participating teachers upon request.
The individual teacher/small group follow-up sessions involved modeling
of instructional techniques and practices with students, assisting teachers on
curriculum development, and/or clarifying concepts presented in the initial
three-day workshop session. The
participants met together again in a final follow-up session at the end of the
three month implementation/individual teacher follow-up period to share their
inquiry-based instructional experiences and student outcomes. All aspects of this workshop were aligned
with the State of Hawaii Science
Content and Performance Standards.
The data for this workshop evaluation were obtained from
assessments of the 25 teacher-participants at the beginning of (Pre-Assessment)
and again at the end (Post-Assessment) of the 3-day initial workshop, and from
questionnaires administered along with the Post-Assessment (Post-Workshop
Questionnaire) and during the follow-up session at the end of the program
(Post-Follow-Up Questionnaire). Items on
the assessments required demonstration of knowledge about the scientific
inquiry process, data analyses procedures, and decision-making in science. A number of these items required teachers to
demonstrate their knowledge through application. Self-report items measured teacher confidence
levels in understanding and using scientific inquiry in the classroom and in
comprehending and applying the scientific inquiry content standards to their
instruction. The response scale for the
confidence items included “not at all confident” (‘0’-value), “somewhat
confident” (‘3’-value), “confident” (‘6’-value), and “completely confident”
(‘9’-value). A concept inventory
determined teachers’ familiarity with and ability to teach elements of
scientific inquiry and data summary and analysis techniques. The answer scale for the concept inventory
items included “I am completely unfamiliar with this concept” (value=1), “I am
somewhat familiar with this concept, but do not really understand what it
means” (value = 2), “I am familiar with this concept , and have a fair
understanding of what it means” (value = 3), “I am very familiar with this
concept, but would have some difficulty teaching it to others” (value = 4), and
“I am completely familiar with this concept and could easily teach it to
others” (value = 5). The pre-workshop and post-workshop assessment
items were the same. The Post-Workshop
Questionnaire containing five items was also administered to assess the
teachers’ perceptions of how much their understanding of scientific inquiry and
the research investigation process changed and improved as a result of
participation in the workshop. Finally,
the Post-Follow-Up Questionnaire, containing a number of the teacher confidence
and perception items on the Pre- and Post-Assessments, as well as additional
items related to the impact of the individual/small group teacher follow-up
sessions and activities on teacher perceptions, was administered. Paired t-tests were used to determine
significant differences (indicating change) between Pre- and Post-Assessment
mean values and between Post-Workshop Questionnaire and Post-Follow-Up
Questionnaire responses. One-way
repeated measures ANOVAs were used to determine significant differences
(indicating change) in responses on items from the common items on the
Pre-Assessment, Post-Assessment, and Post-Follow-Up Questionnaire. In the latter cases, following a significant
effect, Tukey’s Tests were used for multiple comparisons. The criterion for statistical significance (a) for all tests was set at 0.05.
Teacher Knowledge and Understanding of the Scientific
Research Investigation Process (RIP), and Confidence in Teaching Scientific
Inquiry
Workshop participants demonstrated a large, statistically
significant increase in their knowledge and understanding of the individual
elements of the RIP by the end of the 3-day workshop (Figure 1, below). This included the logical order of the RIP elements,
understanding of components involved in each element, and demonstration of the
ability to construct testable hypotheses.
Figure 1. Demonstration of knowledge and understanding
of the elements of the RIP.
There were a total of 25 points available on this
portion of the assessment.
* Mean Post-Assessment score
is significantly greater than mean pre-assessment score [t (24) =
8.56, p<0.001].
The post-workshop increase in teacher-participant knowledge
and understanding of the research process was accompanied by a significant
increase in teacher’ self-reported familiarity and understanding of concepts
related to the scientific research process in the concepts inventory (Figure 2,
below). The average participant’
response rose from “familiar with a fair understanding of the concept” to “very
familiar with the concept with some difficulty in teaching it to others” by the
end of the workshop. This showed that
teachers recognized their increased knowledge and understanding.
Mean (+SEM) RIP Concept Inventory Score
|
|
Figure 2. Familiarity and
understanding of concepts related to elements of the RIP.
The answer scale for the concept inventory items
included “I am completely unfamiliar with this concept” (value=1), “I am
somewhat familiar with this concept, but do not really understand what it
means” (value = 2), “I am familiar with this concept, and have a fair
understanding of what it means” (value = 3), “I am very familiar with this
concept, but would have some difficulty teaching it to others” (value = 4), and
“I am completely familiar with this concept and could easily teach it to
others” (value = 5).
* Mean Post-Assessment score is significantly greater
than mean pre-assessment score [t (24) = 3.91, p<0.001].
By the end of the 3-day workshop, participants’
self-reported confidence levels for their ability to use scientific inquiry,
their understanding of teaching science through inquiry, and their ability to
teach and engage students in scientific research activities all increased
significantly f (Figures 3, 4 and 5, below) from less than “confident” to
“confident” or higher.
Mean (+SEM) Confidence
Score
|
|
Figure 3. Self-reported
confidence levels for ability to use scientific inquiry. The response scale for the confidence items
included “not at all confident” (‘0’-value), “somewhat confident” (‘3’-value),
“confident” (‘6’-value), and “completely confident” (‘9’-value).
·
Mean Post-Assessment score is significantly greater than mean pre-assessment
score [t (24) = 5.20, p<0.001].
Mean (+SEM) Confidence
Score
|
|
Figure 4. Self-reported
confidence levels for understanding of teaching science through inquiry. The response scale for the confidence items
included “not at all confident” (‘0’-value), “somewhat confident” (‘3’-value),
“confident” (‘6’-value), and “completely confident” (‘9’-value).
*Mean Post-Assessment score is significantly greater
than mean pre-assessment score [t (24) = 4.81, p<0.001].
Mean (+SEM) Confidence
Score
|
|
Figure 5. Self-reported
confidence levels for ability to teach and engage students in scientific
research activities. The response scale
for the confidence items included “not at all confident” (‘0’-value), “somewhat
confident” (‘3’-value), “confident” (‘6’-value), and “completely confident”
(‘9’-value).
* Mean Post-Assessment score
is significantly greater than mean pre-assessment score [t (24) =
4.58, p<0.001].
Teacher Understanding of and Ability to Apply Data
Summary, Presentation, and Analysis techniques to Decision-Making in Science
By the end of the workshop, participants demonstrated a
large, statistically significant increase, almost doubling their Pre-Assessment
score, in their knowledge and ability to correctly organize data into a summary
table and to construct a bar graph for comparing the central tendency for two
groups of data (Figure 6, below).
Mean (+SEM) Data Summary & Presentation Score
|
|
Figure 6.
Demonstration of understanding and ability to apply data organization
and presentation techniques to data.
This section was worth a total of 10 points.
* Mean Post-Assessment score
is significantly greater than mean pre-
assessment score [t (24) = 6.52, p<0.001].
Workshop participants also demonstrated a dramatic change in
their knowledge and ability to apply data analysis techniques to research
data. Comparison of the pre-and
Post-Assessments revealed that by the end of the workshop, they significantly
increased their understanding of how to calculate descriptive statistics and their
ability to determine which measure of central tendency is most appropriate for
a group of data (Figure 7, below).
Mean (+SEM) Data Analysis Score
|
|
Figure 7.
Demonstration of understanding of the calculations for descriptive
statistics and ability to determine the most appropriate statistic to represent
central tendency for a group of data.
This section was worth a total of 10 points.
* Mean Post-Assessment score
is significantly greater than mean pre-assessment score [t (24) =
8.21, p<0.001].
Participants demonstrated a statistically
significant increase in their ability to interpret data presented in
scatterplots and summarized in bar graphs by the end of the workshop (Figure 8,
below).
Mean (+SEM) Graph Interpretation Score
|
|
Figure 8.
Demonstration of ability to interpret scatterplots and bar graphs. This section was worth a total of 10 points.
* Mean Post-Assessment score
is significantly greater than mean pre-assessment score [t (24) =
4.90, p<0.001].
The participant increase
in knowledge of and ability to apply data presentation and analyses were
accompanied by a significant increase in teacher’ self-reported familiarity and
understanding of concepts related to data presentation and analysis in the
concepts inventory (Figures 9 and 10, below).
By the end of the workshop, the average participant’ response for the
three measures of central tendency rose significantly from between “somewhat
familiar with concept, but do not really understand what it means” and “I am
familiar with this concept, and have a fair understanding of what it means” to between “I very
familiar with this concept but would have some
difficulty teaching it to others” and “I am
completely familiar with this concept and could easily teach it to others (Figure 9).
Mean (+SEM) Central Tendency Concept Inventory
Score
|
|
Figure 9. Familiarity and
understanding of concepts related to measuring central tendency. The answer scale for the concept inventory
items included “I am completely unfamiliar with this concept” (value=1), “I am
somewhat familiar with this concept, but do not really understand what it
means” (value = 2), “I am familiar with this concept, and have a fair
understanding of what it means” (value = 3), “I am very familiar with this
concept, but would have some difficulty teaching it to others” (value = 4), and
“I am completely familiar with this concept and could easily teach it to
others” (value = 5).
* Mean Post-Assessment score is significantly greater
than mean pre-assessment score [t (23) = 7.11, p<0.001].
Similarly, the average participant’ concept inventory
response for tables and graphs rose significantly from “familiar with the
concept with a fair understanding of what it means” to “very familiar with the
concept, but would have some difficulty teaching it to others” (Figure
10).
Mean (+SEM) Tables & Graphs Concept Inventory
Score
|
|
Figure 10. Familiarity and
understanding of concepts related to tables and graphs. The answer scale for the concept inventory
items included “I am completely unfamiliar with this concept” (value=1), “I am
somewhat familiar with this concept, but do not really understand what it
means” (value = 2), “I am familiar with this concept, and have a fair
understanding of what it means” (value = 3), “I am very familiar with this
concept, but would have some difficulty teaching it to others” (value = 4), and
“I am completely familiar with this concept and could easily teach it to
others” (value = 5).
*Mean Post-Assessment score
is significantly greater than mean pre-assessment score [t (24) =
4.57, p<0.001].
Benchmarks and Standards
General teacher
confidence in and awareness of ability to understand and apply scientific
inquiry to the teaching of science, and in ability to successfully address the
scientific inquiry standards, was enhanced by their participation in the
workshop. Participant self-reported
confidence in ability to address content standards in the classroom rose
significantly from less than “confident” to above “confident” by the end of the
workshop (Figure 11, below).
Mean (+SEM) Confidence
Score
|
|
Figure 11. Self-reported
confidence levels for ability to address content standards in the
classroom. The response scale for the
confidence items included “not at all confident” (‘0’-value), “somewhat
confident” (‘3’-value), “confident” (‘6’-value), and “completely confident”
(‘9’-value).
* Mean
Post-Assessment score is significantly greater than mean Pre-Assessment score [t
(24) = 3.71, p<0.001].
Similarly, by the end of the workshop, participant
confidence about ability to accurately and completely address the scientific
inquiry standards dramatically increased from “somewhat confident” to above
“confident” (Figure 12, below).
Mean (+SEM) Confidence
Score
|
|
Figure 12. Self-reported
confidence levels for ability to accurately and completely address the
scientific inquiry benchmarks. The
response scale for the confidence items included “not at all confident”
(‘0’-value), “somewhat confident” (‘3’-value), “confident” (‘6’-value), and
“completely confident” (‘9’-value).
* Mean
Post-Assessment score is significantly greater than mean pre-assessment score [t
(24) = 7.05, p<0.001].
Finally, by the end of the 3-day workshop, teachers
significantly increased their familiarity and understanding of inquiry
standards from being “somewhat familiar with this concept,” but not really
understanding what it means to being between “familiar
with this concept, with “a fair understanding of what it means” and “very
familiar” with this concept, but with “would have some difficulty teaching it
to others.” This increase was
statistically significant and was consistent with the increase in
teacher-participant confidence regarding scientific inquiry and addressing the
inquiry standards (Figure 13, below).