Program: RIP™ -Based
Inquiry and Statistics for Grades 6-12 Teachers at the University
of Hawaii, Graduate Professional
Development Series Seminar for Science Teachers 2002, ANOVA Science Education
Corporation, Honolulu, HI.
Program evaluation
submitted: 9-18-2002
The overall purposes of this graduate seminar were to review
and strengthen the participants’ understanding of and ability and confidence to
use the research investigation process (RIP) used in scientific inquiry, and to
introduce 6-12 grade teachers to research design, data handling and summary,
and data analyses techniques and procedures as they are used in the RIP. Specifically, this three-day seminar was
designed for teachers to explore the research investigation process; to use the
inquiry process to learn how to design and conduct scientific research studies;
to become familiar with techniques to assist in guiding students through the
scientific inquiry process; understand and be able to obtain random samples;
develop research investigations using independent and dependent designs; to
learn how to represent and present data obtained from research investigations;
to examine, practice, understand, and become competent in the ability to apply
data analysis techniques, including the quantification of error and statistical
tests for significance, to decision-making in science; and to increase
confidence in using scientific research in their approach to instructing
students in science and in addressing the scientific inquiry benchmarks and
science inquiry content standards. The
research investigation process (RIP) was reviewed and teachers were provided
the opportunity to further develop their understanding of each of the elements
of the RIP through their participation in and development of actual research
investigations. Techniques in data
summary, analysis and presentation were explored in the context of hypothesis
testing and decision-making in science.
Finally, numerous strategies for the teaching of science through true
scientific inquiry were also emphasized.
All aspects of this seminar were aligned with the State of Hawaii Science Content
and Performance Standards.
The data for this graduate seminar evaluation were obtained
from assessments of the 14 teacher-participants at the beginning of
(Pre-Assessment) and again at the end (Post-Assessment) of the 3-day
seminar. Items on the assessments
required demonstration of knowledge and understanding about the scientific
inquiry process, data analyses procedures, and decision-making in science. A number of these items required participants
to demonstrate their knowledge and understanding through application. Self-report items measured teacher confidence
levels in understanding and using data summary, representation, and analysis
techniques in scientific inquiry conducted in the classroom, and in comprehending
and applying the scientific inquiry content standards to their
instruction. A concept inventory
determined teachers’ familiarity with and ability to teach elements of
scientific inquiry, research design, and data summary and statistical analysis
techniques. The pre-seminar and
post-seminar assessment items were the same except for three additional
self-report items included on the post-assessment. These additional items assessed the teachers’
perceptions of how much their understanding of the research investigation
process and scientific inquiry process improved, and understanding of
application of statistics to data increased, as a result of participation in
the seminar. The data were statistically
analyzed using dependent t-tests to determine significant differences
(indicating change) between pre- and post-assessment mean values. For comparisons in which the assumption of
normality was violated, the Wilcoxon Signed Rank Test (nonparametric t-test)
was used to compare central tendencies.
The Pearson Product Moment Correlation Coefficient was used to test for
linear relationships. The criterion for
statistical significance (a) was set at
0.05.
Scientific Inquiry, Scientific Research, and the Research
Investigation Process (RIP)
This seminar focused on review and strengthening of teacher
understanding, abilities, and confidence related to instruction of science
through scientific inquiry, conducting scientific research, and engaging their
students in scientific research.
Although it was generally expected that teacher-participants would be
familiar with these aspects of scientific inquiry, a pre-seminar phone survey
of the participants’ perceived knowledge and abilities indicated that it would
be important to address the RIP before examining research design and analysis
strategies and procedures. Specific
emphases were placed on teacher demonstration of understanding the logic of
organization of the RIP elements, as well as the components and concepts
involved in each element.
Although participant self-reported familiarity
with/understanding of the RIP appeared to slightly increase from its
pre-seminar level by the end of the three-day seminar (Figure 1, below), the
difference was not statistically significant.
However, the power of the t-test was low and so the negative
statistical finding should be interpreted cautiously (see Figure 1 caption for
more details).
Mean (+SEM) RIP Concept Inventory Score
|
|
Figure 1. Familiarity with and
understanding of concepts related to methodology in the RIP.
Mean post-assessment value
did not differ from the mean pre-assessment value [t (13) =
-1.99, p=0.068]. Note: Because the power of the performed
statistical test (0.345) was below the desired power of 0.800, the results
should be interpreted cautiously.
At first glance, the lack of a significant change in
self-reported familiarity/understanding of the RIP concept appears surprising
because participant-demonstrated knowledge and understanding, and self-reported
confidence (discussed below) increased consistently for all of the RIP
assessment items. However, both the pre-
and post-assessment concept inventory mean values were relatively high and
indicated that the participants felt that they were already very familiar with
and had a good understanding of (could teach the concept to others) the RIP
before participating in the seminar.
Post-seminar teacher-participant demonstrated knowledge and
understanding of the RIP components and their contents (Figure 2, below) and of
the organization of the RIP elements (Figure 3, below) significantly increased
compared to pre-seminar levels.
Mean (+SEM) Level of Demonstrated Knowledge and
Understanding
of the RIP
|
|
Figure 2. Knowledge and
understanding of the research investigation process (RIP) components and
content. (Highest possible value for
knowledge and understanding was six.)
*Mean post-assessment value differs from the mean
pre-assessment value [t (13) =
-3.29, p=0.006].
Mean (+SEM) Level of Demonstrated Knowledge of
Organization of the RIP Elements
|
|
Figure 3. Knowledge and
understanding of the organization of the elements of the research investigation
process (RIP). (Highest possible level
of knowledge was seven.)
*Mean post-assessment value differs from the mean
pre-assessment value [t (13) = -2.49, p=0.027].
By the end of the three-day seminar compared to
pre-seminar levels, participants’ self-reported confidence levels for both
ability to use scientific inquiry in their instruction and their ability to
teach and engage students in scientific research activities increased from
“confident” to midway between “confident” and “very confident.” Although these increases were only slight to
moderate, they were both statistically significant (Figures 4 and 5, below).
Mean (+SEM) Confidence
Score
|
|
Figure 4. Self-reported
confidence levels for ability to use scientific inquiry in instruction.
Mean post-assessment value is significantly greater
than mean pre-assessment value [t (13) = -2.60, p=0.022].
Figure 5. Self-reported
confidence levels for ability to teach and engage students in scientific
research activities.
* Mean post-assessment value
is significantly greater than mean pre-assessment value [W (N=14)
= -54.0, p=0.034].
Design and Analysis
The seminar participants exhibited increases in their
knowledge and understanding as well as in their confidence related to the
elements involved in designing research investigations so that they can be
analyzed using the appropriate statistical tests. By the end of the 3-day seminar, participants
demonstrated a statistically significant increase in their knowledge and
understanding of research designs, statistical concepts including, but not
limited to, random sampling, error (variability), and statistical tests (Figure
6, below).
Mean (+SEM) Research Design and Statistics Score
|
|
Figure 6. Demonstration of knowledge and understanding
of research design and statistical concepts.
(Scores were based on the number of items correct
out of 10.)
* Mean post-assessment value
is significantly greater than mean pre-
assessment value [t (13) = -4.70, p<0.001].
The post-seminar increase in teacher-participant knowledge
and understanding of the research process was accompanied by a significant increase in teacher’ self-reported
familiarity and understanding of concepts related to research design and
methodology (Figure 7, below) and data analysis including descriptive and
inferential statistics (Figure 8, below).
Mean (+SEM) Research Design/Methodology Concept
Inventory Score
|
|
Figure 7. Familiarity with and
understanding of concepts related to research design and methodology.
* Mean post-assessment value is significantly greater
than mean pre-assessment value [t (13) = 6.38, p<0.001].
Mean (+SEM) Data Analysis Concept Inventory Score
|
|
Figure 8. Familiarity with and
understanding of concepts related to data analysis, including descriptive and
inferential statistics.
* Mean post-assessment value is significantly greater
than mean pre-assessment value [t (13) = -5.57, p<0.001].
By the end of the seminar, the average participant’ response
to the concept of research design and methodology rose from between “somewhat
familiar, but do not really understanding what it means” and “familiarity with
fair understanding of the concept” to “very familiar with concept and could
teach it to others.” Likewise, the
average participant’ response to the concept of data analysis rose from
“somewhat familiar, but do not really understanding what it means” to between
“familiarity with fair understanding of the concept” and “very familiar with
concept and could teach it to others.”
These findings indicated that teachers recognized their increased
knowledge and understanding related to research design and statistical
concepts.
With the increase in knowledge and understanding, the
teacher-participants also demonstrated an increase in self-reported confidence
in their understanding of the application of basic statistics to analyze research
data (Figure 9, below).
Mean (+SEM) Confidence
Score
|
|
Figure 9. Self-reported
confidence levels for understanding of the application of basic statistics to
analyze research data.
* Mean post-assessment value
is significantly greater than mean pre-assessment value [t (13) =
6.07, p<0.001].
By the end of the three-day seminar, the mean participant
confidence score more than doubled, significantly rising from between “not at
all confident” and “somewhat confident” to near “confident.”
Teacher Understanding of and Ability to Apply Data
Summary, Presentation, and Analysis techniques to Decision-Making in Science
By the end of the three-day workshop, seminar participants
demonstrated a significant 33% increase in their ability to determine the
appropriate statistic for describing the center of a group of data (Figure 10,
below).
|
|
|
|
|
Mean (+SEM) Score for Ability to Use Appropriate
Measure of Central Tendency
|
|
|
|
|
Figure 10.
Demonstration of the ability to determine the appropriate measure of
central tendency for a group of data.
(Highest possible ability score was four.)
* Mean post-assessment value
is significantly greater than mean pre-
assessment value [t (13) = 2.79, p=0.015].
In addition to gains in understanding of the application of
descriptive statistics, participants also demonstrated a post-seminar increase
in knowledge of the procedures used to calculate the three measures of central
tendency (Figure 11, below).
Mean (+SEM) Score for Ability to Calculate the
Three Measures of Central Tendency
|
|
Figure 11.
Demonstration of the ability to calculate the three measures of central
tendency. (Highest possible ability
score was six.)
* Mean post-assessment value
is significantly greater than mean pre-assessment value [t (13) =
3.16, p=0.008].
The seminar participants’ self-reported familiarity with,
and understanding of the concepts of, the three measures of central tendency
also significantly increased by the end of the three-day seminar (Figure 12,
below). This increase coincided with the
demonstrated increases in understanding and ability to calculate the measures
of central tendency (both discussed above), indicating that the participants
were able to accurately perceive and report their relative knowledge of and
ability to summarizing groups of data.
Mean (+SEM) Measures of Central Tendency Concept
Inventory Score
|
|
Figure 12. Familiarity with and
understanding of concepts related to the three measures of central tendency.
* Mean post-assessment value is significantly greater
than mean pre-assessment value [t (13) = 3.60, p=0.003].
Figure 13 (below) presents a scatterplot showing the pre- to
post-seminar change in participants’ familiarity/understanding of the concept
of measures of central tendency plotted as a function of change in demonstrated
participant knowledge of descriptive statistics. Pearson Product Moment Correlation
Coefficient revealed a significant moderate positive relationship between these
two variables (Figure 13).
|
|
|
|
|
|
|
|
 |
|
Amount of
Change in Self-Reported Familiarity/ Understanding
|
|
|
|
|
|
Amount of Change in Demonstrated Knowledge
|
|
Figure
13. Scatterplot of pre- to post-seminar
change in familiarity/understanding as a function of change in demonstrated
knowledge of measures of central tendency.
By the end of the three-day semina