Alaskan Data-Driven Program Planning, Development, and Evaluation

Goldstream Group, Inc.

Informal Science Education

Evaluation of Winterberry Project

The Project

The Winterberry project – or Arctic Harvest – seeks to improve participation in and effectiveness of citizen science, particularly in high-latitude communities where populations are underrepresented in STEM. The project’s research goals include examining the relationship between citizen science learning outcomes and attributes of social-ecological resilience, and assessing the impact of direct mentor and scientist support, as well as a story-based content delivery method, on engagement and resilience-related outcomes.

Our Work

Looking at key performance indicators for the project, the evaluation seeks to assess the effectiveness of three different citizen science program models, and identify program characteristics that contribute to each model’s rating. Data collection includes document review, observations of program activities, written youth and club leader surveys, and interviews with program staff and club leaders.

Usefulness of Our Work

Through formative and summative reporting, evaluation work will provide recommendations for program improvements both during and at the end of the project. It is also providing data that will be utilized in research papers around the topic of citizen science.

Evaluation of Arctic and Earth SIGNs Project

The Project

Arctic and Earth SIGNs project is focused on three central goals: 1) develop a high-quality climate change education program that includes NASA assets (resources and experts), citizen science, and mobile technology for formal and informal science education settings; 2) Engage educators and community members in learning experiences to model best practice for inquiry-based, culturally responsive climate change science teaching; and 3) Engage youth, community members, and educators in locally and globally important science where they produce and apply new information on the impact of a changing global climate.

Our Work

Through both formative and summative evaluation components, three main areas of focus for the evaluation of this project are the extent to which project activities improve STEM instruction, the extent to which underrepresented youth and members of the public are engaged in STEM, and an increase in scientific literacy among participants.

Usefulness of Our Work

Formative reporting will help the project fine-tune its engagement of teachers, youth, and members of the public in citizen science activities – including training opportunities and mid-project support. Summative findings will aim to provide data that is useful for creation of future programs as well as viable for publications around engagement in citizen science.

Evaluation of Alaska EPSCoR Education, Outreach, and Diversity Workforce Development Grant

The Project

The University of Alaska received National Science Foundation (NSF) Experimental Program to Stimulate Competitive Research (EPSCoR) funding to implement “Alaska Adapting to Changing Environments” (Alaska ACE) in 2013. The evaluation focused on EPSCoR’s education, outreach, and diversity/workforce development (EOD) activities, measuring participant satisfaction and the extent to which activities contributed to building community readiness for adaption to changing environments and to future research capacity.

The Evaluation

An outcomes-based approach was used to evaluate the impact of the EOD activities. The evaluation plan was framed around the EOD logic model, which defined several short-term outcomes to help determine if the goals were met. These outcomes were organized around themes common to informal science education: interest or engagement; self-efficacy (sense of ability); awareness, knowledge or understanding; and behavior changes.

Usefulness of Findings

Evaluation results were suitable for determining if the project’s EAD requirements were met, but also suggested ways in which the project – or other education and outreach efforts pursued by EPSCoR – could be made even better going forward.

Evaluation of Hot Times in Cold Places: The Hidden World of Permafrost

The Project

Through a four-year, $1.5 million grant provided to the collaborating team of the University of Alaska Fairbanks’ Geophysical Institute and the Oregon Museum of Science and Industry (OMSI), the Permafrost project is part of NSF’s Innovations in Development initiative for Advancing Informal STEM Learning. The Permafrost Project builds and expands on a half-century of climate-related education and outreach activities at the nation’s only research permafrost tunnel, located near Fox, Alaska. Using permafrost as a focus, the team is exploring the power of immersive experiences and real objects to improve learning about climate change. At the same time, the project broadens the reach of the tunnel, with its demonstrated ability to get people excited about permafrost, by developing programs and exhibits that will travel throughout Alaska and the rest of the U.S.

Our Work

Goldstream Group’s evaluation of this project includes formative assessment of travelling Alaska outreach materials, and intensive formative and summative evaluations of the larger travelling museum exhibit spearheaded by OMSI. Through surveys, observations, prototype testing with target audiences, and other research-based methods within the informal science learning and museum studies fields of evaluation, the evaluation will examine the success of the project in areas including participant learning about permafrost- and climate change-related concepts, and participant attitudes about and engagement with climate change.

Use

Our formative evaluation provided data-driven suggestions about refining Alaska-specific and OMSI-created exhibit pieces. The summative evaluation will provide resources for publications related to real vs. fabricated items in museum learning, as well as further programmatic analysis that can inform this projects and similar initiatives into the future.

Evaluation of the University of Alaska Fairbanks GK-12 CASE Project

The Project

Funded by National Science Foundation, the GK-12 project paired individual graduate students with a K-12 teacher for a school year to 1) Enhance graduate students’ skill in communicating science; 2) Equip K-12 teachers with tools to teach STEM topics related to climate change; and 3) Strengthen the pipeline of K-12 students into STEM careers, particularly those from underserved groups.

Our Work

A mixed method evaluation design was used to measure changes in graduate students’, teachers’, and K-12 students’ perceptions, knowledge, and attitudes. Through both formative and summative evaluation reporting, the project’s three main goals were examined using analysis of teacher and graduate student surveys, student interviews, and field observations at summer institutes.

Usefulness of Our Work

We provided evaluation findings that can be used to improve subsequent teacher and graduate student professional development projects.

Evaluation of Arctic Research Consortium of the United States (ARCUS) PolarTREC Program

Polar Teachers and Researchers Exploring and Collaborating (PolarTREC) has been successfully partnering K-12 teachers with polar research scientists since 2007. The program brings K-12 educators together with Arctic and Antarctic-focused scientists to give the teachers the experience of participating in active science. It also provides researchers with outreach support as well as enthusiastic field hands.

Our Work

The PolarTREC evaluation has utilized year-over-year program assessment and utilization of emergent themes to build a robust picture of this long-term program and further understand the program’s impacts on teachers, students, and researchers.  The evaluation measures teacher STEM content knowledge gains, teacher confidence to teach STEM, student interest in polar science, and long-term impacts of the project on teacher and researcher relationships.

Use

Our findings contribute to year-over-year program refinement to continually improve support for teacher learning and describe long-term impacts via annual alumni survey results.