Leveraging Portfolios in Professional Development for Middle School Science Teachers' Assessment and Data-Use Practice
Saved in:
| Title: | Leveraging Portfolios in Professional Development for Middle School Science Teachers' Assessment and Data-Use Practice |
|---|---|
| Language: | English |
| Authors: | Kloser, Matthew (ORCID |
| Source: | Science Education. Jul 2022 106(4):924-955. |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 32 |
| Publication Date: | 2022 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Junior High Schools Middle Schools Secondary Education |
| Descriptors: | Portfolios (Background Materials), Faculty Development, Middle School Students, Science Teachers, Evaluation Methods, Data Use, Teacher Attitudes, Student Evaluation |
| DOI: | 10.1002/sce.21712 |
| ISSN: | 0036-8326 |
| Abstract: | Using evidence of student thinking and performance is crucial to enacting ambitious science teaching because intimate, formative data can be responsive to student ideas about science. However, narratives and policies about assessment and data use do not always position teachers to draw on the breadth and variety of evidence from the classroom to make instructional decisions that foster sensemaking. Thus, understanding whether, when, and how teachers view and use assessment and other forms of student data is critical to understanding classroom interactions and learning. This study explores teachers' baseline assessment conceptions and practices as well as their changes with relation to assessment and data use across a year-long intervention using a portfolio of classroom assessment artifacts. Middle school science teachers participated in professional development in which they reflected on assessment artifacts in light of nine dimensions of effective science assessment and data-use practice. Drawing on interviews, as well as baseline and outcome portfolios from the classroom, we noticed that initial conceptions of practice focused on the variety and frequency of assessment. Nine months later, participants' practice showed growth in several dimensions, but differentially among teachers based on their baseline proficiency. Teachers were more likely to show growth in dimensions related to feedback and data use if they had initially showed proficiency in articulating clear learning goals and had previously aligned their assessments to the stated goals. These findings suggest a possible developmental trajectory for improving teachers' assessment and data use practice. |
| Abstractor: | As Provided |
| Entry Date: | 2022 |
| Accession Number: | EJ1337307 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEHqMaLWEhOJ10M4K2lsPi_AAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDEb5XQT9SUMoHZWJRgIBEICBmzw5plmp9eMOTG31zcO90Hfm4HyTNF3cooUogi1I0_xXtBQPimsiRTk8eb_XQP7OpxVfL5_qpWxo1pBQD8ND93GrFIlFaDe4JQt_tD8WPDskY2KnDqwslZmlREBYydfE9Ka8S5DlARXERBSSgdYlUs0tuLD_qsgjkEQQ3mv7abY74Z7vQS0KqQUkaCwJQ6_Mm_R3AiiyWy6CjlFH Text: Availability: 1 Value: <anid>AN0157236319;sed01jul.22;2022Jun06.09:01;v2.2.500</anid> <title id="AN0157236319-1">Leveraging portfolios in professional development for middle school science teachers' assessment and data‐use practice </title> <p>Using evidence of student thinking and performance is crucial to enacting ambitious science teaching because intimate, formative data can be responsive to student ideas about science. However, narratives and policies about assessment and data use do not always position teachers to draw on the breadth and variety of evidence from the classroom to make instructional decisions that foster sensemaking. Thus, understanding whether, when, and how teachers view and use assessment and other forms of student data is critical to understanding classroom interactions and learning. This study explores teachers' baseline assessment conceptions and practices as well as their changes with relation to assessment and data use across a year‐long intervention using a portfolio of classroom assessment artifacts. Middle school science teachers participated in professional development in which they reflected on assessment artifacts in light of nine dimensions of effective science assessment and data‐use practice. Drawing on interviews, as well as baseline and outcome portfolios from the classroom, we noticed that initial conceptions of practice focused on the variety and frequency of assessment. Nine months later, participants' practice showed growth in several dimensions, but differentially among teachers based on their baseline proficiency. Teachers were more likely to show growth in dimensions related to feedback and data use if they had initially showed proficiency in articulating clear learning goals and had previously aligned their assessments to the stated goals. These findings suggest a possible developmental trajectory for improving teachers' assessment and data use practice.</p> <p>Keywords: assessment; data‐use; portfolios; science education; science teaching practice</p> <p>Ambitious science teaching (AST) is a "coherent and accessible vision of highly effective teaching" (Windschitl et al., 2018, p. 3) designed to support students' conceptual and epistemic growth in science, regardless of background or prior achievement (Grinath &amp; Southerland, 2019). AST positions student thinking as a central focus of instruction that shapes how students come to understand core ideas of and how to engage in the practices of science (Thompson et al., 2013; Windschitl &amp; Calabrese Barton, 2016, Windschitl et al., 2012). This instructional and philosophical approach centers the work of science classrooms less on memorization of science facts and more on student sensemaking of disciplinary ideas via participation in more authentic science practices, such as constructing models, analyzing and interpreting data, and constructing arguments from evidence. Explicit in the AST framework is the premise that science teachers prioritize evidence of student thinking to adapt their instructional decisions to be responsive to student understanding.</p> <p>Concurrent with the evolution of AST, recent policy initiatives have also emphasized the important role of using evidence from within and beyond the classroom—including, but not limited to, evidence of student thinking—that is critical to informing ambitious science instruction. These policy initiatives, termed, "data‐driven decision making" (DDDM), include a broad range of practices that use not only intimate classroom data, but also achievement, affective, and other demographic data. Whether it is standardized test data, measures of student interest or identity in particular disciplines, or the disaggregation of behavioral records by demographic variables, DDDM rests on the assumption that careful analysis of evidence of student outcomes can support teachers' ongoing instructional decisions and allow for more targeted responses to students' needs (Hamilton et al., 2009).</p> <p>Unfortunately, while AST and DDDM share a common trait of valuing evidence from student work to make on‐going instructional decisions, the integration of these two frameworks may be more challenging in practice. DDDM approaches range in efficacy; some processes lead to sustained reflection on multiple data sources while others are a "sporadic examination of test results... [that do] not have the desired results" (Datnow &amp; Hubbard, 2016, p. 8). Consequently, in many classrooms, approaches to DDDM have not resulted in more ambitious science classroom environments (Kerr et al., 2006). This is particularly true in an era focused on high‐stakes testing. For the limited times when high‐stakes science tests are administered, the resulting data is often difficult to use formatively, as it is untimely and broad (Kerr et al., 2006; Marsh et al., 2006).</p> <p>Perhaps more problematic, many districts have advocated for extensive data use as a means to promote equity by identifying students' specific needs, only to enact deficit approaches in which data use counters the inequities they are trying to address, resulting in narrowed curriculum, inappropriate grouping, and "gap gazing" (Booher‐Jennings, 2005; Diamond, 2012; Horn, 2016; Settlage &amp; Meadows, 2002). Adding to these concerns, the assessments used for making decisions at different levels often take the form of comprehension‐level measures that mask how students make sense of complex core disciplinary ideas and practices. To address these challenges, teachers, administrators, and policymakers would be wise to interrogate the mindsets and practices of data use that drive DDDM in its various forms.</p> <p>If the goals of AST are to be realized, it is important to better understand how science teachers use evidence of student thinking to create validity arguments in relation to learning goals, what data they use, and how they make sense of student thinking to adapt instruction. Furthermore, given existing literature on the misuse of data within schools and classrooms (e.g., Kerr et al., 2006; Marsh et al., 2006), it is important to determine how, if at all, science teachers' conceptions about and practice of assessment and data use can be shaped by professional development (PD). Our project, The Quality Assessment in Science (QAS) PD study, begins to investigate middle school science teachers' conceptions and practices of assessment and data use before and after an intervention using a portfolio of artifacts and student work. Central to QAS PD is a teacher‐created portfolio of assessment artifacts and reflections called the <emph>Quality Assessment in Science Notebook</emph> (The Notebook). Over the course of an intensive summer institute and academic‐year PD, teachers collected multiple multiweek Notebooks that they brought to team‐based PD where they analyzed their assessment tasks and student work in light of nine dimensions of effective assessment practice. (<emph>Note</emph>: To differentiate between general dimensions or facets of a construct, we capitalize "Dimensions" through the remainder of this paper when referring to the specific framework of the nine "Dimensions of Effective Assessment Practice" used in this study). Specifically, we explore the following research questions:</p> <p></p> <ulist> <item> 1. How and when do a sample of middle school science teachers use assessments and evidence of student thinking to inform their instructional decisions before a common PD experience?</item> <p></p> <item> 2. What is the relationship between PD using a portfolio and an explicit framework for practice (the Dimensions) and changes in teachers' assessment and data‐use practice over time?</item> <p></p> <item> 3. How do teachers' conceptions of quality assessment and data use relate to their actual practice in the classroom?</item> </ulist> <hd id="AN0157236319-2">THE QAS STUDY: FRAMING THE WORK</hd> <p>A decade ago, Coburn and Turner (2011) noted the field's lack of an evidentiary basis for data‐use practice. Much work has been done over the past ten years to help us better understand the high variability of data‐use practice (Datnow &amp; Hubbard, 2016), the nuanced differences between DDDM and teachers' assessment literacy (Xu &amp; Brown, 2016), and how critical notions of DDDM can move from deficit approaches to those focused on promoting equity (Dodman et al., 2021). Of note, a more recent literature review on DDDM indicated two findings relevant to our study. First, while the DDDM theory of action embraces a range of data sources, benchmark and standardized test data have traditionally held higher status as a result of district influence. Second, the collaboration model for analyzing and using data to inform instruction is the most common model, but many of these collaborations lack structured training on what to do with the data once it is accessed (Datnow &amp; Hubbard, 2016).</p> <p>To further the discussion on these two points, the QAS PD study helps contribute to our on‐going understanding of data use in science classrooms by exploring: (<reflink idref="bib1" id="ref1">1</reflink>) whether, when, and how science teachers use both intimate and benchmark assessment evidence of student thinking to adapt instruction; and (<reflink idref="bib2" id="ref2">2</reflink>) how science teachers can grow and improve their practice through a collaborative PD experience that uses structured training around a portfolio of classroom artifacts. The following sections elaborate on both the theoretical underpinnings and the practical structures of how this project worked with small groups of teachers from four schools who participated in a year‐long, facilitated professional learning community (PLC) that focused on specific aspects of effective assessment and data‐use practice.</p> <hd id="AN0157236319-3">BACKGROUND LITERATURE AND CONCEPTUAL FRAMEWORK</hd> <p>Our project sits at the intersection of three conceptual frameworks: (<reflink idref="bib1" id="ref3">1</reflink>) A framework for high‐quality PD; (<reflink idref="bib2" id="ref4">2</reflink>) Dimensions of quality assessment practice; and (<reflink idref="bib3" id="ref5">3</reflink>) A theory of effective data use. When synthesized, these three distinct frameworks guided how we engaged adult learners collaboratively in a PD experience that drew on evidence from the classroom to shift practice in ways that reflect more AST and the effective use of data to inform instruction. The Dimensions were the primary framework used to shape our analysis of the research data. We address each framework in the context of describing the respective elements of the project and how they were motivated by existing literature.</p> <hd id="AN0157236319-4">Framing effective professional development</hd> <p>The research team adopted a sociocultural perspective (Vygotsky, 1978) of adult learning and assessment (Penuel &amp; Shepard, 2016a) that helped shape the use of the Notebook in the PD experience. This lens privileged a collaborative, dialogic environment, and like the fundamental principles of AST, valued participating teachers' voices, unique backgrounds, and their diverse school contexts. Our PD design was informed by the following elements: teachers' collective participation, teachers' active engagement in meaning making, PD experiences that coherently aligned the lived experience of the teachers in schools and the PD, planning tasks and conversations that were centrally focused on science content, and an experience that allowed for extended interaction together (Darling‐Hammond et al., 2017; Desimone, 2009; Jeanpierre et al., 2005; Penuel et al., 2007).</p> <p>We invited middle grades science teachers to participate as small teams, recognizing that developing communities of practice can only occur through on‐going interactions focused on common work (Lave, 1991; Vázquez‐Bernal et al., 2012; Wenger, 1998). Working with school teams allowed for collective professional formation (Desimone, 2009) in which teachers engaged with ideas and artifacts of practice during a summer institute and throughout the year as part of a PLC. In the summer institute, teachers were exposed to new ideas through active participation in inductive tasks that were drawn from the classroom. Working collaboratively, they reviewed and provided feedback on student‐ and teacher‐created artifacts from their own science classrooms and those of colleagues (Borko, 2004; Johnson &amp; Fargo, 2010).</p> <p>Through individual reflection and extended opportunities for small and whole group talk, teachers developed a learning community that could support their implementation of effective assessment and data‐use practices in their classrooms (Darling‐Hammond et al., 2017; Johnson et al., 2016). Extending the collective work throughout the school year also helped support opportunities for an improved culture around assessment and data‐use practice (Wenner &amp; Campbell, 2017).</p> <p>The PD experience was also supported with both conceptual and practical tools that provided a bridge between existing levels of expertise and the goal of a given task (Grossman et al., 1999). The two central tools used in this study were the Notebook—a practical tool, and the Dimensions of quality assessment practice—a conceptual tool. Teachers used the Notebook to collect sets of instructional artifacts across consecutive days of a science unit for reflection and discussion (described more fully in the Section 3). The Notebook allowed participating teachers to view assessment artifacts taken from their own teaching context, as well as collect evidence of tasks and student work, which provided the raw materials upon which growth could be monitored (Antoniou &amp; Kyriakides, 2013). The physical tool of the Notebook focused the professional learning on science content and classrooms.</p> <p>In addition, it was important to pair this practical tool with the conceptual tool of the Dimensions, as existing studies of PLCs have documented instances of conversations about artifacts that either failed to interrogate practice or that were tangential to student learning altogether (Nelson et al., 2012; Slavit et al., 2013; Turner et al., 2018). The Dimensions (described more fully below), helped facilitate teachers' thinking about crucial elements of practice related to artifacts collected in the Notebook. They also facilitated the development of a common language and understanding among the teachers and PD facilitators about aspects of assessment and data use. While the Dimensions exist as nine discrete entities consisting of definitions, examples, and a five‐level continuum of practice, they were operationalized as interacting parts of a broader assessment system or approach. These Dimensions guided our discussions and analysis of artifacts and practice that took place over the course of 12 months, with over 40 h of direct contact and 20 more hours of individual collection and artifact annotation. The duration of the project surpassed the minimum of 20 h suggested by some PD literature as necessary for effecting change (Desimone, 2009; Johnson et al., 2016).</p> <hd id="AN0157236319-5">Framing dimensions of quality assessment practice</hd> <p>Research has shown that quality assessment can significantly improve students' understanding of and participation in the sciences (Black &amp; Wiliam, 2005; Penuel &amp; Shepard, 2016b). Assessment serves not only as a measure for what students know and can do in relation to a standard or learning goal, but also as a guide to students and teachers about student thinking and what learning opportunities can help further that thinking. Unfortunately, diversions from this active and formative approach to assessment have prompted one researcher to note that "assessment illiteracy abounds" (Stiggins, 2010, p. 233). Our model builds on theory and practices from existing literature (including Briscoe &amp; Wells, 2002; Chin &amp; Brown, 2002; Earl, 2012; Furtak, 2009; Havnes et al., 2012; Martínez, Borko, &amp; Stecher, 2012; National Research Council, 2001a, 2001b; Shavelson et al., 2003; Xu &amp; Brown, 2016) and the <emph>Framework for K‐12 Science Education</emph> (National Research Council, 2012), to develop nine Dimensions (Table 1) of effective science assessment practice that, when practiced, would indicate both assessment literacy and ambitious teaching. Whereas our framework for PD guided how we worked with and learned from groups of teachers, our synthesis of the literature resulting in the Dimensions provides a framework for how we discussed assessment practice and for how we analyzed data from the project in terms of shifting conceptions and practices of teachers. We organized these Dimensions in groups based on an analogy of a GPS navigation system that, like teachers with assessments, addresses the following questions:</p> <p></p> <ulist> <item> Where are you going?;</item> <p></p> <item> Where are you now?; and</item> <p></p> <item> How can you get there? (Martínez, Borko, &amp; Stecher, 2012; National Research Council, 2001a).</item> </ulist> <p>1 TableCategorization of Dimensions aligned to guiding question</p> <p> <ephtml> &lt;table&gt;&lt;thead valign="bottom"&gt;&lt;tr valign="bottom"&gt;&lt;th align="left"&gt;National Research Council (2001a)&lt;/th&gt;&lt;th align="left"&gt;Revised Dimensions from Mart&amp;#237;nez, Borko, Stecher, et al. (2012)&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td align="left"&gt;Q1: Where are you going?&lt;/td&gt;&lt;td align="left"&gt;1. Set Clear Learning Goals (Set Goals)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td /&gt;&lt;td align="left"&gt;2. Align Assessments to Learning Goals (Align Assessments)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Q2: Where are you now?&lt;/td&gt;&lt;td align="left"&gt;3. Assess Frequently (Frequency)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;4. Vary Assessment (Variety)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;5. Assess with Appropriate Cognitive Complexity (Cognitive Complexity)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;6. Reflect Scientific Practices and Core Ideas (Science Practices)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;7. Involve Students in Self&amp;#8208;Assessment (Student Involvement)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Q3: How can you get there?&lt;/td&gt;&lt;td align="left"&gt;8. Provide Specific Feedback (Feedback)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;9. Use Evidence of Student Thinking to Adapt Instruction (Adapt Instruction)&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <p>1 <emph>Note</emph>: Shortened titles are shown in parentheses that are used throughout the manuscript.</p> <hd id="AN0157236319-6">"Where are you going?" Dimensions</hd> <p>Validity arguments about student achievement lie in the quality of the relationship between the learning goal and the evidence of student thinking and skills (Downing, 2003; Sullivan, 2011). Generating a valid argument that can thoughtfully inform decisions in planning and instruction necessitates a clear understanding of the desired outcome; therefore, the learning goal must be clear to both the student and the teacher. Setting explicit and developmentally appropriate goals ("Set Goals", Dimension 1) is essential for both students and teachers to focus their learning and move productively toward the three‐dimensional outcomes valued in science education. Explicit goal setting has been related to positive impacts on student learning (DeShon &amp; Alexander, 1996; Lipsey &amp; Wilson, 1993). But knowing the intended learning goal alone does not create an environment in which valid arguments can be made about students' achievement or directions for future instruction. Assessments—whether informal questions from teacher to student, more formal quizzes at the end of a lesson, or long‐term projects—must align with the constructs they are intended to measure ("Align Assessments", Dimension 2) and reflect attributes of the goals so as to provide meaningful evidence. In looking at alignment, teachers must carefully work to ensure that the assessment measures the stated goal, to the exclusion of other variables such as reading ability, that may, for example, misrepresent students' conceptual thinking about core science ideas.</p> <hd id="AN0157236319-7">"Where are you now?" Dimensions</hd> <p>Understanding students' wide range of ideas and abilities requires a host of Dimensions that often work in concert. For both students and teachers, the frequency ("Frequency", Dimension 3) and the variety ("Variety", Dimension 4) of ways in which student thinking is captured contributes to the portrait of what students know and are able to do in relation to the unit goal. Nearly a century of cognitive studies have shown that more frequent assessment can positively influence student learning, providing students an opportunity to consolidate information and monitor their status in relation to the learning goal (e.g., Roediger &amp; Karpicke, 2006; Rowland, 2014). Furthermore, providing students opportunities to show their understanding in different ways (e.g., open‐ended questions vs. restricted response or performance tasks vs. traditional tests) provides different forms of evidence that can be used in making a valid argument about the impact of instruction and student needs going forward (Treagust et al., 2001). Furthermore, providing students diverse avenues for showing their thinking has the potential to create trust between students and teachers that evaluations of their performance are valid.</p> <p>Crafting assessments that capture students' science outcomes must attend to both science practice and content. The <emph>Framework for K–12 Science Education</emph> (National Research Council, 2012) establishes a core set of science and engineering ideas that are central to the physical, biological, and earth sciences. Furthermore, the three‐dimensional organization of the framework asks that students not only understand core ideas, but also come to understand them by engaging in the practices of science (e.g., Manz, 2012). This three‐dimensional structure requires that students comprehend ideas and apply, analyze, and evaluate their thinking about natural phenomena in increasingly complex ways. An instructional unit's suite of assessments benefits both teachers and learners when they reflect the varying levels of cognitive demand required to acquire and use core conceptual science ideas ("Cognitive Complexity", Dimension 5) and recognize that science practices are critical features to participating in science ("Science Practices", Dimension 6).</p> <p>Finally, assessments may act as learning tools for both teachers <emph>and</emph> students (Black et al., 2003). Involving students in their own assessment ("Student Involvement", Dimension 7) can take multiple forms and has been shown to have an impact on student learning. For example, students can reflect on rubric criteria before and throughout a performance assessment. Analysis of one's work in light of exemplar responses can help students take ownership in evaluating their current relationship to the learning goal (Falchikov, 2004). Importantly, explicit opportunities for student self‐reflection have shown the greatest impact for historically low‐performing students (White &amp; Frederickson, 1998).</p> <hd id="AN0157236319-8">"How can you get there?" Dimensions</hd> <p>Dimensions 1–7 describe assessment features that contribute to validity and reliability arguments about measuring student outcomes in reference to the robust goals of the <emph>Framework for K–12 Science Education</emph>. The final set of Dimensions focus on, "how do we move from our existing understanding and skill to more fully addressing the learning goal?" Dimension 8—related to providing quality feedback ("Feedback")—and Dimension 9—related to using the information to inform and adapt instruction ("Adapt Instruction")—are mutually supportive elements of high‐quality practice. Although these Dimensions work closely together, we parse out the process of conveying information <emph>to students</emph> in Dimension 8 and information used <emph>by teachers</emph> in Dimension 9.</p> <p>Specific feedback has been shown empirically to have a large average effect on student achievement. In a meta‐analysis of nearly 200 studies on feedback, an average effect size on student achievement of 0.79 was reported for instances in which quality feedback was provided (Hattie &amp; Timperley, 2007). However, despite its promise, the creation and use of feedback can be complicated. Numeric percentages or letter grades provide a message to students, but little in the way of helping them move their thinking toward the learning goal (Guskey, 2019). In addition, the way in which students receive feedback can depend on the trust and relationship that is built between the student and teacher wherein noncognitive factors can influence how feedback is received (Cohen et al., 1999). In contrast, quality feedback between a student and trusted teacher focuses on the mastery of topics with specific questions or comments that can improve student performance or understanding (Butler, 1988; Wiggins, 2012; Yeager et al., 2014). Of course, this type of feedback can be laborious for teachers, although new technologies can aid the process with saved comment banks and annotation tools.</p> <p>The final Dimension, using evidence of student thinking to adapt instruction ("Adapt Instruction", Dimension 9), is a key component of the feedback cycle that can potentially close the gap between learning goals and students' current understandings and skills (Treagust et al., 2001). This Dimension is at the heart of formative assessment in that assessments are only formative so long as they <emph>inform</emph> what happens instructionally going forward. As this Dimension is nested within a broader theory of data use and given its prominence in this study, it is described in more detail in the following section.</p> <hd id="AN0157236319-9">Framing a theory of effective data use</hd> <p>The focus on data use over the past two decades has emerged from standards‐based accountability educational policies such as NCLB (Braaten et al., 2017; McDonald et al., 2018). These policies have advocated for rigorous content standards and accountability through standardized assessments from which decisions are made about instructional quality and, in theory, changes are made accordingly (Mehta, 2013). This model of educational data use was theorized as a way of addressing inequities among students. Unfortunately, literature from the past two decades has shown that the use of data with this approach has often exacerbated inequalities, partly because certain subpopulations are singled out for remediation and viewed with a deficit lens (Diamond, 2012; Horn, 2016; Settlage &amp; Meadows, 2002) and partly because the use of data is not neutral, but situated in relation to histories, incentives, and other contextual forces (Dodman et al., 2021; Wang, 2019). In some instances, the focus on "instructional management" (Horn et al., 2015) has resulted in teachers shifting toward didactic instruction to maximize test scores with little concern for students' sensemaking of core disciplinary ideas or engagement with disciplinary practices. Furthermore, many of the data‐centric initiatives define what "counts" as data so narrowly that teachers have few opportunities to think about what data should be used, how they can be analyzed, and what changes can occur for students in need (Braaten et al., 2017; Horn et al., 2015).</p> <p>Mehta (2013) describes another, potentially more productive, framework for data use in which the development of teachers as professionals is central. Rigorous content standards are still important, but data are defined, interpreted, and acted upon in concert with developing pedagogical practice that address the findings in the data (Datnow &amp; Hubbard, 2016). This latter approach, what Horn et al. (2015) calls the "instructional improvement" approach, addresses equity in practice by using data to investigate student thinking and then aligning instructional practice to help students achieve the learning goals. An instructional improvement lens is particularly important in a science classroom as attention to standardized test scores limits the type of data that align with the current consensus on quality science learning experiences (Rivera Maulucci, 2010).</p> <p>The conceptual framework for data use that underlies our PD is rooted in an instructional improvement approach to achieve more AST. We acknowledge that data alone do not improve students' science sensemaking or engagement in science practices, but rather that teachers have to respond to data through their practice. The Notebook was a tool that allowed teachers in this study to look at "intimate data" (McDonald et al., 2018)—data that have the potential to reflect evidence of student thinking very proximally to learning experiences rather than annual standardized data that provide only "the distant mirror of achievement" (p. 84). The Notebook also provided space for medial data, such as local district benchmarks that can occur at more frequent intervals than the annual assessments, but still exist outside the context of unit‐based science learning.</p> <p>Our framing of Dimension 9 ("Adapt Instruction") was influenced by the cycle of data use outlined by Boudett et al. (2005), but was adapted to address both benchmark and more intimate classroom data. DDDM logics often recognize that data use is an active, iterative process in which evidence of student thinking must be transformed into actionable information that can improve instruction and ultimately result in better student sensemaking of core science ideas (Datnow &amp; Hubbard, 2016). Effective teachers do not merely gather data and then continue to move through their curriculum. Rather, they use information about student thinking and competencies to make instructional decisions for students (Treagust et al., 2001). This process is unnatural for many teachers and requires support (Celio &amp; Harvey, 2005).</p> <p>The data‐use framework guides teachers to identify pieces of data aligned with the learning goals at hand, analyze the data—in our PD this included both qualitative and quantitative analysis—and then identify what the data say about student thinking in relation to making sense of science ideas. Analyzing student thinking from different forms of assessment can sometimes stop here. For example, Shepard et al. (2011) showed that teachers working with benchmark data identified which questions were missed without providing specific ways in which students could better meet the learning objective. In contrast, our PD pushed teachers to reflect on their practice and the opportunities for future instruction that might help students develop their thinking to be more aligned with canonical understandings of science ideas. We placed substantial emphasis on adapting instruction that moves beyond reteaching or remediation—the "louder and slower" approach—pushing teachers to identify specific instructional improvements that provide novel ways for students to engage content and make sense of big science ideas. Thus, Dimension 9, Adapting Instruction, provides teachers guidance through iterative cycles of preparing assessments to yield high‐quality data, inquiring about the data, and developing an instructional action plan (Van Geel et al., 2016).</p> <p>Viewed together, the Dimensions and framework for effective data use influenced the decisions we made about the content of the professional learning experience. The Dimensions provided guidance for <emph>what</emph> content the teachers engaged and the connections across that content while our framework for effective PD guided <emph>how</emph> we put teachers in positions to make sense of new ideas and practices.</p> <hd id="AN0157236319-10">METHODS</hd> <p>The following section provides an overview of the methods, including background information on the participating teachers, our research instruments, and our data analysis plan. To supplement the above short description of the project, we detail how the Notebook was used and the different elements of the PD experience.</p> <hd id="AN0157236319-11">Sample</hd> <p>In this study, we sought to investigate teachers' data use in‐depth. Thus, we focused on a small, longitudinal sample. Twelve public middle school teachers participated in this study. Participants were recruited as teams of at least two and no more than five teachers from a given school. Eligible participants were required to teach at least one middle grade (6th–8th) science class, teach in a school that used benchmark science assessments, and agree to participate in a PLC.</p> <p>Recruitment occurred in two states—California and Indiana—to view how state and district context might influence teachers' uptake of the PD. Due to attrition related to illness and course schedule change, respectively, two teachers did not have complete data and were not used in the final analysis. Participant and school information for the ten teachers included in this analysis is displayed in Table 2.</p> <p>2 TableParticipant and school information</p> <p> <ephtml> &lt;table&gt;&lt;thead valign="bottom"&gt;&lt;tr valign="bottom"&gt;&lt;th align="left"&gt;State&lt;/th&gt;&lt;th align="left"&gt;School0001&lt;/th&gt;&lt;th align="left"&gt;Participants0001&lt;/th&gt;&lt;th align="left"&gt;Curriculum source&lt;/th&gt;&lt;th align="left"&gt;Enrollment&lt;/th&gt;&lt;th align="left"&gt;Female %&lt;/th&gt;&lt;th align="left"&gt;Minoritized student %&lt;/th&gt;&lt;th align="left"&gt;Student FRL %&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td align="left"&gt;CA&lt;/td&gt;&lt;td align="left"&gt;Washington MS&lt;/td&gt;&lt;td align="left"&gt;Abby&lt;/td&gt;&lt;td align="left"&gt;Textbook&lt;/td&gt;&lt;td align="left"&gt;1227&lt;/td&gt;&lt;td align="left"&gt;50&lt;/td&gt;&lt;td align="left"&gt;28&lt;/td&gt;&lt;td align="left"&gt;5&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Nancy&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td /&gt;&lt;td align="left"&gt;Amy Li MS&lt;/td&gt;&lt;td align="left"&gt;Wayne&lt;/td&gt;&lt;td align="left"&gt;Kit (e.g., FOSS)&lt;/td&gt;&lt;td align="left"&gt;582&lt;/td&gt;&lt;td align="left"&gt;50&lt;/td&gt;&lt;td align="left"&gt;39&lt;/td&gt;&lt;td align="left"&gt;35&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Erica&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;IN&lt;/td&gt;&lt;td align="left"&gt;Adventure MS&lt;/td&gt;&lt;td align="left"&gt;Martha&lt;/td&gt;&lt;td align="left"&gt;Kit&lt;/td&gt;&lt;td align="left"&gt;908&lt;/td&gt;&lt;td align="left"&gt;51&lt;/td&gt;&lt;td align="left"&gt;20&lt;/td&gt;&lt;td align="left"&gt;14&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Brent&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Danielle&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Wendy&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td /&gt;&lt;td align="left"&gt;John Glenn MS&lt;/td&gt;&lt;td align="left"&gt;Bob&lt;/td&gt;&lt;td align="left"&gt;Kit&lt;/td&gt;&lt;td align="left"&gt;783&lt;/td&gt;&lt;td align="left"&gt;53&lt;/td&gt;&lt;td align="left"&gt;24&lt;/td&gt;&lt;td align="left"&gt;64&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Harriet&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <p>2 a Pseudonyms.</p> <hd id="AN0157236319-12">Intervention overview</hd> <p>This study sought to explore and improve middle school science teachers' assessment and data‐use practice using a structured portfolio, a summer institute, and a school‐year PLC. The intervention targeted teachers' conceptions of assessment and data‐use practice by providing a conceptual framework of Dimensions of assessment and data‐use practice identified in the literature, described above. It supported changes in their classroom practice by providing strategies related to the Dimensions and opportunities for reflection and discussion in the artifact collection process and the professional learning experiences.</p> <hd id="AN0157236319-13">QAS Notebook</hd> <p>Participating teachers collected four portfolios (Notebooks) of classroom artifacts as both an intervention and a research instrument (two 10‐day Notebooks, which bookended the intervention, and two 5‐day Notebooks, which were collected for discussion in the PLC meetings). Teachers were asked to collect assessment artifacts from their practice as well as lesson plans and samples of student work. They were also asked to complete reflections for each artifact on a prelabeled sticky note. The reflections asked the teachers to provide context for the artifact, identify the levels of student achievement for each artifact, and reflect on how the artifact might inform their teaching. A full description of the Notebook's structure is provided in the research instrumentation section below.</p> <hd id="AN0157236319-14">Summer institute PD</hd> <p>Teachers attended one of two identically planned, intensive (∼25 h) summer institutes focused on developing assessment and data‐use literacy, one in Indiana and the other in California. The PD was structured around two main tools: (<reflink idref="bib1" id="ref6">1</reflink>) the nine Dimensions of assessment and data‐use practice (Martínez, Borko, Stecher, et al., 2012; Wilsey et al., 2020) and (<reflink idref="bib2" id="ref7">2</reflink>) a set of Notebooks including two representative Notebooks from other teachers and a 10‐day baseline Notebook of artifacts that teachers collected before the intervention. Our PD was tightly aligned with the framework of effective professional formation described above and was founded on a principle of colearning amongst the facilitators and the participants (Darling‐Hammond et al., 2017; Desimone, 2009).</p> <p>To provide various ways of showing their thinking, participants were asked to draw representations of their mental models of quality assessment practice and discuss their initial conceptions with other participants. Next, they were given a 5‐day Notebook of artifacts collected from a previous study (Martínez, Borko, Stecher, et al., 2012). Teachers were asked to review the artifacts and inductively construct a set of Dimension categories based on the positive or negative examples found within the sample Notebook. These categories were discussed in light of the nine formal Dimensions presented by the facilitators.</p> <p>The PD proceeded with a focus on each Dimension organized by the central questions, "Where are you going?," "Where are you now?," "How do you get there?." Participants engaged in pair or small group analysis tasks as a means for inductively deriving the main elements of each Dimension. The approach—working inductively based on teachers' experience—reflected important aspects of adult learning theory (Knowles et al., 2012). After completing this activity for each Dimension, the Dimension description and rubric were formally introduced.</p> <p>An extended amount of time was spent working with Dimension 9. Several explicit strategies were provided for using evidence of student thinking to adapt future instruction. Teachers were also introduced to several spreadsheet tools that aided the analysis of whole‐class, quantitative performance data. Case studies provided an opportunity for participants to apply their new understandings and practice using these tools to make decisions about on‐going instruction.</p> <p>Teachers also learned by analyzing a highly‐rated Notebook taken from a previous study that provided high‐quality representations of most Dimensions. After discussing the assessment and data‐use practice in the exemplar Notebook, participants rated their own 10‐day baseline Notebooks and provided written justification of their ratings. Opportunities for written reflection in the form of exit cards and journal entries were provided throughout the PD.</p> <p>The summer institute concluded with teachers drawing and discussing new representations of their assessment and data‐use mental models. An additional half‐day PD positioned teachers to work with the Dimensions in the first month of the new school year.</p> <hd id="AN0157236319-15">PLC meetings</hd> <p>PLCs at each school met six times during the school year, three per semester. Teachers collected a 5‐day Notebook early in the fall and spring semesters, which served as the raw material for analysis and reflection during each meeting. Each PLC meeting was facilitated by one member of the research team and focused on one or two of the Dimensions, in combination with Dimension 9, Adapt Instruction (Table 3). Structured prompts or artifacts were examined and discussed before asking teachers to find evidence of or opportunities for achieving the Dimension's goals in their Notebooks. Each PLC had a formal opportunity for written reflection on practice.</p> <p>3 TablePLC content overview</p> <p> <ephtml> &lt;table&gt;&lt;thead valign="bottom"&gt;&lt;tr valign="bottom"&gt;&lt;th align="left"&gt;PLC session&lt;/th&gt;&lt;th align="left"&gt;Focal dimensions&lt;/th&gt;&lt;th align="left"&gt;Brief session description&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;td align="left"&gt;8, 9&lt;/td&gt;&lt;td align="left"&gt;Formative assessment and types of feedback provided to students&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td align="left"&gt;5, 8, 9&lt;/td&gt;&lt;td align="left"&gt;Summative assessment and using statistics to inform instruction&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;3&lt;/td&gt;&lt;td align="left"&gt;2, 3, 5, 8, 9&lt;/td&gt;&lt;td align="left"&gt;Holistic review of 5&amp;#8208;day Notebook and assessment practice&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;4&lt;/td&gt;&lt;td align="left"&gt;Variable&lt;/td&gt;&lt;td align="left"&gt;Differentiated analysis of focal Dimension for each teacher&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;5&lt;/td&gt;&lt;td align="left"&gt;5, 9&lt;/td&gt;&lt;td align="left"&gt;Using quantitative analysis to inform instruction&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;6&lt;/td&gt;&lt;td align="left"&gt;9&lt;/td&gt;&lt;td align="left"&gt;Benchmark assessments and using benchmark data&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <p>3 Abbreviations: PLC, professional learning community.</p> <hd id="AN0157236319-16">Instruments and data sources</hd> <p>Though multiple sources of data were collected for this project, this paper focuses on two sources: (<reflink idref="bib1" id="ref8">1</reflink>) pre‐/poststudy interviews; and (<reflink idref="bib2" id="ref9">2</reflink>) The Notebook. Jimerson (2014) notes that teachers' data‐use practice is tied to their mental models of what counts as assessment, evidence, and the interpretation of evidence. Relatedly, drawn conceptual models of assessment and data‐use practice were also collected as key artifacts of both the PD and research. Analysis of these drawn mental models can be found in Wilsey et al. (2020), and the findings from that study are revisited in light of the interviews and Notebook artifacts in Section 5. A summary and chronology of data collection across the three instruments is represented in Figure 1.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/SED/01jul22/sce21712-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="sce21712-fig-0001.jpg" title="1 Project intervention and data collection timeline" /> </p> <p></p> <p>Pre and poststudy interviews were conducted in‐person by the research team. The prestudy interview focused on four constructs. First, teachers were asked about what types of assessments they used, examples of these assessment types, what information was obtained from each assessment and how, if at all, the data were used for each of three major assessment categories: summative, formative, and benchmark. Second, teachers were asked how students participated in their own assessment, especially in terms of when and how they worked with teacher feedback. Third, teachers provided context about their typical science instruction and classroom interactions. Finally, teachers reflected on the strengths and weaknesses of their existing assessment and data‐use practices.</p> <p>Poststudy interviews again focused on the categories of summative, formative, and benchmark assessments and asked teachers to think about the most important elements of each assessment type and the value of those assessments. Teachers were also prompted to talk about elements of data‐use practice. They then reflected on the nine Dimensions and identified the Dimensions in which they had grown the most over the past year and Dimensions that remained areas for growth. Before reflecting for a final time on the entire project, teachers engaged in a stimulated recall exercise based on the artifacts in their final Notebook in which they were asked to select and then discuss artifacts that represented their growth in assessment practice during the project as well as artifacts that provided evidence of student thinking.</p> <p>The Notebook was a structured portfolio in which teachers collected artifacts of assessments and reflected on how assessments were used, if at all, to inform future instruction. Each Notebook contained an initial folder, 5 or 10 daily folders, a concluding folder, and a benchmark folder. Participants were asked to include in the initial folder a baseline reflection, unit plan, and any relevant standards used for planning. Each daily folder captured the selected artifacts from the classroom, including two copies of student work with each assessment, preferably representing higher and lower performance on the task. Each artifact and its related student work were annotated with a sticky note containing preprinted prompts, including: (a) the assessment's purpose; (b) the achievement level of students on the assessment; and (c) how, if at all, the results from the assessment would impact instruction. The concluding folder included a final reflection as well as any major summative assessments that did not occur in the context of the collection period. Finally, the benchmark folder contained benchmark assessments that were taken most proximal to the Notebook collection period and student data on the assessment.</p> <hd id="AN0157236319-18">Data analysis</hd> <p>Above, we described three conceptual frameworks that informed both how we worked with participating teachers and the content that was the focus of those interactions. Our analytic framework, however, draws exclusively on our development of the Dimensions. These nine Dimensions were drawn from the literature and shaped into continua of practice for each Dimension (Table 4 provides a summary of the ratings and Supporting Information Appendix provides the full scales). Interviews as well as Notebook artifacts were coded according to these Dimensions.</p> <p>4 TableNine Dimensions of effective assessment practice</p> <p> <ephtml> &lt;table&gt;&lt;thead valign="bottom"&gt;&lt;tr valign="bottom"&gt;&lt;th align="left"&gt;Dimension name&lt;/th&gt;&lt;th align="left"&gt;Dimension description (highest level)&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td align="left"&gt;1. Setting clear learning goals&lt;/td&gt;&lt;td align="left"&gt;The learning goals and/or performance expectations for the unit and the individual lessons are explicit and clear, proximal to the learning unit, and connections among the learning goals are apparent&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;2. Aligning assessments to goals&lt;/td&gt;&lt;td align="left"&gt;In combination, the collection of formal and informal assessments provides complete information about the extent to which students have achieved the learning goals for the unit that are based on appropriate science standards&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;3. Assessing frequently&lt;/td&gt;&lt;td align="left"&gt;Assessment of one form or another is a regular occurrence in the classroom throughout the unit; at least one informal assessment is present nearly every day throughout the unit and at least one formal assessment is present every 3&amp;#8211;5 days&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;4. Varying assessment&lt;/td&gt;&lt;td align="left"&gt;The teacher uses a wide variety of formal and informal assessment methods (quiz, test, projects, oral q&amp;a) and forms (multiple choice, short answer, open ended, thumbs up/down, clickers)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;5. Assessing with appropriate cognitive complexity&lt;/td&gt;&lt;td align="left"&gt;Higher order cognitive processes, such as applying, analyzing, or explaining complex scientific concepts are a central focus of the assessment. Higher order questions are novel to students, not the reconstruction of ideas constructed entirely by the teacher&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;6. Reflecting math/science practices&lt;/td&gt;&lt;td align="left"&gt;Assessments require students to engage and show competency in scientific practices. Assessments collectively indicate students' ability to engage in at least one practice identified in the Next Generation Science Standards as a means to construct explanations about the natural and material world. Assessments collectively indicate that scientific practices are central to science achievement&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;7. Involving students in own assessment&lt;/td&gt;&lt;td align="left"&gt;Student self assessment is a consistent feature in the classroom; self&amp;#8208;assessment goes beyond summative judgment and involves reflecting on the quality of understanding and progress toward the learning goals&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;8. Providing specific feedback&lt;/td&gt;&lt;td align="left"&gt;Students consistently receive feedback from their teacher or peers that focuses on their learning, highlights potential strengths and weaknesses&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;9. Using evidence of student thinking to adapt instruction&lt;/td&gt;&lt;td align="left"&gt;The teacher consistently uses the information collected through assessment to inform decisions to adapt and improve instruction and assessment for their current students&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <p>4 <emph>Note</emph>: Dimension descriptions only include text from the highest level on the 5‐point Likert scale rubric designed for this study.</p> <hd id="AN0157236319-19">Analysis of interviews</hd> <p>Interviews were coded by two members of the research team using DeDoose™ (2018) Parent and child codes were created for each of the major assessment types that participating teachers could mention: formative, summative, and benchmark, as well as each of the nine Dimensions. Specific interview questions that highlighted the strengths and areas for growth in participants' practice were also coded based on the Dimensions. Interviews were coded independently and then reconciled to 100% agreement.</p> <p>Frequency of assessment type and Dimension reference were calculated across the sample. After creating frequency tables for assessment type and Dimension references, we identified patterns in thought by school and qualitatively by how teachers talked about aspects of each Dimension. Using data memos, we then summarized the major themes that reflected teachers' assessment conceptions.</p> <hd id="AN0157236319-20">Notebook artifact analysis</hd> <p>The 10‐day baseline and final Notebooks were rated by two members of the research team blind to state, school, and time (10 baseline and 9 final, due to illness attrition). Ratings were based on the Dimensions' 5‐level rubrics (see Table 4 to see the Level 5 rating description for each Dimension and Supporting Information Appendix for the full rubric). A holistic rating of the entire set of artifacts also was scored on a five‐point scale. Raters recorded evidence from the Notebook to justify each rating. Two Notebooks from teachers not included in the study served as training Notebooks and halfway through rating, all raters were assigned the same Notebook as a reliability check. Agreement exceeded 0.80 on each Dimension.</p> <hd id="AN0157236319-21">RESULTS</hd> <p>The results below are organized to provide evidence of a group of teachers' assessment and data‐use practice and how that practice and their conceptions changed, if at all, through a professional growth opportunity. We first present data about teachers' initial states, beginning with their conceptions and then exploring their practice. We then compare these initial states to how conceptions and practices shifted over time, both right after the summer institute and at the end of the project.</p> <hd id="AN0157236319-22">Teachers' initial assessment and data use</hd> <p>Evidence of teachers' initial conceptions about, knowledge of, and practice of assessment and data use varied not only across teachers, but also, at times, across data sources. In some instances, teachers' conceptions of assessment and data‐use practice as well as evidence from their actual practice reflected elements of AST. At other times, a mismatch existed between what teachers said in interviews and the artifacts shared from their classroom practice.</p> <hd id="AN0157236319-23">Evidence from interviews</hd> <p>Across the four schools, teacher interviews emphasized the need for assessment practice that pushed students to show cognitively deep understanding of core science ideas. Especially when talking about summative assessments, six of the teachers juxtaposed the use of textbook or kit‐based traditional tests that probed rote memorization and basic comprehension with supplemental questions that they included to focus on "critical thinking" and "deep understanding." For example, Erica valued cognitively complex assessment items in which students engaged in scientific practices stating, "[I want students to have] question[s] that they have to think outside the box" especially in terms of "developing arguments with claims, evidence, and reasoning." Similarly, Wayne desired to push past low‐level thinking as he talked about his assessments "looking for concept integration, 'how much did you remember,' but [also] looking for conceptual operation." He acknowledged that declarative knowledge had a place in assessment, but placed more emphasis on higher order thinking, saying, "[At the] base level you have to know the facts, but I'm far more interested in how they can [transfer] the fact."</p> <p>AST assumes that students can engage in cognitively rich activities by engaging in the practices of science for the purposes of sensemaking about the natural world. Value is placed on students constructing useful knowledge that is iteratively formed through the engagement in tasks, dialog, and feedback. While many teachers promoted parts of this vision, a majority also felt constrained by perspectives of people in the school community who placed significant value and emphasis on grades, which in turn disrupted formative feedback cycles in the classroom. Six teachers raised concerns about providing quality feedback due to school, parent, and student positioning that elevated grades and not evidence of sensemaking, as paramount. Nancy addressed this tension when discussing a recently assigned set of essays:</p> <p>Well, in this community and this district, the school, students are constantly checking their grades... I mentioned that paper that they had to write. I spent a lot of time grading their papers and writing all these comments about their strengths and weaknesses. I mean it took me a long time to grade these papers. But I had on the cover the sheet that had the rubric and their points and they just looked at that. They did not go through their paper to look at the comments and feedback I gave them. So, that was where I was like, "Well, that was a lot of work and they just really care about the grade."</p> <p>Four of the ten teachers voiced support for the importance of feedback, but downplayed its practical importance due to the community's culture related to accountability and grades, rather than sensemaking.</p> <p>Teacher interviews reported various assessment and data use strengths at the start of the project (Table 5). Four teachers identified the variety and frequency of their assessment as strengths—even before being introduced to the Dimensions. Abby cited her desire to assess students with more than one method by going "[beyond] a written test... grading the labs equally as the written test and [having students] do oral presentations and diagrams and models. Individual teachers identified each of the following Dimensions as assessment practice strengths: setting of explicit goals, the alignment of assessments with goals, the cognitive depth of assessments, and the use of data to inform instruction. The remaining three teachers identified dispositions unrelated to our set of Dimensions like, "the willingness to try something new". More commonality was found among teachers' perceived weaknesses and areas for improvement. As mentioned above, four teachers identified feedback as an area for growth. For example, Abby represents the four teachers who have tried to improve their feedback, but have run into obstacles. She stated, "I would like to be more systematic in my observations and feedback... I've tried that but I haven't always sustained that as a practice." Another three teachers identified a construct outside of our Dimensions by citing the need to improve their assessment differentiation as a means to meet the individual needs of particular students.</p> <p>5 TableSelf‐reported frequencies of teacher assessment strengths and weaknesses before and after the project as reported in project interviews</p> <p> <ephtml> &lt;table&gt;&lt;thead valign="bottom"&gt;&lt;tr valign="bottom"&gt;&lt;th /&gt;&lt;th align="left"&gt;Strength/most growth&lt;/th&gt;&lt;th align="left"&gt;Weakness/area for growth&lt;/th&gt;&lt;/tr&gt;&lt;tr valign="bottom"&gt;&lt;th align="left"&gt;Before&lt;/th&gt;&lt;th align="left"&gt;After&lt;/th&gt;&lt;th align="left"&gt;Before&lt;/th&gt;&lt;th align="left"&gt;After&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td align="left"&gt;Dimensions&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Set Goals&lt;/td&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Align Goals&lt;/td&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td /&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Frequency&lt;/td&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td /&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Variety&lt;/td&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Cognitive Complexity&lt;/td&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Science Practices&lt;/td&gt;&lt;td /&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Student Involvement&lt;/td&gt;&lt;td /&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;td /&gt;&lt;td align="left"&gt;3&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Feedback&lt;/td&gt;&lt;td /&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td align="left"&gt;4&lt;/td&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Adapt Instruction&lt;/td&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;td align="left"&gt;2&lt;/td&gt;&lt;td align="left"&gt;1&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Miscellaneous&lt;/td&gt;&lt;td align="left"&gt;3&lt;/td&gt;&lt;td /&gt;&lt;td align="left"&gt;6&lt;/td&gt;&lt;td /&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <p>5 <emph>Note</emph>: Frequencies represent unique teacher responses. Although teachers were asked about their greatest strength and weakness, multiple teachers responded by citing multiple aspects of practice. These are all included in the count.</p> <p>Teachers were prompted throughout the interview to talk about how they used evidence of student thinking and assessment information to inform instruction. Although several teachers acknowledged its importance, specifics about this practice were often absent. For example, Nancy stated, "Well [evidence of student thinking] should inform my teaching practice, right? That's the skill of being a good teacher. If they're not getting it then definitely that's my cue that I'm not explaining it or we need to redo it." But while an acknowledgment of the importance of data use was consistent across the sample, conceptions of data‐use practice indicated a lack of understanding of a data‐use theory of action. For instance, Abby stated that she wanted to do more with data and evidence of student thinking but that she lacked a conceptual framework for doing so. When providing an example of how she used data, she mentioned leveraging a computer program in which students took quizzes that provided her with data about which students missed different questions. However, the data itself did not allow her to act on the student level, because "the fun part about [the computer program] is they can...[pick] a nickname. The scores go to [me], but I can't tell [who thought what] because I let them use nicknames. I can't tell exactly who it is but it gives me an idea if [the class is] getting it." The computer program provided a high‐level summary of student achievement, but instruction could not be targeted to individual students who needed support. Her comments pointed to a need for PD on collecting good data that can be interpreted and then acted upon in future instruction to help specific students.</p> <p>Teachers faced obstacles at different points in the assessment process. For teachers like Wayne, time constraints were offered as the primary obstacle to quality assessment and data use. He discussed how the amount of required curriculum minimizes his ability to use evidence of student thinking to address difficult content in new ways. Three other teachers—Brent, Martha, and Danielle—echoed Wayne's focus on the limitations of time whereas Bob's perception of quality practice broke down at the data interpretation stage. He gathered a great deal of evidence of student thinking from the science notebook implemented in his class that served as a longitudinal portfolio of student thinking. Yet in reflecting about how he used the notebooks he was concerned more about "penmanship", whether [students] were "filling the charts out properly", and other elements distal to student sensemaking of science ideas. In contrast, Harriet acknowledges that using students' ideas revealed through assessment tasks "is something [she] struggles with." She looked at data—even to the level of individual questions and individual student thinking—but her pedagogical response was less effective as she most often cited the need to "reteach." Across the sample, teachers recognized the importance of using data, but whether due to time, interpretation, or application, their data use practices were not optimized.</p> <p>In addition, interview data revealed tensions between interim benchmark data and the more intimate classroom level data collected by teachers. The six teachers in the two districts that mandated the use of one or more interim benchmark assessments all valued classroom‐level, intimate assessment data far more than the standardized district‐level benchmark data. For example, Wayne described the benchmarks as "an end." These assessments "are used by the next level (the district), [t]hey're not used by me as much because they're just—it's an end product. It's not a window. It's a door." Teachers from both districts talked about how the data were not useful formatively for current classes. As Harriett stated, the benchmark data "allows me to, maybe over the summer or at the end of a unit, to look back at how I administered a particular unit and the next time through—the real reteach opportunity is more about me and how I'm gonna do it next year."</p> <p>For the four teachers from the second district, though, the value of using the benchmark data was overshadowed by confusion about whether the data would even be collected and who would look at the data. Martha, in her 19th year of teaching, but having moved to middle school science from the primary grades, questioned the purpose of the district benchmark assessment and how it would be used. She said that the prompt was covering material that was not in the standards and so she asked at the faculty meeting, "Why are we doing this?... What do we do with this information?... Are they going to collect it?" She recalled that another teacher replied to these questions, "I think they collect it. I don't remember, actually, if they collect it." So Martha asked herself, "Do they need the data? How do we score it?"; she ultimately complied by giving students the benchmark assessment, although at the time of the interview, she had not done anything with the data.</p> <hd id="AN0157236319-24">Evidence from practice</hd> <p>Analysis of baseline Notebooks on each Dimension provided a point of comparison between teachers' conceptions of assessment and data use and their actual practice. Baseline Notebook ratings indicated sporadic implementation of the nine Dimensions and generally low‐level proficiency of assessment and data use across the sample. Baseline Notebooks averaged a 2.10 on the 5.00 rubric scale for their holistic ratings, indicating only marginal quality of practice overall. In light of the 5‐point scale that rated Dimensions from Unrealized to Fully Realized, three different groupings of quality appeared. Two Dimensions approached or reached a level of moderately realizing quality practice (2.50 <ephtml> &lt;math altimg="urn:x-wiley:00368326:media:sce21712:sce21712-math-0001" xmlns="http://www.w3.org/1998/Math/MathML"&gt;&lt;semantics&gt;&lt;mrow&gt;&lt;mo&gt;&amp;#8804;&lt;/mo&gt;&lt;/mrow&gt;&lt;annotation encoding="application/x-tex"&gt; $\le $&lt;/annotation&gt;&lt;/semantics&gt;&lt;/math&gt; </ephtml> <emph>n</emph> &lt; 3.50), four Dimensions fell in the range of barely realizing quality practice (1.50 <ephtml> &lt;math altimg="urn:x-wiley:00368326:media:sce21712:sce21712-math-0002" xmlns="http://www.w3.org/1998/Math/MathML"&gt;&lt;semantics&gt;&lt;mrow&gt;&lt;mo&gt;&amp;#8804;&lt;/mo&gt;&lt;/mrow&gt;&lt;annotation encoding="application/x-tex"&gt; $\le $&lt;/annotation&gt;&lt;/semantics&gt;&lt;/math&gt; </ephtml> <emph>n</emph> &lt; 2.50) and three Dimensions were rated as not realizing quality practice (1 <ephtml> &lt;math altimg="urn:x-wiley:00368326:media:sce21712:sce21712-math-0003" xmlns="http://www.w3.org/1998/Math/MathML"&gt;&lt;semantics&gt;&lt;mrow&gt;&lt;mo&gt;&amp;#8804;&lt;/mo&gt;&lt;/mrow&gt;&lt;annotation encoding="application/x-tex"&gt; $\le $&lt;/annotation&gt;&lt;/semantics&gt;&lt;/math&gt; </ephtml> <emph>n</emph> &lt; 1.50) (Table 6). Assessment Variety (<emph>M</emph><subs>baseline</subs> = 2.71) and Frequency (<emph>M</emph><subs>baseline</subs> = 3.00) scored the highest, albeit only at the level of moderately realizing high quality practice. These ratings reflect what teachers identified as their strengths in the interviews. Teachers used multiple types of assessments—both formative and summative—as well as a variety of question types on their different assessments to present a variety of opportunities for students to show their understanding.</p> <p>6 TableDimension averages across all participants for baseline and final Notebooks</p> <p> <ephtml> &lt;table&gt;&lt;thead valign="bottom"&gt;&lt;tr valign="bottom"&gt;&lt;th /&gt;&lt;th align="left"&gt;Dimension averages&lt;/th&gt;&lt;/tr&gt;&lt;tr valign="bottom"&gt;&lt;th align="left"&gt;D1&lt;/th&gt;&lt;th align="left"&gt;D2&lt;/th&gt;&lt;th align="left"&gt;D3&lt;/th&gt;&lt;th align="left"&gt;D4&lt;/th&gt;&lt;th align="left"&gt;D5&lt;/th&gt;&lt;th align="left"&gt;D6&lt;/th&gt;&lt;th align="left"&gt;D7&lt;/th&gt;&lt;th align="left"&gt;D8&lt;/th&gt;&lt;th align="left"&gt;D9&lt;/th&gt;&lt;th align="left"&gt;D10&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td align="left"&gt;Baseline Notebook&lt;/td&gt;&lt;td align="char" char="."&gt;2.43&lt;/td&gt;&lt;td align="char" char="."&gt;2.33&lt;/td&gt;&lt;td align="char" char="."&gt;3.00&lt;/td&gt;&lt;td align="char" char="."&gt;2.71&lt;/td&gt;&lt;td align="char" char="."&gt;2.48&lt;/td&gt;&lt;td align="char" char="."&gt;2.43&lt;/td&gt;&lt;td align="char" char="."&gt;1.43&lt;/td&gt;&lt;td align="char" char="."&gt;1.48&lt;/td&gt;&lt;td align="char" char="."&gt;1.57&lt;/td&gt;&lt;td align="char" char="."&gt;2.10&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final Notebook&lt;/td&gt;&lt;td align="char" char="."&gt;2.55&lt;/td&gt;&lt;td align="char" char="."&gt;2.65&lt;/td&gt;&lt;td align="char" char="."&gt;4.00&lt;/td&gt;&lt;td align="char" char="."&gt;3.25&lt;/td&gt;&lt;td align="char" char="."&gt;2.75&lt;/td&gt;&lt;td align="char" char="."&gt;2.30&lt;/td&gt;&lt;td align="char" char="."&gt;1.15&lt;/td&gt;&lt;td align="char" char="."&gt;1.75&lt;/td&gt;&lt;td align="char" char="."&gt;1.65&lt;/td&gt;&lt;td align="char" char="."&gt;2.55&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Net Change&lt;/td&gt;&lt;td align="char" char="."&gt;0.12&lt;/td&gt;&lt;td align="char" char="."&gt;0.32&lt;/td&gt;&lt;td align="char" char="."&gt;1.00&lt;/td&gt;&lt;td align="char" char="."&gt;0.54&lt;/td&gt;&lt;td align="char" char="."&gt;0.27&lt;/td&gt;&lt;td align="char" char="."&gt;&amp;#8722;0.13&lt;/td&gt;&lt;td align="char" char="."&gt;&amp;#8722;0.28&lt;/td&gt;&lt;td align="char" char="."&gt;0.27&lt;/td&gt;&lt;td align="char" char="."&gt;0.08&lt;/td&gt;&lt;td align="char" char="."&gt;0.45&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <p>6 <emph>Note</emph>: Scale values for D1–D9: 5 = fully present or realized, 4 = sufficiently...; 3 = moderately...; 2 = barely...; 1 = not... Scale for D10 (overall): 5 = advanced practice; 4 = proficient...; 3 = average...; 2 = marginal...; 1 = inadequate...</p> <p>Of lower quality were two pairs of related Dimensions. Setting learning goals and aligning assessments to these goals (Dimensions 1 and 2) were present in four and absent in six Notebooks. Similarly, the cognitive depth (Dimension 5) and the engagement of students in scientific practices (Dimension 6) were present minimally in eight Notebooks, although two Notebooks scored quite high on these two Dimensions, raising the means for both constructs (see Table 7 for disaggregated ratings). Although interviews generally did not acknowledge the importance of Dimensions 1 and 2, they did reveal high value for the cognitive depth of assessments. Curiously, four teachers recognized the importance of engaging students in scientific practices, but this did not translate into their assessments. For example, Erica emphasized in her interview the need to push on critical thinking and rigor as well as help develop students' argumentation skills and their ability to use claims, evidence, and reasoning. Yet her scores in practice on Dimensions 5 and 6 were 2.50 and 1.50, respectively.</p> <p>7 TableDimension averages for all participants across baseline and final Notebooks</p> <p> <ephtml> &lt;table&gt;&lt;thead valign="bottom"&gt;&lt;tr valign="bottom"&gt;&lt;th align="left"&gt;Teacher&lt;/th&gt;&lt;th align="left"&gt;Dimension averages&lt;/th&gt;&lt;/tr&gt;&lt;tr valign="bottom"&gt;&lt;th align="left"&gt;D1&lt;/th&gt;&lt;th align="left"&gt;D2&lt;/th&gt;&lt;th align="left"&gt;D3&lt;/th&gt;&lt;th align="left"&gt;D4&lt;/th&gt;&lt;th align="left"&gt;D5&lt;/th&gt;&lt;th align="left"&gt;D6&lt;/th&gt;&lt;th align="left"&gt;D7&lt;/th&gt;&lt;th align="left"&gt;D8&lt;/th&gt;&lt;th align="left"&gt;D9&lt;/th&gt;&lt;th align="left"&gt;D10&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td align="left"&gt;Abby&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;5.00&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Nancy&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;4.50&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;4.50&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;5.00&lt;/td&gt;&lt;td align="left"&gt;5.00&lt;/td&gt;&lt;td align="left"&gt;4.50&lt;/td&gt;&lt;td align="left"&gt;5.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Erica&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;5.00&lt;/td&gt;&lt;td align="left"&gt;4.50&lt;/td&gt;&lt;td align="left"&gt;4.50&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Wayne&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;2.33&lt;/td&gt;&lt;td align="left"&gt;2.33&lt;/td&gt;&lt;td align="left"&gt;1.33&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.33&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Wendy&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;5.00&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Danielle&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Martha&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;4.50&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;4.50&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Brent&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;4.50&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;4.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Harriet&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;3.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;3.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Bob&lt;/td&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Base&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;Final&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;2.50&lt;/td&gt;&lt;td align="left"&gt;2.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;td align="left"&gt;1.00&lt;/td&gt;&lt;td align="left"&gt;1.50&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <p>7 <emph>Note</emph>: Teachers within the same school are separated by each section. Scale values for D1–D9: 5 = fully present or realized, 4 = sufficiently...; 3 = moderately...; 2 = barely...; 1 = not... Scale for D10 (overall): 5 = advanced practice; 4 = proficient...; 3 = average...; 2 = marginal...; 1 = inadequate...</p> <p>Finally, three Dimensions—involving students in their own assessments (Dimension 7), providing quality feedback (Dimension 8), and using student thinking to adapt instruction (Dimension 9), all scored, on average, at or below a 1.57, with the lowest possible mean being a 1.00. Three teachers talked about the importance of peer and self‐assessment in their interviews, one stating that the Dimension is "underrated" in classroom practice. Yet four teachers scored a 1.00 (meaning that no evidence was included for this Dimension) and four teachers scored a 1.50 on this Dimension (<emph>M</emph><subs>baseline</subs> = 1.43).</p> <p>Given interview data in which teachers described a culture of grades rather than the intrinsic value of feedback, it may be unsurprising that teachers also scored poorly on providing feedback (Dimension 8, <emph>M</emph><subs>baseline</subs> = 1.48). In many cases, feedback on the artifacts existed as a set of symbols—checks or X's indicating right or wrong, accompanied by a numerical grade at the top of the paper (Figure 2a). Evidence showed minimal instances of feedback that moved even slightly beyond marking. Figure 2c shows only slight improvement by pointing out the construct in question that presumably would be discussed with the teacher.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/SED/01jul22/sce21712-fig-0002.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="sce21712-fig-0002.jpg" title="2 Example artifacts that reflect less and more fully realized instantiations of the Dimensions focused on providing quality feedback and using evidence of student thinking to adapt future instruction. (a) Unrealized quality feedback with questions marked right or wrong and just a total score (+28). (b) Feedback that goes beyond marking and grades. The arrows point to a part that the teacher has circled and notes that they need to discuss this value with the student. This represents low‐level feedback, but higher than just marking. (c) The teacher does not articulate specific ways in which students are not meeting the learning goal and provides nonspecific next steps with the statement, &quot;more practice.&quot; (d) Reflections on helping students improve the details on their diagrams by giving more specific directions and models of quality diagrams" /> </p> <p></p> <p>More surprising, however, is that teachers generally acknowledged the importance of using evidence of student thinking to inform instruction, but Notebooks often lacked evidence of attempts at Dimension 9 in practice (<emph>M</emph><subs>baseline</subs> = 1.57), even though the artifact annotations explicitly prompted teachers to reflect on and provide evidence of this Dimension. In many cases, when prompted to discuss how they used evidence of student thinking to inform future instruction, teachers often cited the need to "reteach" or "do more problems" (Figure 2b) rather than provide salient, targeted ways in which current or future students could engage differently with the material (Figure 2d shows a moderately realized example). Taken together, teachers' positioning toward the importance of using evidence to inform future instruction was tenuous. Interviews generally suggested that teachers valued this part of practice in principle, but representations evidence from the Notebook did not, on average, reflect high‐quality execution of this Dimension.</p> <hd id="AN0157236319-26">Mini‐cases of initial assessment practice</hd> <p>While only one Dimension received an average rating at or above 3.00, instances of proficient and advanced levels of practice were found in particular Dimensions in participants' Notebooks (Table 7). Two short cases provide a qualitative comparison of Wendy and Nancy's Notebooks that reflect holistic lower and higher practice, respectively, while also illustrating the variability within individual teachers' practice.</p> <p>Wendy's scores are reflective of teachers who scored well on the "Where do you want to go?" Dimensions, but poorly on Dimensions related to how students can move from their current thinking toward greater achievement of the learning goal (Dimensions 8 and 9). Wendy showed evidence of explicitly stating learning goals (<emph>M</emph><subs>baseline</subs> = 3.50) and aligning assessments to those learning goals (<emph>M</emph><subs>baseline</subs> = 3.00). Her unit plan had a table that aligned the unit and lesson plan objectives with state standards and the related assessments. For the benefit of students, the opening slide for each day had the lesson objective explicitly stated at the top, presenting students with their purpose for the day. On average, the assessments within the units did measure the learning goals. She also showed well‐realized levels of assessment Variety (<emph>M</emph><subs>baseline</subs> = 3.00) and Frequency (<emph>M</emph><subs>baseline</subs> = 3.50). Students were provided opportunities to annotate scientific drawings, show understanding with quick formative assessments like thumbs‐up and thumbs down, and answer questions that were both restricted response (e.g., multiple‐choice and matching questions) and open‐ended analysis tasks. Formative assessment occurred for each multi‐day lesson, generally after every three major activities provided by her kit‐based curriculum. Graded quizzes and summative assessments were far less frequent.</p> <p>Despite attention to setting learning goals and providing students with varied and frequent opportunities to show evidence of their thinking, Wendy did little to provide Feedback to students (<emph>M</emph><subs>baseline</subs> = 1.00) or to use the information to Adapt Instruction (<emph>M</emph><subs>baseline</subs> = 1.00). On three occasions, her artifact annotations mentioned that students received verbal feedback, but no written feedback was provided on any student samples. In terms of using the evidence of student thinking, the annotations in Wendy's Notebook showed effective analysis of student thinking. However, she never identified specific ways in which she could adapt her instruction going forward in relation to their thinking.</p> <p>In contrast to Wendy, Nancy's strengths in practice were not in the setting of goals or the alignment with goals. The artifacts and the annotations did not explicitly Set Goals (<emph>M</emph><subs>baseline</subs> = 1.00), and Aligning Assessments was difficult to rate due to the lack of clear learning goals (<emph>M</emph><subs>baseline</subs> = 1.50). Like Wendy, her artifacts were rated highly for their Variety and Frequency of assessment (3.50 and 4.50, respectively)—the two Dimensions that appeared to be the most addressed initially by teachers in our sample. Students in Nancy's class were able to show their learning in multiple ways whether it be via extensive written tasks that were open‐ended in nature or through group‐level tasks. Students also had the opportunity to write creatively about the water cycle and roleplay what happens to water molecules throughout the process. Some form of assessment was found in each of the 10 days, with one summative assessment occurring at the end of the ten days.</p> <p>In contrast to Wendy, Nancy pushed her students to engage in tasks with great Cognitive Complexity (<emph>M</emph><subs>baseline</subs> = 4.50) through the engagement of Scientific Practices (<emph>M</emph><subs>baseline</subs> = 4.00) on the assessments. Student activity and assessment progressed from early comprehension tasks toward higher‐order tasks such as analyzing maps. Students also were asked to analyze given sets of data and design their own experiments. These tasks were paired with groups of analysis questions that were submitted for review. Ultimately, the summative task required students to synthesize the evidence to recommend a plan of action for addressing water contamination.</p> <p>In terms of Feedback and Adapting Instruction, Nancy's Notebook provided the highest rating among the baseline Notebooks, although the feedback provided was still minimal. Her mean rating of 3.50 for Dimension 9, the use of evidence of student thinking to inform instruction, was a full 1.50 points higher than any other teacher's score. This rating was achieved by consistently attending to evidence from student thinking. On 6 of the 10 days, Nancy recognized some opportunities to address learning for both current and future students. This ranged from unsophisticated adjustments like providing clearer instructions or providing students more time on a task to a more robust set of adjustments like showing exemplars of high‐level work that can serve as models for student thinking.</p> <hd id="AN0157236319-27">Summary: Patterns of initial assessment and data use</hd> <p>Several common themes appeared in the baseline data. First, teachers had fluency with and could talk about a variety of assessment types, especially at the meta‐level of formative and summative assessment; benchmark assessments were discussed, but with more uncertainty. Unlike some of the intimate data accrued through formative assessment, teachers generally questioned the use of benchmark assessments because these assessments lacked clear purpose and the teachers did not receive useful data or, at times, any data at all. Many teachers also discussed the need for rigorous assessments and mentioned their need to supplement existing curricula with more cognitively demanding assessment tasks. Finally, evidence from initial artifacts of teachers' baseline practice mirrored many of their conceptions, with Notebooks rated highly in terms of their variety and the frequency of assessment, but low in terms of their goal setting and feedback and data use.</p> <hd id="AN0157236319-28">Postintervention: How PD influenced assessment and data‐use practice</hd> <p>Comparing data from multiple sources at the end of the project showed that teachers made nuanced changes to their conceptions and practice. As detailed below, conceptions and beliefs were most likely to align with the goals of PD experiences. On average, practice as evidenced by final Notebook ratings showed improvement, but improvement typically was seen for the Dimensions of greatest focus in the PD (Feedback and Adapting Instruction) <emph>only</emph> if other Dimensions were already showing evidence of at least adequate practice.</p> <hd id="AN0157236319-29">Evidence from the interviews</hd> <p>Interviews suggested that PD structured on the Notebook and Dimensions provided a differentiated path for professional growth for participants. When asked in interviews to identify the features of assessment practice in which they grew the most during the project, participants mentioned eight different Dimensions (Table 5). Align Assessments, Frequency, Variety, Scientific Practices, and Feedback were each mentioned by at least two different participants.[<reflink idref="bib1" id="ref10">1</reflink>] Similarly, in identifying the Dimension in most need of growth, teachers mentioned seven different Dimensions, with three teachers mentioning Student Involvement and two teachers each identifying Cognitive Complexity and Feedback (Table 5). Interestingly, in the same interview, teachers were asked to identify artifacts in their Notebook that showed evidence of growth. Five teachers pointed to the Dimension that was not mentioned previously in the interview—Adapting Instruction. This Dimension was central to each of the PD experiences, and several teachers spoke in detail about how their practice was shaped positively by using the Notebooks. For instance, Wendy talked about one quiz that involved interpreting multiple graphs. She noted that in the past she would have "just moved on or [gone] back and red[id] it a little bit and then moved on." However, in light of this PD experience, she reflected more on the need to address students' struggles that were presented in the quiz data. After analyzing the data, she did some follow up discussion and instruction. Then she put them in groups and they completed a paper quiz together and had to try and agree on the answers. If they disagreed, they had to initial the question to note their disagreement. She then gave written feedback to each of the groups and the groups had to engage in discussion and address the feedback.</p> <p>Martha also shifted her perception of the Adapting Instruction Dimension. Previous to the experience she viewed adapting instruction "as some sort of problem instead of a normal part of the process" leaving her to see this element of practice as "a negative thing." She used to view having to change her instruction as "I've messed something up," but now she views it as "normal...almost like the editing process." Instruction is "just as much a work in progress as the human brain."</p> <p>Despite teachers identifying different areas of strength and growth after the intervention, one theme emerged from the end‐of‐project interviews—the participants overwhelmingly acknowledged the superior value of intimate data stemming from formative assessment as compared to more distal summative or benchmark data. Five teachers described the quick feedback cycles that can occur with formative assessments like exit cards or brief, in‐person conversations. When asked what was most useful in her assessment practice, Martha stated, "The formative, most definitely. [Because] those are during the instruction and so it becomes obvious immediately if something needs to be retaught or restructured or if they are getting it or if we can move on." Similarly, Abby talked about the biggest shift in her practice during the study as attending to the formative data that her students produced. She talked about "shifting a lot more of [her] attention to the formative assessment" and "tweaking things so that [she] could better understand what [students] know." By focusing more on the formative, in‐the‐moment student data produced by brief student conversations, she was able to "look more closely" at the questions that were asked, at student responses, and at what might need to be done to adjust instruction.</p> <p>Six participants vocalized their skepticism of the usefulness of more distal data such as state tests or district benchmarks. Harriet talked about how administrators and teachers used math and ELA standardized test data that "separate[d] out the students based off ability level using [standardized test] results or those scantron results." Similarly, Wendy talked about the limited usefulness of state test data "since the tests are not until the end of the year." In the subsequent year, teachers who do not know the students are suddenly presented with data from standardized tests about performance, basically limiting the use of data to placing students in different tracks. Wayne knows he is "supposed to say [that] standardized tests [provide the best information], but [he] doesn't look at them because they don't really judge the whole person." Instead, he prefers the more intimate data found in their class notebooks over the multiple choice or other restricted response items, especially since the majority of standardized data captures ELA and math, not science achievement.</p> <p>For Erica, the lack of utility for non‐intimate data is exacerbated by the amount of time spent testing. "It's been 30 days of assessment this year. It is the most daunting, disturbing part of my career." She notes that the district level benchmarks are not even aligned with what the teachers teach and that while the teachers have to give them four times a year, the district no longer scores them, making them essentially useless.</p> <p>The strong affinity for formative assessment data at the expense of using summative or benchmark data conflicted, slightly, with some PD activities and resources focused on summative data. One section of the PD engaged teachers in discussion focused on the formative nature of summative assessments—the ability to use these end‐of‐unit assessments as a means to inform instruction for current students in subsequent units. For example, if students struggled analyzing and interpreting graphs, teachers could adapt their instruction in the next unit given this information. Yet, six of the teachers referenced only using summative data to inform instruction for <emph>next</emph> year's students. For some teachers, like Harriet, the reasons were structural. Her curriculum is integrated and in any given year her units might jump from physical science to life science to earth science. Therefore, when the summative assessment was completed, she would tell students, "When we're done with a unit, we're done; we're moving on." Since she might have one unit on physics followed by a unit on cell biology and disease, "It doesn't allow a lot of time to have overlap or even go back. So [she] spends more time thinking about what next year's gonna be like." Nancy also pointed to structures within her school as a means for shaping her perspective on summative assessment data saying, "Usually we're ready to move on to another topic and it's just really hard... if I want to go back and try to reteach, I would probably never get anywhere... you do have to move on, at least the way school is structured."</p> <p>For five other teachers, the issue with summative assessments was not structural, but philosophical. The summative assessment was still seen as a means for capturing final student achievement whereas formative assessment was seen as informing practice. Wayne provided the analogy of a gas gauge, "Formative assessment can give me an in‐progress snapshot of the things we need to handle while we're going. It's almost like a gas gauge in a car. I know that once the gas runs out, I'm gonna—the car's gonna stop, but if I monitor the gauge, I can, knowing where I need to go, gauge how much I need to spend on gas fill ups in between." For Wayne, the summative assessment does not seem to be a timely gauge of student understanding or things he "needs to handle" while teaching a unit. Abby, Nancy, Wendy, and Erica all mentioned similar positions that formative assessment data is of significant use, but summative assessment data is only useful for next year because "the summative is really for me, we're done, we've done everything we're going to do at this point, I just need to know how much you know" (Abby).</p> <p>Finally, the interview asked broadly about participants' experiences with the project and if it was helpful for their professional growth. Being interviewed by part of the research team likely created an opportunity for social desirability bias and it may be unsurprising that all participants indicated a positive, effective experience with the project. Of interest, however, is the variety of different tools and experiences that participants valued. Two teachers identified the Dimensions as central to their learning. For Wayne, the Dimensions made explicit the things he already did with assessment, but allowed him to "name it and claim it." Nancy really benefited from the sticky note annotations that provided a space for systematic reflection across her artifacts. Bob explained that the portfolio allowed for a comprehensive vision of his assessment practice over a bound period of time and that he would continue to collect artifacts in the coming year in the Notebook. And while Martha noted that the Notebook was "not [her] style" for reflection, she acknowledged the PD on the Dimensions as central to her growth. Overall, their responses suggested that tools such as the Notebook, the sticky notes, and the Dimensions were central to professional growth and that a systematic focus on assessment is novel for teachers' practice and a welcomed approach to improving student outcomes.</p> <hd id="AN0157236319-30">Evidence from practice</hd> <p>The holistic "Overall" rating of baseline and final Notebooks showed that teachers attended to more high‐quality representations of the Dimensions after the intervention, with an average change of nearly a half point, <emph>M</emph><subs>baseline</subs> = 2.10; <emph>M</emph><subs>final</subs> = 2.55 (Table 6). Thus, practice as measured by the Overall Rating, moved from marginal toward adequate assessment and data use on our scale. Although major shifts in practice were not reflected in the final Notebook rating, the holistic average and the Dimension averages mask changes in performance over time. Growth was not consistent across Dimensions or across participants (Tables 5 and 6).</p> <p>Ratings of individual Dimensions in the collection of final Notebooks can be organized into three groups based on their relationship to the median rating score of 3. Align Assessments (<emph>M</emph><subs>final</subs> = 2.65), Frequency (<emph>M</emph><subs>final</subs> = 4.00), Variety (<emph>M</emph><subs>final</subs> = 3.25), and Cognitive Complexity (<emph>M</emph><subs>final</subs> = 2.75) all approached or surpassed the median value of 3.00, with the frequency of assessment reaching, on average, proficient levels. Set Goals (<emph>M</emph><subs>final</subs> = 2.55) and Scientific Practices (<emph>M</emph><subs>final</subs> = 2.30) represented less effective practice that was, on average, deemed closer to "marginal" than adequate or proficient practice. Finally, similar to the scores on the baseline Notebooks, Student Involvement (<emph>M</emph><subs>final</subs> = 1.15), Feedback (<emph>M</emph><subs>final</subs> = 1.75), and Adapting Instruction (<emph>M</emph><subs>final</subs> = 1.65) represented the lowest grouping that failed to reach even somewhat realized levels of practice.</p> <p>Interestingly, the largest positive changes to practice occurred not with the previously lowest scored Dimensions—the Dimensions that would have the most room for growth—but rather with the Dimensions that were rated <emph>highest</emph> at baseline. Ratings for Frequency and Variety increased by 1.00 and 0.54, respectively. More modest, but noticeable changes ranging from 0.25 to 0.50 were observed for Align Assessments, Cognitive Complexity, and Feedback. Basically, no change was observed for Set Goals, Scientific Practices, and the Dimension of most focus in the PD, Adapting Instruction. Furthermore, <emph>negative</emph> change was seen for Student Involvement with over a quarter point decrease to 1.15, suggesting the near total absence of any evidence of teachers engaging their students in thinking about their own assessment during the collected units. The most common Dimension for negative change was Scientific Practices. Five participants showed declines on this Dimension. However, two participants showed sizeable growth, thus resulting in a more bimodal distribution of change than would be interpreted by the minimal change in average from baseline to final Notebooks.</p> <p>Perhaps the most interesting pattern is the relationship between change in the ratings for Dimensions that are grouped under the questions, "Where are you going?," "Where are you now?," and "How will you get there?." We noticed a pattern indicating that certain Dimensions may function as pre‐requisites for improvement of the more difficult Dimensions. Although the "How will you get there?" Dimensions, like Feedback and Adapting Instruction, were central tenets of the PD and were discussed significantly in the interviews and represented in drawn mental models, the change in Notebooks on these Dimensions was minimal. Although five participants' Notebooks showed small positive growth on Feedback and four showed small growth on Adapt Instruction, we noticed that this growth only occurred if other Dimensions from the "Where are you going?" and "Where are you now?" domains had reached an adequate or proficient level of practice. Teachers who at the baseline initially scored high on Frequency and Variety generally showed growth on these same two Dimensions in the final Notebook, and they were also more likely to show some growth on the Set Goals or Align Assessment Dimensions. However, the Dimensions most related to closing the feedback loop and adjusting instruction still remained at inadequate or marginal levels of practice. The only teachers who started showing signs of growth on the "How will you get there?" Dimensions (Feedback and Adapt Instruction) had their artifacts rated at a level of proficiency on the Set Goals, Align Assessments, Frequency, and Variety Dimensions in the baseline Notebook.</p> <hd id="AN0157236319-31">Mini‐cases of postintervention assessment practice</hd> <p>The developmental progression in which growth on the "How will you get there" Dimensions generally occurred only when preceded by adequate levels of practice on the "Where are you going?" and some "Where are you now?" Dimensions, is reflected in Martha's and Wendy's Notebooks. Martha's baseline Notebook was scored, overall, as "inadequate" assessment practice with the absence of any evidence for involving students in their own assessment (Dimension 7), quality Feedback (Dimension 8), or the use of evidence of student thinking for Adapting Instructional decisions (Dimension 9). Even the Frequency (Dimension 3) and Variety (Dimension 4) of assessments were scored as barely realized practice. Her final Notebook, however, exhibited some of the greatest growth across Dimensions 1–6. Set Goals improved by 1.50 points while the next five Dimensions all increased by 2.50 points—over half of the entire scale. For example, whereas assessment goals were not made explicit nor were many assessments aligned to a lesson's goals in the baseline Notebook, in the final Notebook goals appeared explicitly on many assessments for students to see, and Aligning Assessments with the goal was supported by a Variety of question types that pushed students to identify evidence for claims and generate explanations.</p> <p>Despite Martha's increased ratings on many Dimensions, no change was seen for any of the "How will you get there?" Dimensions—Dimensions that had the most room for growth quantitatively. Student Involvement in assessment was absent, Feedback that went beyond marking right/wrong or grades was not included, and evidence of student thinking was not used to Adapt Instruction based on these assessments. Interestingly, Martha discussed Adapting Instruction at length in her interview and represented this part of assessment practice as central in her drawn mental model, but no change was seen in practice.</p> <p>Wendy's pattern of change is different from Martha's, starting with different baseline scores on the "Where are you going?" Dimensions as well as Frequency and Variety. Unlike Martha, Wendy's baseline Notebook was rated moderately to sufficiently realized for Set Goals, Align Assessments, Frequency, and Variety Dimensions. Each of these ratings reached a 3 or higher whereas Martha did not exceed a 2 (barely present or realized) on any of these Dimensions. Similar to Martha, as Wendy's mini‐case showed, she had no evidence in the baseline Notebook of quality Feedback or Adapting Instruction.</p> <p>On the final Notebook, Wendy's growth was more significant than Martha's growth and in specific ways. Each of the first five Dimensions was scored at a 3 or higher, with Frequency even reaching a fully realized 5‐pt. rating. Unlike nearly all the other teachers, although many spoke of using evidence to adapt their instruction, Wendy's Notebook was one of the few in which the "How will you get there?" Dimension ratings increased. Feedback increased by 1.50 points and Adapting Instruction increased by 1.0 points. This was seen with increased amounts of written feedback to students that provided specific ways in which their work could improve and three instances in which instructional decisions were made based on student data. Notedly, the changes were modest in these two Dimensions, but Wendy's holistic final Notebook score of 3.50 approached proficiently rated practice as she started to attend to the "How do you get there?" Dimensions. Whereas Martha improved on many of the lower numbered Dimensions, but not on the higher numbered Dimensions, Wendy had initial levels of proficiency in Setting Goals and Aligning Assessments to these goals, perhaps better supporting her growth in practice with the Dimensions that proved to be most difficult for the majority of participating teachers.</p> <hd id="AN0157236319-32">Summary: Changes to conceptions and practice following intervention</hd> <p>Following the year‐long PD, teachers' conceptions and practice shifted, but in nonuniform ways. In self‐identifying areas of strength and areas in need of further growth, a diversity of views suggested that the Dimensions functioned as a differentiated tool for learning. In addition, teachers talked much more about their use of evidence of student thinking to inform their instruction, and they strongly underscored the importance and value of formative, intimate data over other forms of assessment data.</p> <p>While teachers' conceptions of effective assessment practice were influenced by project participation, artifacts from the final Notebook give a more varied perspective of impact on practice. Overall, ratings of Notebooks increased from baseline to final Notebook by nearly one‐half point on a 5‐point scale. Rating increases were most prominent for Dimensions on which participants scored most highly on the baseline Notebook, while the Dimensions that changed most in evidence of teachers' conceptual commitments—Feedback and Adapting Instruction—showed little change in practice and overall low scores. Exceptions occurred, though, for participants who originally scored relatively higher on Variety, Frequency, Setting Goals, and Aligning Goals in their Notebooks. In these cases, evidence of changes to feedback and data‐use practice were generally observed.</p> <hd id="AN0157236319-33">DISCUSSION</hd> <p>This study investigated middle school science teachers' conceptions of and practices in assessment and data use from two different states and four schools. Although we did not see extensive changes to practice after our intervention, we discuss several important themes drawn from our results that have implications for practitioners and inform the field about future areas for research. They include: (<reflink idref="bib1" id="ref11">1</reflink>) Disaggregating aspects of assessment and data‐use practice reveals disproportionate needs to improve student involvement in their own assessment, the quality of feedback provided to students, and guidance for using evidence to make actionable changes to instructional practice; (<reflink idref="bib2" id="ref12">2</reflink>) Preferences for increased use of intimate data require learning communities within schools to confront the often differing data logics among teachers and administrators so greater change can occur for the use of this preferred proximal data; and (<reflink idref="bib3" id="ref13">3</reflink>) PD on assessment and data use may significantly benefit from differentiated approaches that reflect what may be a learning progression for teachers' assessment practice.</p> <hd id="AN0157236319-34">Targeting specific elements of assessment practice for professional learning</hd> <p>At the start of the study, interviews and drawn mental models (Wilsey et al., 2020) indicated that teachers were conversant in a variety of different assessment types and assessment question formats and they stressed the importance of cognitively rich tasks, even if this played out only marginally in practice (see Table 6). These Dimensions generally focus on the production of tasks and assessments. Conversely, our analysis illuminated the need for growth particularly in areas requiring metacognitive reflection on assessment: student involvement in their own assessment, providing high‐quality feedback on assessments, and using evidence of student thinking to adapt future instruction. In some cases, teachers' mandated curriculum or the school culture around grades impeded their capacity to engage in what our framework deemed as high‐quality feedback. In other instances, participants lacked the self‐identified knowledge or desire to engage students in their own assessments. Unfortunately, the PD in this project most greatly shifted practice on those Dimensions that originally scored on the higher end of the continuum, while those that were relatively absent initially, only showed sporadic and minimal positive change.</p> <p>Perhaps these findings were unsurprising as literature on teacher PD presents mixed results – for instance, Garet et al., 2010, 2008) point to the minimal effects of PD on teacher and student outcomes in elementary English language arts and mathematics. In both cases, there were either nonstatistical differences, or the changes were not sustained. But instances of effective PD in science classrooms do exist. Heller et al. (2012), Penuel et al. (2009), Johnson et al. (2016), and Osborne et al. (2019) provide evidence of the positive effects of PD on science teacher practice and self‐confidence. Our study reflects this range of findings as growth appeared in particular areas and with particular teachers, but not comprehensively across the project. For example, participating teachers' drawn mental models (Wilsey et al., 2020)—important for influencing their assessment and data‐use practice (Jimerson, 2014)—indicated a shift after the Summer PD Institute toward more iterative approaches to feedback and the use of evidence to inform instruction. But Dimension ratings from artifacts of their practice suggest that most teachers still had significant room for growth, particularly in the Dimensions that showed initial change in their drawn models and interviews—those related to providing feedback and using evidence to inform instruction. These findings and those discussed above contribute to the conversation on PD design. If particular conceptions and mental models are necessary, but not sufficient for changes in practice, PD may be more effective when it consistently places teachers in positions to analyze the differences between their conceptions and their actual practice in an on‐going manner. Teachers in our PD reflected on their shifting drawn models, but they never compared those shifting models directly with evidence from their Notebooks of actual classroom practice; that analysis happened separately.</p> <p>Furthermore, addressing nine Dimensions, even over the course of a year, may be too much of a cognitive load for effective professional growth. The three Dimensions on which teachers were rated the lowest (Student Involvement, Feedback, Adapt Instruction) might need more targeted focus amongst learning communities and more tools to help support changes in practice. As described below, other Dimensions might need to reach levels of proficiency before beginning work on these difficult aspects of assessment. In addition, as elaborated below, PD must recognize and address existing contextual barriers related to these Dimensions that could prevent growth regardless of the quality and length of the professional learning experience for teachers.</p> <hd id="AN0157236319-35">Intimate data preferences and use</hd> <p>The focus on and value of students' ideas about science are central tenets of AST. Through social discourse, engagement in scientific practices, and interactive direct instruction, ambitious science classrooms help students make sense of the major accounts of the natural world (Windschitl et al., 2018). For this to occur, science teachers must have windows into students' ideas; all of these windows are formative assessment opportunities by which in‐the‐moment questions can be posed, the next day's or week's instruction can be modified, and changes to curriculum can occur for future students. As highlighted by the interview data, teachers in this study greatly valued intimate data collected from students. These types of data allowed them to have conversations about student thinking that was in front of them rather than just talking about improving test scores (Braaten et al., 2017).</p> <p>However, even as interviews elicited enthusiasm for these intimate data, the omnipresence of accountability and the type of data use associated with more distal measures created tensions in their practice. Like evidence from Braaten et al. (2017) case studies of middle school science teachers, some participants in this study questioned whether their contributions to student learning in science were largely viewed as a means for improving math and ELA scores. These views may be closely linked to years of district and state investment in more distal evidence of student learning, including state standardized tests or district benchmark assessments (see Kerr et al., 2006). As voiced by the teachers in these studies, the purpose of these more distal measures is sometimes in question as data systems do not provide the necessary, immediate feedback that can help teachers inform their instruction.</p> <p>The tension between intimate data valued by teachers and more distal data devalued by teachers in this project raises questions about what supports teachers need to increase opportunities for making timely decisions about how to adapt instruction. Various data systems and data visualizations have been introduced to school communities, but in many cases, these tools have done little to promote the capacity of teachers to improve the learning environment (McDonald et al., 2018). The lack of significant change in our outcome measures captured in the Notebook indicates that other supports are necessary to better effect change.</p> <p>We envisioned the use of tools and our PD encounters as professionalization through developing shared knowledge and skill among practitioners (Horn et al., 2015). This approach contrasts with one of competition and accountability. Our work focused on the conceptual and practical tools (Grossman et al., 1999) in the day‐to‐day provenance of teachers. This study, of course, is complex. Shavelson et al. (2008) intervention that trained physical science teachers to use embedded formative assessment data in their physical science classes showed generally no differences in outcomes for students when compared experimentally with students in classes in which formative assessment was not embedded. They attributed the null results at least partly to a lack of fidelity by the teachers; the participants did not use the data in ways that targeted new instructional approaches and therefore, struggled to remain faithful to its use. Future research must address how we can best support teachers' formative use of intimate data to move forward the goals of science education. More work must be done to investigate how tools like the Notebook and Dimensions can be used to shift teachers' conceptions <emph>and</emph> their practice. Particularly, if teachers are equipped with tools for easily collecting and reflecting on a variety of data, how might PD help them use this data to concretely inform their future instructional decisions. This study might begin with case studies of teachers navigating well the tensions of accountability data and using evidence from their science classrooms.</p> <p>As we saw the need for teachers to navigate school culture and policies, one area for future investigation may be in the PLCs. PLCs generally include only teachers as a means to empower their voice and reduce power dynamics. However, structures that elevate the role of teacher learning side‐by‐side with the administrator may create bridges to what type of data is most useful for informing instruction. Similarly, PD providers must better recognize not only the constraints of teachers' particular contexts, but how to move from tensions with district mandates toward what opportunities are available within these mandates to enact elements of effective practice. For example, the Dimensions highlighted the importance of high‐quality feedback, but teachers raised concerns about a grade‐driven culture in their school. Within our own PD, we might have better responded to these contextual factors by having teachers chart out specific opportunities in which they could provide more impactful feedback and have students use that feedback to chart their own learning. In short, the Dimensions may serve as a common anchor for teachers wherein PD and PLCs benefit from integration of these Dimensions in context.</p> <hd id="AN0157236319-36">A potential developmental trajectory for growth</hd> <p>The theme of supporting teachers' professional growth in relation to the collection and use of intimate data moves beyond just tools. Our findings suggest that PD providers may benefit from attending to potential developmental learning trajectories of teachers around assessment practice, thus providing a nuanced and targeted approach. While the small sample size prevents broad generalizations across science teachers, it did allow for individual comparison of change across the Dimensions. The findings suggest that particular Dimensions pave the way for growth in other Dimensions. More proficient baseline ratings of Variety and Frequency appeared to be related to growth in Dimensions related to the Set Goals and Align Assessments Dimensions after the intervention. Furthermore, for participants who scored more proficient on these latter two Dimensions at the baseline Notebook collection, some positive changes were seen in the more difficult Feedback and Adapting Instruction Dimensions by the end of the project.</p> <p>From the patterns in our data, we propose that PD providers might frame the breadth of assessment practice, but then differentiate learning opportunities for teachers based on their baseline conceptions and practice (Desimone &amp; Garet, 2015). For those needing the greatest growth, beginning with the "Where are you now?" Dimensions and building up to the "Where are you going?" Dimensions may lead to more optimized opportunities to address the domains most conceptually aligned to improved student learning—the "How do you get there?" Dimensions of Feedback and Adapting Instruction. These latter two Dimensions are complex in their iterative loops and the ability to make connections across the goals, evidence of student thinking, and the possibilities in curricular and instructional changes that can lead to growth.</p> <p>Attitudes and conceptions of the Dimensions may all change after PD, but actual practice may need to be preceded by particular work on foundational Dimensions. This finding corroborates existing research in which teachers have difficulty using student data to inform instructional practice, even if they can articulate its importance (Horn et al., 2015). The scope and sequence of focus on Dimensions might also be tied to teachers' own motivations. Heredia's (2020) case study of teachers engaged in a 3‐year PD on formative assessment indicated stronger uptake of ideas from the PD if questions about assessment were raised from within the classroom. Future research should more fully explore the sequencing of PD and PD differentiation on teachers' assessment conceptions and practice. Our findings might also inform PD providers as they make decisions on areas of foci, especially in longitudinal models of assessment and data use PD.</p> <hd id="AN0157236319-37">Limitations</hd> <p>This study presents windows into the practice of middle school science teachers across four different school contexts. However, the Notebook's use of hard copy artifacts limited what could be collected. For instance, oral feedback given by the teacher to a student in class could not be captured. Our current work addresses this limitation as we, with other colleagues, have developed a tablet‐based application that captures live videos from classrooms as part of the Notebook.</p> <p>While some diversity of sampling is present, the small sample size limits the generalizability of teachers' conceptions and practice. In addition, the small sample size prevented meaningful statistical comparisons between ratings on the baseline and final Notebooks. For this set of teachers, though, the diverse and extensive amounts of data have allowed us to triangulate data amongst the quantitative ratings and the qualitative interviews and drawn models. In general, we focused our coding and subsequent claims on low‐inference decisions.</p> <hd id="AN0157236319-38">CONCLUSION</hd> <p>Science teaching that is both intellectually rigorous and equitable—two tenets foundational to AST—necessarily requires on‐going and active use of intimate classroom data, specifically evidence of student thinking and skills. While science teachers in our study valued the intimate forms of assessment data, their practice showed that one of AST's central features of using and responding to student ideas is an unnatural act. In practice, our teachers showed inconsistent use of proximal, formative assessments. While we did not see level of anticipated growth in critical Dimensions across the life of the project, we did see that two particular tools—also key elements for AST—the portfolio of artifacts and conceptual Dimensions of practice, were identified as key in shifting teachers' thinking and pedagogical aspirations, even if changes to actual practice are less prevalent.</p> <p>Importantly, our work raises again the ever‐present question about teacher change and the influence of professional learning experiences on the work of teaching (e.g., Garet et al., 2010). Despite a longitudinal intervention aligned to existing frameworks for quality PD, change was incremental and difficult for some of the teachers on two of the most consequential elements for students' learning. As Johnson (2007) succinctly notes, "Changing instructional practice takes time and support from all stakeholders in the school" (p. 657). Similarly, Martin and Hand (2009) demonstrate that it can take significantly longer for teachers to become proficient in complex practices—in this case, assessment—especially when the teachers have developed strategies they perceive as successful. AST is not named so because of its ease, but perhaps because of its high standard that in most school cultures is difficult to achieve. And so, as a field, we must continue to iterate on approaches for making AST possible for all teachers. As Vázquez‐Bernal et al. (2012) found, small changes in conceptions and practice took over nine years of engagement. Our own findings show that change is difficult. Yet, our results may also suggest new avenues to improve the efficacy of PD by attending more to the learning progression of teachers in their assessment practice and differentiating learning opportunities. Future research would benefit from exploring this potential developmental trajectory based on our nine Dimensions.</p> <hd id="AN0157236319-39">ACKNOWLEDGMENTS</hd> <p>The authors are grateful to the teachers who participated in this study, from whom we learned a great deal. The authors also acknowledge the generous support of the Spencer Foundation (Grant # 201400153) for their funding and professional development throughout the course of the grant.</p> <hd id="AN0157236319-40">CONFLICTS OF INTEREST</hd> <p>The authors declare no conflicts of interest.</p> <hd id="AN0157236319-41">DATA AVAILABILITY STATEMENT</hd> <p>The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.</p> <p>GRAPH: Supplementary information.</p> <ref id="AN0157236319-42"> <title> Footnotes </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> It should be noted that although the question asked for the Dimension of most growth, three teachers mentioned two Dimensions simultaneously.</bibtext> </blist> </ref> <ref id="AN0157236319-43"> <title> REFERENCES </title> <blist> <bibtext> Antoniou, P., &amp; Kyriakides, L. (2013). A dynamic integrated approach to teacher professional development: Impact and sustainability of the effects on improving teacher behaviour and student outcomes. Teaching and Teacher Education, 29, 1 – 12.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> Black, P., &amp; Wiliam, D. (2005). Developing a theory of formative assessment. In J. Gardner (Ed.), Assessment and learning (pp. 81 – 100). Sage.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref5" type="bt">3</bibl> <bibtext> Black, P., Harrison, C., Lee, C., Marshall, B., &amp; Wiliam, D. (2003). Assessment for learning. Open University Press.</bibtext> </blist> <blist> <bibl id="bib4" type="bt">4</bibl> <bibtext> Booher‐Jennings, J. (2005). Below the bubble: "Educational triage" and the Texas accountability system. American Educational Research Journal, 42 (2), 231 – 268.</bibtext> </blist> <blist> <bibl id="bib5" type="bt">5</bibl> <bibtext> Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33 (8), 3 – 15.</bibtext> </blist> <blist> <bibl id="bib6" type="bt">6</bibl> <bibtext> Boudett, K. P., City, E., &amp; Murnane, R. (2005). Data wise: A step‐by‐step guide to using assessment results to improve teaching and learning. Harvard Education Press.</bibtext> </blist> <blist> <bibl id="bib7" type="bt">7</bibl> <bibtext> Braaten, M., Bradford, C., Kirchgasler, K. L., &amp; Barocas, S. F. (2017). How data use for accountability undermines equitable science education. Journal of Educational Administration, 55 (4), 427 – 446.</bibtext> </blist> <blist> <bibl id="bib8" type="bt">8</bibl> <bibtext> Dedoose. (2018). Web application for managing, analyzing, and presenting qualitative and mixed method research data. SocioCultural Research Consultants, LLC.</bibtext> </blist> <blist> <bibl id="bib9" type="bt">9</bibl> <bibtext> Briscoe, C., &amp; Wells, E. (2002). Reforming primary science assessment practices: A case study of one teacher's professional development through action research. Science Education, 86 (3), 417 – 435.</bibtext> </blist> <blist> <bibtext> Butler, R. (1988). Enhancing and undermining intrinsic motivation: The effects of task‐involving and ego‐involving evaluation on interest and performance. British Journal of Educational Psychology, 58 (1), 1 – 14.</bibtext> </blist> <blist> <bibtext> Celio, M. B., &amp; Harvey, J. (2005). Buried treasure: Developing an effective management guide from mountains of educational data. Center on Reinventing Public Education.</bibtext> </blist> <blist> <bibtext> Chin, C., &amp; Brown, D. E. (2002). Student‐generated questions: A meaningful aspect of learning in science. International Journal of Science Education, 24 (5), 521 – 549.</bibtext> </blist> <blist> <bibtext> Coburn, C. E., &amp; Turner, E. O. (2011). Research on data use: A framework and analysis. Measurement, 9 (4), 173 – 206.</bibtext> </blist> <blist> <bibtext> Cohen, G. L., Steele, C. M., &amp; Ross, L. D. (1999). The mentor's dilemma: Providing critical feedback across the racial divide. Personality and Social Psychology Bulletin, 25 (10), 1302 – 1318.</bibtext> </blist> <blist> <bibtext> Darling‐Hammond, L., Hyler, M. E., &amp; Gardner, M. (2017). Effective teacher professional development. Learning Policy Institute.</bibtext> </blist> <blist> <bibtext> Datnow, A., &amp; Hubbard, L. (2016). Teacher capacity for and beliefs about data‐driven decision making: A literature review of international research. Journal of Educational Change, 17 (1), 7 – 28.</bibtext> </blist> <blist> <bibtext> DeShon, R. P., &amp; Alexander, R. A. (1996). Goal setting effects on implicit and explicit learning of complex tasks. Organizational Behavior and Human Decision Processes, 65 (1), 18 – 36.</bibtext> </blist> <blist> <bibtext> Desimone, L. M. (2009). Improving impact studies of teachers' professional development: Toward better conceptualizations and measures. Educational Researcher, 38 (3), 181 – 199.</bibtext> </blist> <blist> <bibtext> Desimone, L. M., &amp; Garet, M. S. (2015). Best practices in teachers' professional development in the United States. Psychology, Society, and Education, 7 (3), 252 – 263.</bibtext> </blist> <blist> <bibtext> Diamond, J. B. (2012). Accountability policy, school organization, and classroom practice: Partial recoupling and educational opportunity. Education and Urban Society, 44 (2), 151 – 182.</bibtext> </blist> <blist> <bibtext> Dodman, S. L., Swalwell, K., DeMulder, E. K., &amp; Stribling, S. M. (2021). Critical data‐driven decision making: A conceptual model of data use for equity. Teaching and Teacher Education, 99, 3 – 12.</bibtext> </blist> <blist> <bibtext> Downing, S. M. (2003). Validity: On the meaningful interpretation of assessment data. Medical Education, 37 (9), 830 – 837.</bibtext> </blist> <blist> <bibtext> Earl, L. M. (2012). Assessment as learning: Using classroom assessment to maximize student learning. Corwin.</bibtext> </blist> <blist> <bibtext> Falchikov, N. (2004). Involving students in assessment. Psychology Learning &amp; Teaching, 3 (2), 102 – 108.</bibtext> </blist> <blist> <bibtext> Furtak, E. M. (2009). Formative assessment for secondary science teachers. Corwin Press.</bibtext> </blist> <blist> <bibtext> Garet, M. S., Wayne, A. J., Stancavage, F., Taylor, J., Walters, K., Song, M., Brown, S., Hurlburt, S., Zhu, P., Sepanik, S., &amp; Doolittle, F. (2010). Middle school mathematics professional development impact study: Findings after the first year of implementation (NCEE 2010‐4009). National Center for Education Evaluation and Regional Assistance.</bibtext> </blist> <blist> <bibtext> Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M., Jones, W., Uekawa, K., Falk, A., Bloom, H. S., Doolittle, F., Zhu, P., &amp; Sztejnberg, L. (2008). The impact of two professional development interventions on early reading instruction and achievement (NCEE 2008‐4030). National Center for Educational Evaluation and Regional Assistance, Institute for Education Sciences, U.S. Department of Education.</bibtext> </blist> <blist> <bibtext> Grinath, A. S., &amp; Southerland, S. A. (2019). Applying the ambitious science teaching framework in undergraduate biology: Responsive talk moves that support explanatory rigor. Science Education, 103 (1), 92 – 122. https://doi.org/10.1002/sce.21484</bibtext> </blist> <blist> <bibtext> Grossman, P. L., Smagorinsky, P., &amp; Valencia, S. (1999). Appropriating tools for teaching English: A theoretical framework for research on learning to teach. American Journal of Education, 108 (1), 1 – 29.</bibtext> </blist> <blist> <bibtext> Guskey, T. (2019). Grades versus comments: Research on student feedback. Phi Delta Kappan, 101 (3), 42 – 47.</bibtext> </blist> <blist> <bibtext> Hamilton, L., Halverson, R., Jackson, S. S., Mandinach, E., Supovitz, J., &amp; Wayman, J. (2009). IES Practice Guide: Using student achievement data to support instructional decision making (NCEE 2009‐4067). National Center for Education Evaluation and Regional Assistance. <ulink href="http://ies.ed.gov/ncee/wwc/pdf/practice%5fguides/dddm%5fpg%5f092909.pdf">http://ies.ed.gov/ncee/wwc/pdf/practice%5fguides/dddm%5fpg%5f092909.pdf</ulink></bibtext> </blist> <blist> <bibtext> Hattie, J., &amp; Timperley, H. (2007). The power of feedback. Review of educational research, 77 (1), 81 – 112.</bibtext> </blist> <blist> <bibtext> Havnes, A., Smith, K., Dysthe, O., &amp; Ludvigsen, K. (2012). Formative assessment and feedback: Making learning visible. Studies in Educational Evaluation, 38 (1), 21 – 27.</bibtext> </blist> <blist> <bibtext> Heller, J. I., Daehler, K. R., Wong, N., Shinohara, M., &amp; Miratrix, L. W. (2012). Differential effects of three professional development models on teacher knowledge and student achievement in elementary science. Journal of Research in Science Teaching, 49 (3), 333 – 362.</bibtext> </blist> <blist> <bibtext> Heredia, S. C. (2020). Exploring the role of coherence in science teachers' sensemaking of science‐specific formative assessment in professional development. Science Education, 104 (3), 581 – 604.</bibtext> </blist> <blist> <bibtext> Horn, I. S. (2016). Accountability as a design for teacher learning: Sensemaking about mathematics and equity in the NCLB era. Urban Education, 1 – 26.</bibtext> </blist> <blist> <bibtext> Horn, I. S., Kane, B. D., &amp; Wilson, J. (2015). Making sense of student performance data use logics and mathematics teachers' learning opportunities. American Educational Research Journal, 52 (2), 208 – 242.</bibtext> </blist> <blist> <bibtext> Jeanpierre, B., Oberhauser, K., &amp; Freeman, C. (2005). Characteristics of professional development that effect change in secondary science teachers' classroom practices. Journal of Research in Science Teaching, 42 (6), 668 – 690.</bibtext> </blist> <blist> <bibtext> Jimerson, J. B. (2014). Thinking about data: Exploring the development of mental models for 'data use'among teachers and school leaders. Studies in Educational Evaluation, 42, 5 – 14.</bibtext> </blist> <blist> <bibtext> Johnson, C. C. (2007). Whole‐school collaborative sustained professional development and science teacher change: Signs of progress. Journal of Science Teacher Education, 18 (4), 629 – 661.</bibtext> </blist> <blist> <bibtext> Johnson, C. C., &amp; Fargo, J. D. (2010). Urban school reform enabled by transformative professional development: Impact on teacher change and student learning of science. Urban Education, 45 (1), 4 – 29.</bibtext> </blist> <blist> <bibtext> Johnson, C. C., Bolshakova, V. L., &amp; Waldron, T. (2016). When good intentions and reality meet: Large‐scale reform of science teaching in urban schools with predominantly Latino ELL students. Urban Education, 51 (5), 476 – 513.</bibtext> </blist> <blist> <bibtext> Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., &amp; Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education, 112 (4), 496 – 520.</bibtext> </blist> <blist> <bibtext> Knowles, M. S., Holton, E. F., III, &amp; Swanson, R. A. (2012). The adult learner. Routledge.</bibtext> </blist> <blist> <bibtext> Lave, J. (1991). Situating learning in communities of practice. Perspectives on Socially Shared Cognition, 63 – 82.</bibtext> </blist> <blist> <bibtext> Lipsey, M. W., &amp; Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta‐analysis. American Psychologist, 48 (12), 1181 – 1209.</bibtext> </blist> <blist> <bibtext> Manz, E. (2012). Understanding the codevelopment of modeling practice and ecological knowledge. Science Education, 96 (6), 1071 – 1105.</bibtext> </blist> <blist> <bibtext> Marsh, J. A., Pane, J. F., &amp; Hamilton, L. S. (2006). Making sense of data‐driven decision making in education (OP‐170‐EDU). RAND Corporation. <ulink href="http://www.rand.org/pubs/occasional%5fpapers/OP170/">http://www.rand.org/pubs/occasional%5fpapers/OP170/</ulink></bibtext> </blist> <blist> <bibtext> Martin, A. M., &amp; Hand, B. (2009). Factors affecting the implementation of argument in the elementary science classroom. A longitudinal case study. Research in Science Education, 39 (1), 17 – 38.</bibtext> </blist> <blist> <bibtext> Martínez, J. F., Borko, H., &amp; Stecher, B. M. (2012). Measuring instructional practice in science using classroom artifacts: Lessons learned from two validation studies. Journal of Research in Science Teaching, 49 (1), 38 – 67.</bibtext> </blist> <blist> <bibtext> Martínez, J. F., Borko, H., Stecher, B., Luskin, R., &amp; Kloser, M. (2012b). Measuring classroom assessment practice using instructional artifacts: A validation study of the QAS notebook. Educational Assessment, 17 (2–3), 107 – 131.</bibtext> </blist> <blist> <bibtext> McDonald, J. P., Isacoff, N. M., &amp; Karin, D. (2018). Data and teaching: Moving beyond magical thinking to effective practice. Teachers College Press.</bibtext> </blist> <blist> <bibtext> Mehta, J. D. (2013). The penetration of technocratic logic into the educational field: Rationalizing schooling from the progressives to the present. Teachers College Record, 115 (5), 1 – 36.</bibtext> </blist> <blist> <bibtext> National Research Council. (2001a). Classroom assessment and the national science education standards. National Academies Press.</bibtext> </blist> <blist> <bibtext> National Research Council. (2001b). Knowing what students know: The science and design of educational assessment. National Academies Press.</bibtext> </blist> <blist> <bibtext> National Research Council. (2012). A framework for K‐12 science education: Practices, crosscutting concepts, and core ideas. The National Academies Press.</bibtext> </blist> <blist> <bibtext> Nelson, T. H., Slavit, D., &amp; Deuel, A. (2012). Two dimensions of an inquiry stance toward student‐learning data. Teachers College Record, 114 (8), 1 – 42.</bibtext> </blist> <blist> <bibtext> Osborne, J. F., Borko, H., Fishman, E., Gomez Zaccarelli, F., Berson, E., Busch, K. C., Reigh, E., &amp; Tseng, A. (2019). Impacts of a practice‐based professional development program on elementary teachers' facilitation of and student engagement with scientific argumentation. American Educational Research Journal, 56 (4), 1067 – 1112.</bibtext> </blist> <blist> <bibtext> Penuel, W. R., &amp; Shepard, L. A. (2016a). Sociocognitive and sociocultural models of assessment. In A. Rupp, &amp; J. P. Leighton (Eds.), Cognitive models of assessment (pp. 146 – 173). Wiley.</bibtext> </blist> <blist> <bibtext> Penuel, W. R., &amp; Shepard, L. A. (2016b). Teaching and assessment. In D. H. Gitomer, &amp; C. A. Bell (Eds.), Handbook of research on science teaching (pp. 787 – 851). AERA.</bibtext> </blist> <blist> <bibtext> Penuel, W. R., Fishman, B. J., Yamaguchi, R., &amp; Gallagher, L. P. (2007). What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44 (4), 921 – 958.</bibtext> </blist> <blist> <bibtext> Penuel, W. R., McWilliams, H., McAuliffe, C., Benbow, A. E., Mably, C., &amp; Hayden, M. M. (2009). Teaching for understanding in earth science: Comparing impacts on planning and instruction in three professional development designs for middle school science teachers. Journal of Science Teacher Education, 20 (5), 415 – 436.</bibtext> </blist> <blist> <bibtext> Rivera Maulucci, M. S. (2010). Resisting the marginalization of science in an urban school: Coactivating social, cultural, material, and strategic resources. Journal of Research in Science Teaching, 47 (7), 840 – 860.</bibtext> </blist> <blist> <bibtext> Roediger III, H. L., &amp; Karpicke, J. D. (2006). Test‐enhanced learning: Taking memory tests improves long‐term retention. Psychological Science, 17 (3), 249 – 255.</bibtext> </blist> <blist> <bibtext> Rowland, C. A. (2014). The effect of testing versus restudy on retention: A meta‐analytic review of the testing effect. Psychological Bulletin, 140 (6), 1432 – 1464.</bibtext> </blist> <blist> <bibtext> Settlage, J., &amp; Meadows, L. (2002). Standards‐based reform and its unintended consequences: Implications for science education within America's urban schools. Journal of Research in Science Teaching, 39 (2), 114 – 127.</bibtext> </blist> <blist> <bibtext> Shavelson, R. J., Ruiz‐Primo, M. A., Li, M., &amp; Ayala, C. C. (2003). Evaluating new approaches to assessing learning (CSE Report 604). Center for the Study of Evaluation, National Center for Research on Evaluation, Standards, and Student Testing, Graduate School of Education &amp; Information Studies, University of California. <ulink href="http://www.cse.ucla.edu/products/Reports/R604.pdf">http://www.cse.ucla.edu/products/Reports/R604.pdf</ulink></bibtext> </blist> <blist> <bibtext> Shavelson, R. J., Young, D. B., Ayala, C. C., Brandon, P. R., Furtak, E. M., Ruiz‐Primo, M. A., Tomita, M. K., &amp; Yin, Y. (2008). On the impact of curriculum‐embedded formative assessment on learning: A collaboration between curriculum and assessment developers. Applied Measurement in Education, 21 (4), 295 – 314.</bibtext> </blist> <blist> <bibtext> Shepard, L. A., Davidson, K. L., &amp; Bowman, R. (2011). How middle school mathematics teachers use interim and benchmark assessment data (CRESST 807). University of California, Center for Research on Evaluation, Standards and Student Testing.</bibtext> </blist> <blist> <bibtext> Slavit, D., Nelson, T. H., &amp; Deuel, A. (2013). Teacher groups' conceptions and uses of student‐learning data. Journal of Teacher Education, 64 (1), 8 – 21.</bibtext> </blist> <blist> <bibtext> Stiggins, R. J. (2010). Essential formative assessment competencies for teachers and school leaders. In H. L. Andrade, &amp; G. J. Cizek (Eds.), Handbook of formative assessment (pp. 233 – 250). Taylor &amp; Francis.</bibtext> </blist> <blist> <bibtext> Sullivan, G. M. (2011). A primer on the validity of assessment instruments. Journal of Graduate Medical Education, 3 (2), 119 – 120.</bibtext> </blist> <blist> <bibtext> Thompson, J., Windschitl, M., &amp; Braaten, M. (2013). Developing a theory of ambitious early‐career teacher practice. American Educational Research Journal, 50 (3), 574 – 615.</bibtext> </blist> <blist> <bibtext> Treagust, D. F., Jacobowitz, R., Gallagher, J. L., &amp; Parker, J. (2001). Using assessment as a guide in teaching for understanding: A case study of a middle school science class learning about sound. Science Education, 85 (2), 137 – 157.</bibtext> </blist> <blist> <bibtext> Turner, J. C., Christensen, A., Kacker‐Cam, H. Z., Fulmer, S. M., &amp; Trucano, M. (2018). The development of professional learning communities and their teacher leaders: An activity systems analysis. Journal of the Learning Sciences, 27 (1), 49 – 88.</bibtext> </blist> <blist> <bibtext> Van Geel, M., Keuning, T., Visscher, A. J., &amp; Fox, J. ‐P. (2016). Assessing the effects of a school‐wide data‐based decision‐making intervention on student achievement growth in primary schools. American Educational Research Journal, 53 (2), 360 – 394.</bibtext> </blist> <blist> <bibtext> Vázquez‐Bernal, B., Mellado, V., Jiménez‐Pérez, R., &amp; Leñero, M. C. T. (2012). The process of change in a science teacher's professional development: A case study based on the types of problems in the classroom. Science Education, 96 (2), 337 – 363.</bibtext> </blist> <blist> <bibtext> Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.</bibtext> </blist> <blist> <bibtext> Wang, Y. (2019). Is data‐driven decision making at odds with moral decision making?: A critical review of school leaders' decision making in the era of school accountability. Values and Ethics in Educational Administration, 14 (2), 1 – 8.</bibtext> </blist> <blist> <bibtext> Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge University Press.</bibtext> </blist> <blist> <bibtext> Wenner, J. A., &amp; Campbell, T. (2017). The theoretical and empirical basis of teacher leadership: A review of the literature. Review of Educational Research, 87 (1), 134 – 171.</bibtext> </blist> <blist> <bibtext> White, B. Y., &amp; Frederickson, J. (1998). Inquiry, modeling and meta‐cognition: Making science accessible to all students. Cognition and Instruction, 16 (1), 3 – 118.</bibtext> </blist> <blist> <bibtext> Wiggins, G. (2012). Seven keys to effective feedback. Educational Leadership, 73 (7), 10 – 15.</bibtext> </blist> <blist> <bibtext> Wilsey, M., Kloser, M., Borko, H., &amp; Rafanelli, S. (2020). Middle school science teachers' conceptions of assessment practice throughout a year‐long professional development experience. Educational Assessment, 25 (2), 136 – 158.</bibtext> </blist> <blist> <bibtext> Windschitl, M., &amp; Calabrese Barton, A. (2016). Rigor and equity by design: Locating a set of core teaching practices for the science education community. In D. H. Gitomer, &amp; C. A. Bell (Eds.), Handbook of research on teaching (pp. 1099 – 1158). American Educational Research Association.</bibtext> </blist> <blist> <bibtext> Windschitl, M., Thompson, J., &amp; Braaten, M. (2018). Ambitious science teaching. Harvard Education Press.</bibtext> </blist> <blist> <bibtext> Windschitl, M., Thompson, J., Braaten, M., &amp; Stroupe, D. (2012). Proposing a core set of instructional practices and tools for teachers of science. Science Education, 96 (5), 878 – 903.</bibtext> </blist> <blist> <bibtext> Xu, Y., &amp; Brown, G. T. (2016). Teacher assessment literacy in practice: A reconceptualization. Teaching and Teacher Education, 58, 149 – 162.</bibtext> </blist> <blist> <bibtext> Yeager, D. S., Purdie‐Vaughns, V., Garcia, J., Apfel, N., Brzustoski, P., Master, A., Hessert, W. T., Williams, M. E., &amp; Cohen, G. L. (2014). Breaking the cycle of mistrust: Wise interventions to provide critical feedback across the racial divide. Journal of Experimental Psychology, 143 (2), 804 – 824.</bibtext> </blist> </ref> <aug> <p>By Matthew Kloser; Hilda Borko; Matthew Wilsey and Stephanie Rafanelli</p> <p>Reported by Author; Author; Author; Author</p> </aug> |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: EJ1337307 AccessLevel: 3 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Leveraging Portfolios in Professional Development for Middle School Science Teachers' Assessment and Data-Use Practice – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Kloser%2C+Matthew%22">Kloser, Matthew</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-4902-9854">0000-0002-4902-9854</externalLink>)<br /><searchLink fieldCode="AR" term="%22Borko%2C+Hilda%22">Borko, Hilda</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-7565-9307">0000-0002-7565-9307</externalLink>)<br /><searchLink fieldCode="AR" term="%22Wilsey%2C+Matthew%22">Wilsey, Matthew</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-0933-3069">0000-0002-0933-3069</externalLink>)<br /><searchLink fieldCode="AR" term="%22Rafanelli%2C+Stephanie%22">Rafanelli, Stephanie</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-5937-7510">0000-0002-5937-7510</externalLink>) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="SO" term="%22Science+Education%22"><i>Science Education</i></searchLink>. Jul 2022 106(4):924-955. – Name: Avail Label: Availability Group: Avail Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: Y – Name: Pages Label: Page Count Group: Src Data: 32 – Name: DatePubCY Label: Publication Date Group: Date Data: 2022 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Journal Articles<br />Reports - Research – Name: Audience Label: Education Level Group: Audnce Data: <searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink> – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22Portfolios+%28Background+Materials%29%22">Portfolios (Background Materials)</searchLink><br /><searchLink fieldCode="DE" term="%22Faculty+Development%22">Faculty Development</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Teachers%22">Science Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Use%22">Data Use</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Attitudes%22">Teacher Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Evaluation%22">Student Evaluation</searchLink> – Name: DOI Label: DOI Group: ID Data: 10.1002/sce.21712 – Name: ISSN Label: ISSN Group: ISSN Data: 0036-8326 – Name: Abstract Label: Abstract Group: Ab Data: Using evidence of student thinking and performance is crucial to enacting ambitious science teaching because intimate, formative data can be responsive to student ideas about science. However, narratives and policies about assessment and data use do not always position teachers to draw on the breadth and variety of evidence from the classroom to make instructional decisions that foster sensemaking. Thus, understanding whether, when, and how teachers view and use assessment and other forms of student data is critical to understanding classroom interactions and learning. This study explores teachers' baseline assessment conceptions and practices as well as their changes with relation to assessment and data use across a year-long intervention using a portfolio of classroom assessment artifacts. Middle school science teachers participated in professional development in which they reflected on assessment artifacts in light of nine dimensions of effective science assessment and data-use practice. Drawing on interviews, as well as baseline and outcome portfolios from the classroom, we noticed that initial conceptions of practice focused on the variety and frequency of assessment. Nine months later, participants' practice showed growth in several dimensions, but differentially among teachers based on their baseline proficiency. Teachers were more likely to show growth in dimensions related to feedback and data use if they had initially showed proficiency in articulating clear learning goals and had previously aligned their assessments to the stated goals. These findings suggest a possible developmental trajectory for improving teachers' assessment and data use practice. – Name: AbstractInfo Label: Abstractor Group: Ab Data: As Provided – Name: DateEntry Label: Entry Date Group: Date Data: 2022 – Name: AN Label: Accession Number Group: ID Data: EJ1337307 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1337307 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1002/sce.21712 Languages: – Text: English PhysicalDescription: Pagination: PageCount: 32 StartPage: 924 Subjects: – SubjectFull: Portfolios (Background Materials) Type: general – SubjectFull: Faculty Development Type: general – SubjectFull: Middle School Students Type: general – SubjectFull: Science Teachers Type: general – SubjectFull: Evaluation Methods Type: general – SubjectFull: Data Use Type: general – SubjectFull: Teacher Attitudes Type: general – SubjectFull: Student Evaluation Type: general Titles: – TitleFull: Leveraging Portfolios in Professional Development for Middle School Science Teachers' Assessment and Data-Use Practice Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Kloser, Matthew – PersonEntity: Name: NameFull: Borko, Hilda – PersonEntity: Name: NameFull: Wilsey, Matthew – PersonEntity: Name: NameFull: Rafanelli, Stephanie IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 07 Type: published Y: 2022 Identifiers: – Type: issn-print Value: 0036-8326 Numbering: – Type: volume Value: 106 – Type: issue Value: 4 Titles: – TitleFull: Science Education Type: main |
| ResultId | 1 |
