View in EDS HTML Full Text PDF Full Text

An Analysis of the Text Complexity of Leveled Passages in Four Popular Classroom Reading Assessments

Saved in:
Bibliographic Details
Title: An Analysis of the Text Complexity of Leveled Passages in Four Popular Classroom Reading Assessments
Language: English
Authors: Toyama, Yukie, Hiebert, Elfrieda H., Pearson, P. David
Source: Educational Assessment. 2017 22(3):139-170.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 32
Publication Date: 2017
Document Type: Journal Articles
Reports - Research
Education Level: Grade 1
Grade 2
Grade 3
Grade 4
Grade 5
Grade 6
Elementary Education
Descriptors: Reading Materials, Reading Skills, Student Evaluation, Difficulty Level, Instructional Program Divisions, Common Core State Standards, Reading Tests, Reading Fluency, Elementary School Students, Emergent Literacy, Evaluation Methods, Statistical Analysis, Comparative Analysis, Readability Formulas, Readability, Informal Reading Inventories, Reading Achievement, Regression (Statistics), Multivariate Analysis
Assessment and Survey Identifiers: Dynamic Indicators of Basic Early Literacy Skills (DIBELS), Flesch Kincaid Grade Level Formula, Basic Reading Inventory
DOI: 10.1080/10627197.2017.1344091
ISSN: 1062-7197
Abstract: This study investigated the complexity of leveled passages used in four classroom reading assessments. A total of 167 passages leveled for Grades 1-6 from these assessments were analyzed using four analytical tools of text complexity. More traditional, two-factor measures of text complexity found a general trend of fairly consistent across-grade progression of average complexity among the four assessments. However, considerable cross-assessment variability was observed in terms of the size of increase in complexity from grade to grade, the overall range of complexity, and the within-grade text complexity. These cross-assessment differences were less pronounced with newer, multi-factor analytical tools. The four assessments also differed in the extent to which their passages met the text complexity guidelines of the Common Core State Standards. The authors discuss implications of the differences found among and within the classroom assessment systems, on one hand, and among the measures of text complexity, on the other.
Abstractor: As Provided
Number of References: 90
Entry Date: 2017
Accession Number: EJ1150145
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFTNoXYbUYRRQj2kJ0Q4eKXAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDHWrvz8Jf0_iTntUrQIBEICBm4EQed0dWQKzjw06ti3UQ4882KHzTsKn059q4cl66PeLBNjzxbveGAV61MQB3tFOeAdpQAF7dKREXNs3mLPVrJ995Y5LxZVQlcSfliweXj9mN1Yu_NfaYLTrvOjsvyFW8qBnHotq8F3pyJST54FM-RzOWJmcvbp1faf4E0lBBsCfIvNKOPYv3Xwwa9gW9sjVKZoMluxK0ZQTq4pl
Text:
  Availability: 1
  Value: <anid>AN0124435026;7ls01jul.17;2019Feb12.17:08;v2.2.500</anid> <title id="AN0124435026-1">An Analysis of the Text Complexity of Leveled Passages in Four Popular Classroom Reading Assessments. </title> <p>This study investigated the complexity of leveled passages used in four classroom reading assessments. A total of 167 passages leveled for Grades 1–6 from these assessments were analyzed using four analytical tools of text complexity. More traditional, two-factor measures of text complexity found a general trend of fairly consistent across-grade progression of average complexity among the four assessments. However, considerable cross-assessment variability was observed in terms of the size of increase in complexity from grade to grade, the overall range of complexity, and the within-grade text complexity. These cross-assessment differences were less pronounced with newer, multi-factor analytical tools. The four assessments also differed in the extent to which their passages met the text complexity guidelines of the Common Core State Standards. The authors discuss implications of the differences found among and within the classroom assessment systems, on one hand, and among the measures of text complexity, on the other.</p> <p>Most educators and even the general public are aware of the expansion in the influence of group-administered standardized tests as tools to shape policy in American education. Less well known is the steady expansion in the use of classroom assessment tools in teachers' everyday decisions. Indeed, there are many more tools available for making student- or classroom-level decisions than there are for aggregate decisions about schools and districts (see, e.g., Anderson, Schlueter, Carlson, & Geisinger, [<reflink idref="bib4" id="ref1">4</reflink>]). Even so, while the media is filled with stories about the consequential impact of formal tests (Au, [<reflink idref="bib7" id="ref2">7</reflink>]; O'Neill, [<reflink idref="bib64" id="ref3">64</reflink>]), we seldom find stories in the popular press or archival literature about the consequences of classroom assessments for practice (although seldom is not never; see, e.g., Crooks, [<reflink idref="bib16" id="ref4">16</reflink>]; Goodman, [<reflink idref="bib31" id="ref5">31</reflink>]). The mundane purposes for which classroom assessments are used—placing students at the right level within a curriculum, determining which students have mastered certain skills and knowledge, and tracking student progress toward a target goal—do not appear to have earned the notoriety that standardized tests used for high stakes purposes, such as targeting schools for state takeover, do.</p> <p>Perhaps classroom assessment tools should receive greater scrutiny. After all, even though they are not generally used to make large-scale aggregate decisions, they are used to make important daily decisions about individual students within the curricula enacted in classrooms. For students (and their parents) who are placed in or denied access to particular programs, who have to repeat tasks they can already accomplish, or who are asked to read material that is either far below or above their current level of competence, these decisions are consequential in influencing students' school lives (Brookhart, [<reflink idref="bib10" id="ref6">10</reflink>]; Kontovourki, [<reflink idref="bib48" id="ref7">48</reflink>]). The quality of students' everyday activity and the appropriateness of the instruction and scaffolding they receive depend on the validity of these seemingly ordinary decisions.</p> <p>In the current study, we examine two types of the most widely used classroom reading assessments—informal reading inventories (IRIs; Nilsson, [<reflink idref="bib62" id="ref8">62</reflink>], [<reflink idref="bib63" id="ref9">63</reflink>]; Paris & Carpenter, [<reflink idref="bib66" id="ref10">66</reflink>]) and curriculum-based measurements (CBMs; Deno, [<reflink idref="bib19" id="ref11">19</reflink>]; Jenkins & Fuchs, [<reflink idref="bib39" id="ref12">39</reflink>]). IRIs are commonly used to determine the level of difficulty of texts that individual children can read on their own (i.e., independent level) or with teacher support (i.e., instructional level). CBMs serve a different purpose; they measure progress toward a specific learning target within a specific grade, such as reading end-of-year fourth-grade passages. Despite these differences in purpose, they share a common characteristic: both tools require reading passages that become progressively and predictably more difficult from one level to the next. Underlying this characteristic is an assumption that the passages are scaled on some underlying dimension of text complexity (inherent features of text) that predicts text difficulty, as traditionally indexed by student performance on a reading comprehension task or, increasingly often, by human judgment about text (Mesmer, Cunningham, & Hiebert, [<reflink idref="bib57" id="ref13">57</reflink>]).[<reflink idref="bib1" id="ref14">1</reflink>]</p> <hd id="AN0124435026-2">Rationale for the study</hd> <p></p> <hd id="AN0124435026-3">The logic of text complexity in classroom assessments</hd> <p>The primary purpose in giving an IRI is to find a student's instructional level. An underlying assumption of the IRI is that each passage level has a single reference point on a complexity continuum, or perhaps a relatively narrow range, forming a staircase of text complexity. Granted, along with the numeric index of reading levels, teachers can obtain diagnostic information if they take the time to analyze the specific patterns of errors or miscues made by students as they read aloud (Clay, [<reflink idref="bib14" id="ref15">14</reflink>]; Goodman & Burke, [<reflink idref="bib32" id="ref16">32</reflink>]). However, the essential points, when it comes to finding an instructional level, are the following: (a) texts at higher levels should consistently pose more challenge to readers than texts at lower levels, and (b) if there are alternative forms of the IRI (that one might use, for example, at different points in the year or to corroborate that a placement at a given level is accurate), the passages designated to be at a given level are of comparable, if not identical, complexity and difficulty across forms. The operational test of comparability would be that third-grade texts across different forms would be more similar to one another and pose more challenge to readers than second-grade texts but less challenge than fourth-grade texts.</p> <p>In contrast, CBMs are easy-to-administer assessments of targeted processes or practices that represent a desired outcome of instruction. These assessments are administered regularly throughout the school year (in some cases even weekly) to monitor student progress towards a specific performance target, with the expectation that instructional adjustments will be made for students whose progress is below par (Jenkins & Fuchs, [<reflink idref="bib39" id="ref17">39</reflink>]). In reading, the most common target performance is reading grade-level passages with fluency and, sometimes, comprehension. Thus, if the goal is to read end-of-the-year fourth grade passages with fluency and comprehension, then the population of items/tasks would consist of end-of-fourth-grade passages, even for students who are just beginning fourth grade. Guiding this approach is a classic principle of change measurement: <emph>If you want to measure change, don't change the measure</emph>. Of course, to prevent learning and memory effects from compromising the measure of change, one must use equivalent but not identical passages across time points. Unless that assumption is met, it is impossible to monitor progress toward the target because performance variations along the way might otherwise represent little more than students' responses to variations in text challenge.</p> <hd id="AN0124435026-4">An emerging role for classroom assessments</hd> <p>In recent years, classroom assessments such as IRIs appear to have assumed a role in policy decisions (Arthaud, Vasa, & Steckelberg, [<reflink idref="bib6" id="ref18">6</reflink>]; Ford & Opitz, [<reflink idref="bib23" id="ref19">23</reflink>]; Goodman, [<reflink idref="bib31" id="ref20">31</reflink>]; Paris, [<reflink idref="bib65" id="ref21">65</reflink>]). These purposes include universal screening and placement decisions (Albee, Arnold, Dennis, Schafer, & Olson, [<reflink idref="bib2" id="ref22">2</reflink>]; Parker et al., [<reflink idref="bib67" id="ref23">67</reflink>]), documenting student growth for accountability purposes (Paris, [<reflink idref="bib65" id="ref24">65</reflink>]), and evaluating the effectiveness of different forms of instruction (Nilsson, [<reflink idref="bib63" id="ref25">63</reflink>]; Stahl & Heubach, [<reflink idref="bib83" id="ref26">83</reflink>]). The latter two purposes typically require the assumption of equal intervals for IRI passage levels, particularly when users are interested in the magnitude of change or difference in reading performance between different time points or between students.</p> <p>Indeed, we found two specific instances of such assumption being made (or at least implied) to support everyday decisions as well as research-based claims about instructional effectiveness. The first instance is found in the user manuals from the developers of IRIs; they typically represent passage levels in a grid in their forms that document student progress within and across grades. This representation gives an equally sized square to each passage level (for specific examples, see DRA's Student Book Graph in Pearson Education, Inc., [<reflink idref="bib68" id="ref27">68</reflink>]; and Guided Reading's Record of Reading Progress in Fountas & Pinnell, [<reflink idref="bib25" id="ref28">25</reflink>]). In reading from these representations, teachers and other stakeholders who use these tools are transparently authorized to assume equal intervals of student growth from one passage level to next. Another consequential example of the equal interval assumption is found in efficacy studies (e.g., Ransford-Kaldon et al., [<reflink idref="bib73" id="ref29">73</reflink>]; Stahl & Heubach, [<reflink idref="bib83" id="ref30">83</reflink>]). In these studies, researchers express average student reading gain in terms of book/passage reading levels, and compare the treatment and control groups on this outcome variable using statistical procedures such as a t-test or linear regression. Such analysis is only possible if the researchers assume the levels assigned to passages are a continuous variable with equal intervals.</p> <p>While critics have questioned the psychometric properties of IRIs (Klesius & Homan, [<reflink idref="bib47" id="ref31">47</reflink>]; Pikulski & Shanahan, [<reflink idref="bib71" id="ref32">71</reflink>]; Spector, [<reflink idref="bib81" id="ref33">81</reflink>]), the most important observation is that little is known about the validity of text progression, either their complexity or their difficulty, in IRIs. What we do know is that empirical decisions are being made with an assumption of equal intervals, both in research studies and in everyday classroom decision-making.</p> <p>The most prominent CBMs are measures of oral reading fluency (ORF). The ORF component of the Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Good et al., [<reflink idref="bib30" id="ref34">30</reflink>]) was used widely during the No Child Left Behind (NCLB) implementation (Shelton, Altwerger, & Jordan, [<reflink idref="bib78" id="ref35">78</reflink>]) and has been further extended in the implementation of the Response to Intervention (RTI) framework for identifying students with learning disabilities (Mellard, McKnight, & Woods, [<reflink idref="bib55" id="ref36">55</reflink>]). Currently, DIBELS is used in over 28,000 schools worldwide, including approximately 20% of elementary schools in the U.S. (DIBELS Data System, [<reflink idref="bib20" id="ref37">20</reflink>]).</p> <p>The ORF component of DIBELS has been criticized for its narrow conceptualization of the reading process and the potential consequences of its use for instruction and learning (Goodman, [<reflink idref="bib31" id="ref38">31</reflink>]; Samuels, [<reflink idref="bib75" id="ref39">75</reflink>]; Valencia et al., [<reflink idref="bib87" id="ref40">87</reflink>]). For example, critics have suggested that when teachers use DIBELS subtest scores to plan and deliver instruction, speed and accuracy in oral reading are emphasized at the expense of other important aspects of reading such as prosody, vocabulary, and comprehension (Goodman, [<reflink idref="bib31" id="ref41">31</reflink>]; Kuhn, Schwanenflugel, & Meisinger, [<reflink idref="bib50" id="ref42">50</reflink>]; Samuels, [<reflink idref="bib75" id="ref43">75</reflink>]).</p> <p>These critiques aside, the ORF component of DIBELS is recognized as an empirically-validated standardized measure (Goffreda & DiPerna, [<reflink idref="bib29" id="ref44">29</reflink>]; Kame'enui et al., [<reflink idref="bib42" id="ref45">42</reflink>]). Passage equivalency in a given grade level has been identified to be an important property of ORFs, especially for capturing growth (Ardoin, Suldo, Witt, Aldrich, & Mcdonald, [<reflink idref="bib5" id="ref46">5</reflink>]; Christ & Ardoin, [<reflink idref="bib12" id="ref47">12</reflink>]; Francis et al., [<reflink idref="bib26" id="ref48">26</reflink>]; Jenkins, Zumeta, Dupree, & Johnson, [<reflink idref="bib40" id="ref49">40</reflink>]). Since ORF measures are used primarily for monitoring within grade-level progress, one might argue that unlike IRIs, ORFs do not need to meet the assumption of equal (or at least comparable) steps between any adjacent pair of grade levels. However, more careful consideration leads to the rejection of that possibility. It would be odd at best and misleading at worst if a school or district were not able to claim that the fourth-grade passages in their assessment system were more difficult than the third-grade passages by an amount that was similar to the difference between third-and second-grade passages. To assume otherwise would be tantamount to saying that students at different grade levels are required to meet different amounts of challenge to meet a common standard such as a year-worth of growth in school.</p> <p>Another important point about CBMs is that evidence points to their increasing use for diagnostic purposes (Albee et al., [<reflink idref="bib2" id="ref50">2</reflink>]; Kaminski et al., [<reflink idref="bib43" id="ref51">43</reflink>]) and as indices of growth (Christ, Monaghen, Zopluoglu, & Van Norman, [<reflink idref="bib13" id="ref52">13</reflink>]), thereby moving them into definite curricular-shaping roles (e.g., Shelton et al., [<reflink idref="bib78" id="ref53">78</reflink>]). Further, the DIBELS developer's position paper on the use of DIBELS for accountability suggests that DIBELS can be used for system accountability:</p> <p>Aggregation of DIBELS data at the systems level provides information that may be used to examine the effectiveness of the instructional supports within a classroom, school, or district to help determine when changes should be made. (Kaminski et al., [<reflink idref="bib43" id="ref54">43</reflink>], p. 1)</p> <hd id="AN0124435026-5">Scaling text complexity</hd> <p>For the task of scaling text complexity in an accurate, valid manner, literally hundreds of text complexity quantitative tools have been developed since the early 1920s (Klare, [<reflink idref="bib46" id="ref55">46</reflink>]). Mesmer ([<reflink idref="bib56" id="ref56">56</reflink>]) has described two generations of text complexity systems.</p> <hd id="AN0124435026-6">First-generation tools</hd> <p>First-generation tools typically rely on word and sentence difficulty in determining a text's readability, with the calculation done by hand or mechanically and with reference to conversion tables. Examples include the Flesch-Kincaid Grade Level (Kincaid, Fishburne, Rogers, & Chissom, [<reflink idref="bib44" id="ref57">44</reflink>]) and the Fry Readability Graph (Fry, [<reflink idref="bib27" id="ref58">27</reflink>]). Of the first-generation formulas, the Flesch-Kincaid is the most prominent in use today (as part of most word-processing programs).</p> <p>The Flesch-Kincaid formula essentially is a multiple regression equation as shown below:</p> <p>(<reflink idref="bib1" id="ref59">1</reflink>)</p> <p>Graph</p> <p>where ASL represents average sentence length and ASW represents average number of syllables per word. In Equation 1, 11.8 is the weight given to average word difficulty (i.e., average number of syllables per word), while 0.39 is the weight given to sentence difficulty (i.e., average words per sentence). Resulting readability scores correspond to grade levels (e.g., a score of 10 corresponds to Grade-10 readability).</p> <hd id="AN0124435026-7">Second-generation tools</hd> <p>Second-generation tools analyze texts digitally, allowing developers to use large corpora of text in validating formulas. Even with greater digital capability, however, word and sentence factors— the same foci of the first-generation tools—have dominated the digital systems. The Lexile Framework for Reading (Lexiles; Stenner, Burdick, Sanford, & Burdick, [<reflink idref="bib85" id="ref60">85</reflink>]) and Degrees of Reading Power (DRP; Koslin, Zeno, & Koslin, [<reflink idref="bib49" id="ref61">49</reflink>]) illustrate second-generation tools. The Lexile Framework is the most widely used with its influence expanding as evidenced by its use in defining text levels for grade bands in Appendix A of the Common Core State Standards (CCSS; National Governors Association [NGA], Center for Best Practices [CBP], & Council of Chief State School Officers [CCSSO], [<reflink idref="bib59" id="ref62">59</reflink>]).</p> <p>The developers of the Lexile Framework claim that it is not a readability formula (Smith, Stenner, Horabin, & Smith, [<reflink idref="bib80" id="ref63">80</reflink>]). Even so, Lexile's equation for scaling text complexity is based on the same two-factor model of text complexity as the first-generation readability formulas, as shown below (Stenner & Fisher, [<reflink idref="bib86" id="ref64">86</reflink>])[<reflink idref="bib2" id="ref65">2</reflink>]:</p> <p>Text Difficulty (in logit[<reflink idref="bib2" id="ref66">2</reflink>]) = (9.82247 * LMSL)−(2.14634 * MLWF)− constant (<reflink idref="bib2" id="ref67">2</reflink>)</p> <p>[<reflink idref="bib2" id="ref68">2</reflink>]where LMSL is the log of mean sentence length and MLWF is the mean of the log of the frequencies of each word in a text. Frequency is determined by the ranking of a word in a proprietorial multi-billion word corpus of text (Stenner & Fisher, [<reflink idref="bib86" id="ref69">86</reflink>]). A Lexile score typically ranges from below 0 to 2000L, with 200L anchored at the difficulty of first-grade basal texts and 1000L at that of a typical encyclopedia passage (Stenner et al., [<reflink idref="bib85" id="ref70">85</reflink>]). Its developers describe the Lexile scale as an interval scale, with one unit having the same meaning across the scale's range (Stenner et al., [<reflink idref="bib85" id="ref71">85</reflink>]) but several psychometricians have recently challenged this claim (Briggs, [<reflink idref="bib9" id="ref72">9</reflink>]; Domingue, [<reflink idref="bib21" id="ref73">21</reflink>]; Markus & Borsboom, [<reflink idref="bib54" id="ref74">54</reflink>]).</p> <hd id="AN0124435026-8">Third-generation tools</hd> <p>Recent years have seen the rise of quantitative analysis tools that use sophisticated statistical methods and multiple measures to determine text complexity. Coh-Metrix (Graesser, McNamara, & Kulikowich, [<reflink idref="bib34" id="ref75">34</reflink>]), TextEvaluator (TE, formerly known as Source-Rater; Sheehan, Kostin, Napolitano, & Flor, [<reflink idref="bib77" id="ref76">77</reflink>]), and Reading Maturity Metric (RMM; Landauer, Kireyev, & Panaccione, [<reflink idref="bib51" id="ref77">51</reflink>]) illustrate what might be called third-generation tools.</p> <p>The RMM measures a range of text structure features and vocabulary (Landauer et al., [<reflink idref="bib51" id="ref78">51</reflink>]). Vocabulary, identified as Word Maturity, is an application of Latent Semantic Analysis, a mathematical model of human language that simulates the development of word meanings as learners' exposure to language increases. RMM provides an overall text complexity score in grade-level units and, additionally identifies the 10 most difficult words in a given text.</p> <p>The TE system bases a text's complexity on eight dimensions: (a) academic vocabulary, (b) syntactic complexity, (c) word concreteness, (d) word unfamiliarity, (e) interactive/ conversational style, (f) degree of narrativity, (g) lexical cohesion, and (h) argumentation (Sheehan et al., [<reflink idref="bib77" id="ref79">77</reflink>]). These are principal components—moderately or highly correlated text features based on patterns of co-occurrence among 43 text features—derived from Principal Component Analysis. These eight components, in concert, accounted for over 60% of variation in text difficulty across a wide range of passages as judged by human experts. Another unique feature of TE is that it provides three prediction models, each specific to a text type (narrative, informational, or mixed). According to Sheehan et al. ([<reflink idref="bib77" id="ref80">77</reflink>]), these separate models overcome the genre bias in predicting text difficulty (i.e., overestimation of informational text caused by the repetition of rare content words and underestimation of narrative text due to short sentences in dialogue).</p> <hd id="AN0124435026-9">Comparisons across quantitative tools</hd> <p>The four analytical tools of text complexity that are used in this study are listed in Table 1. Nelson, Perfetti, Liben, and Liben ([<reflink idref="bib60" id="ref81">60</reflink>]) compared the strength of these four tools and three additional ones[<reflink idref="bib3" id="ref82">3</reflink>]—ATOS (Milone, [<reflink idref="bib58" id="ref83">58</reflink>]), DRP (Koslin et al., [<reflink idref="bib49" id="ref84">49</reflink>]), and Reader-Specific Practice (REAP; Heilman, Collins-Thompson, Callan, & Eskenazi, [<reflink idref="bib36" id="ref85">36</reflink>]) in predicting (a) grade-level placements of text exemplars from Appendix B of the CCSS made by human judges and (b) student comprehension performance on passages from the Stanford Achievement Test (SAT-9) and the Gates-MacGinite Reading Test, Form S.</p> <p>Table 1. Four Analytical Tools of Text Complexity Used in the Study.</p> <p> <ephtml> <table><thead><tr><td>Analytical Tools (developer)</td><td>Unit</td><td>Linguistic/Text Variables</td></tr><tr><td>Word Level</td><td>Sentence Level</td><td>Discourse/ Text Level</td></tr></thead><tbody><tr><td>Traditional Two-Factor</td><td><bold>Flesch-Kincaid Grade Level</bold>(Kincaid et al., <xref ref-type="bibr" rid="bibr44">1975</xref>)</td><td>Grade level</td><td><list list-type="Bullet"><list-item><p>Word length</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Sentence length</p></list-item></list></td><td /></tr><tr><td /><td><bold>Lexile</bold> (MetaMetrics)</td><td>Lexile</td><td><list list-type="Bullet"><list-item><p>Word frequency</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Sentence length</p></list-item></list></td><td /></tr><tr><td>Newer- Multi-Factor</td><td><bold>Reading Maturity Metric (RMM)</bold> (Pearson Education)</td><td>Grade level</td><td><list list-type="Bullet"><list-item><p>Word Maturity</p></list-item><list-item><p>Word length</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Sentence length</p></list-item><list-item><p>Punctuation</p></list-item><list-item><p>Coherence</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Coherence</p></list-item><list-item><p>Order of info</p></list-item><list-item><p>Paragraph complexity</p></list-item></list></td></tr><tr><td><bold>TextEvaluator (TE)</bold> (ETS)</td><td>Grade level</td><td><list list-type="Bullet"><list-item><p>Word unfamiliarity<sup>1</sup></p></list-item><list-item><p>Word concreteness<sup>1</sup></p></list-item><list-item><p>Academic vocablary<sup>1</sup></p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Syntactic complexity<sup>1</sup></p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Lexical cohesion<sup>1</sup></p></list-item><list-item><p>Interactive style<sup>1</sup></p></list-item><list-item><p>Narrativity<sup>1</sup></p></list-item><list-item><p>Argumentation<sup>1</sup></p></list-item></list></td></tr></tbody></table> </ephtml> </p> <p>10 A component derived from multiple variables based on principal component analysis.</p> <p>Nelson et al. ([<reflink idref="bib60" id="ref86">60</reflink>]) reported that rank-order correlations were reasonably high for all analytic systems except for REAP. Correlations with grade-band placements of CCSS exemplars ranged from a low of.50 (Lexiles) to a high of.76 (SourceRater—the earlier version of TE). Correlations were higher with reference measures that were based on student performance on standardized tests, ranging from.70 (Lexiles) to.80 (SourceRater) for the SAT-9. Nelson et al. ([<reflink idref="bib60" id="ref87">60</reflink>]) concluded that the multiple-factor text analytic systems—RMM and SourceRater—tended to have higher correlations, especially of text levels as determined by human judges, than two-factor tools, such as Lexile and ATOS. This finding lends support for the inclusion of the RMM and TE, along with the two widely used systems—Lexile and Flesch Kincaid— in the current study, given the fact the Developmental Reading Assessment (DRA; Beaver & Carter, [<reflink idref="bib8" id="ref88">8</reflink>]), one of the assessments examined in this study, relied largely on human judgment to scale passages.</p> <p>As a result of the Nelson et al. ([<reflink idref="bib60" id="ref89">60</reflink>]) study, the CCSS text complexity bands for Grades 2–12 were revised to include tools in addition to the Lexile Framework (NGA, CBP, & CCSSO, [<reflink idref="bib61" id="ref90">61</reflink>]). Table 2 provides these ranges for the two-factor tools (Flesch-Kincaid and Lexiles) and for the multi-factor tools (RMM & TE).</p> <p>Table 2. Updated Text Complexity Grade Bands and Associated Ranges from Multiple Measures.</p> <p> <ephtml> <table><thead><tr><td>CCSS Grade Band</td><td>Measures of Text Complexity</td></tr><tr><td>Flesch-Kincaid (grade level)</td><td>Lexile (lexile)</td><td>RMM (grade level)</td><td>TE (grade level)</td></tr></thead><tbody><tr><td>Grades 2 – 3</td><td>1.98–5.34</td><td>420–820</td><td>3.53–6.13</td><td>0.36–5.62</td></tr><tr><td>Grades 4 – 5</td><td>4.51–7.73</td><td>740–1010</td><td>5.42–7.92</td><td>3.97–8.40</td></tr><tr><td>Grades 6 – 8</td><td>6.51–10.34</td><td>925–1185</td><td>7.04–9.57</td><td>5.85–10.87</td></tr><tr><td>Grades 9 – 10</td><td>8.32–12.12</td><td>1050–1335</td><td>8.41–10.81</td><td>8.41–12.26</td></tr><tr><td>Grades 11 – CCR</td><td>10.34–14.2</td><td>1185–1385</td><td>9.57–12.00</td><td>9.62–13.47</td></tr></tbody></table> </ephtml> </p> <ulist> <item>11 <emph>Note</emph>. CCR = College & Career Ready; RMM = Reading Maturity Metric; TE = TextEvaluator.</item> <item>12 <emph>Sources</emph>. CCSS, Supplement to Appendix A (NGA, CBP, & CCSSO, [<reflink idref="bib61" id="ref91">61</reflink>]). TE's ranges are from Sheehan ([<reflink idref="bib76" id="ref92">76</reflink>]).</item> </ulist> <hd id="AN0124435026-10">Need for the current effort</hd> <p>Attention to issues of the text complexity of classroom assessment passages is timely and important for several reasons. First, classroom assessments are used for decisions that have consequences for students (e.g., determining eligibility for an intervention; what a student gets to read in and out of the classroom). The assessments, especially CBMs, are also being used for summative evaluation of student reading growth and/or effectiveness of interventions within the RTI framework (Deeney & Shim, [<reflink idref="bib18" id="ref93">18</reflink>]; Hall, [<reflink idref="bib35" id="ref94">35</reflink>]; Leslie & Caldwell, [<reflink idref="bib53" id="ref95">53</reflink>]; Mellard et al., [<reflink idref="bib55" id="ref96">55</reflink>]; Paris, [<reflink idref="bib65" id="ref97">65</reflink>]; Spector, [<reflink idref="bib81" id="ref98">81</reflink>]). Second, the CCSS has emphasized the need for all U.S. students to engage with texts of ever-increasing complexity to become college-and-career-ready. To meet the mandates of this new policy, educators need to know how assessments that place students into reading materials measure up to the recommended ranges of text complexity identified within the Appendix A of the CCSS (NGA, CBP, & CCSSO, [<reflink idref="bib59" id="ref99">59</reflink>]) and its supplemental document (NGA, CBP, & CCSSO, [<reflink idref="bib61" id="ref100">61</reflink>]). Third, new tools in the form of third-generation measures of text complexity have yet to be applied to the leveled passages widely used in classroom assessments.</p> <p>Given the situation that currently confronts the field—(a) a set of widely used consequential classroom assessments that have either questionable or unknown properties when it comes to leveling of their assessment passages, along with (b) the presence of new tools to scale text complexity—we undertook an investigation of the complexity of leveled passages used in popular classroom assessments. In attempting to achieve that goal, we addressed four related questions:</p> <p></p> <ulist> <item> How do the trajectories of text complexity compare across widely used classroom assessments?</item> <p></p> <item> How do these assessments compare in terms of the within-grade equivalency in text complexity?</item> <p></p> <item> In comparison to more traditional tools, do newer analytic tools of text complexity reveal different or additional information about the across-grade progression and the within-grade equivalency?</item> <p></p> <item> How well is the text complexity progression of these assessments aligned with the expectations of the staircase of complexity in the CCSS?</item> </ulist> <hd id="AN0124435026-11">Method</hd> <p></p> <hd id="AN0124435026-12">Selection of classroom assessments</hd> <p>The passages used in the current study came from four commonly used assessment systems—three IRIs and one CBM (see Table 3). Each of these assessments is administered individually, requiring a student read a passage and respond to comprehension and/or retelling prompts, to determine student reading performance on graded passages. At the same time, each assessment has unique properties, as outlined in the descriptions below, which are adapted from the manuals provided by the publishers.</p> <p>Table 3. Four Classroom Assessments Examined in the Study and Their Passage Leveling Method.</p> <p> <ephtml> <table><thead><tr><td /><td /><td>Method/Data for Passage Leveling Provided by the Developer</td></tr><tr><td>Assessment <italic>Numberof passage levels (range)</italic></td><td>Intended Uses</td><td>Across-Grade Progression</td><td>Within-Level Equivalency</td></tr></thead><tbody><tr><td><bold>BRI</bold> 11th edition (Johns, <xref ref-type="bibr" rid="bibr41">2012</xref>) <italic>15 levels (PP</italic>–<italic>Grade 12)</italic></td><td><list list-type="Bullet"><list-item><p>Estimate instructional level</p></list-item><list-item><p>Make diagnose</p></list-item><list-item><p>Identify struggling readers and plan interventions</p></list-item><list-item><p>Document growth</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Qualitative/anecdotal feedback from users during field testing</p></list-item><list-item><p>Readability formulas (Spache, Fry, Dale-Chall, Lexile) </p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Alternative-form reliability (.62-.92 for subset of Gr3-5 passages)</p></list-item><list-item><p>Generalizability study to detect form/passage effect for Gr4 passages</p></list-item></list></td></tr><tr><td><bold>DIBELS-Next</bold> Oral Fluency Benchmark & Cloze Passages (Good et al., <xref ref-type="bibr" rid="bibr30">2013</xref>) <italic>6 levels (Grades 1</italic>–<italic>6)</italic></td><td><list list-type="Bullet"><list-item><p>Identify struggling readers and plan interventions</p></list-item><list-item><p>Progress monitoring</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Developed own readability index and set target means & ranges of readability for each text level; passages were written to meet these targets</p></list-item><list-item><p>Removed outlier passages based on student oral reading performance data</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Within each level, passages were rank ordered based on student performance and passage triads were created so that triads' average difficulty is similar within a level;</p></list-item><list-item><p>Single-form reliability range from.75-.82. Three form reliability from.89-.94.</p></list-item><list-item><p>Several rigorous independent studies on passage/score equivalency</p></list-item></list></td></tr><tr><td><bold>DRA</bold> 2nd edition (Beaver & Carter, <xref ref-type="bibr" rid="bibr8">2006</xref>) <italic>23 levels (A – 80)</italic></td><td><list list-type="Bullet"><list-item><p>Determine independent reading level</p></list-item><list-item><p>Make diagnosis</p></list-item><list-item><p>Document annual/semi-annual development</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Professional judgment</p></list-item><list-item><p>Teacher feedback on overall passage leveling through surveys</p></list-item><list-item><p>Student accuracy data (<italic>n</italic> = 4)</p></list-item><list-item><p>Fry readability formula</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>For each level, multivariate analysis of variance with accuracy and comprehension as two dependent variables to detect passage effect</p></list-item></list></td></tr><tr><td><bold>QRI</bold> 5th edition (Leslie & Caldwell, <xref ref-type="bibr" rid="bibr53">2010</xref>) <italic>12 levels (PP*-High School)</italic></td><td><list list-type="Bullet"><list-item><p>Estimate instructional level</p></list-item><list-item><p>Make diagnose</p></list-item><list-item><p>Identify struggling readers and plan interventions</p></list-item><list-item><p>Document growth</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Readability formulas (e.g., Lexile, Average of New Dale-Chall, Fry, & Flesch Grade)</p></list-item><list-item><p>Qualitative text leveling systems (e.g., Fountas & Pinnell, <xref ref-type="bibr" rid="bibr24">1996</xref>)</p></list-item><list-item><p>Student rate, retell, & comprehension data</p></list-item></list></td><td><list list-type="Bullet"><list-item><p>Alternative-form reliability (0.75–1)</p></list-item></list></td></tr></tbody></table> </ephtml> </p> <p>13 <emph>Note</emph>. PP = pre-primer.</p> <hd id="AN0124435026-13">Informal reading inventories (IRIs)</hd> <p> <emph>The Basic Reading Inventory</emph> (BRI; 11th edition, Johns, [<reflink idref="bib41" id="ref101">41</reflink>]) and the <emph>Qualitative Reading Inventory</emph> (QRI; 5th edition, Leslie & Caldwell, [<reflink idref="bib53" id="ref102">53</reflink>]) are commercially available IRIs that involve a teacher recording a student's reading miscues when orally reading a passage, rate of reading, and responses to comprehension or retelling questions. Additionally, both IRIs provide graded word lists for the teacher to determine a starting passage for assessment, as well as questions to elicit student prior knowledge. Student's instructional level placement is derived from traditionally agreed criteria for oral reading accuracy and comprehension. Also can be collected are diagnostic information about student sources of errors, and patterns of comprehension and retelling.</p> <p>The length of BRI passages are standardized to around 100 words across Grade 1 though Grade 12 except for very short pre-primer and primer passages (25–50 words) and longer passages (250 words) available at Grade 3 and above. In contrast, QRI's passage length is much more variable across grades from 44 words for a pre-primer passage through 1,224 words for a high school passage. The manuals of BRI and QRI report the use of multiple readability formulas, such as Fry, Flesch-Kincaid, and Lexile, as part of their passage leveling process. However, neither system used the third-generation text complexity analytic tools. For the within-level equivalency, both IRIs report alternative form reliability (for details, see Table 3).</p> <p>The <emph>Developmental Reading Assessment</emph> (DRA; 2nd edition, Beaver & Carter, [<reflink idref="bib8" id="ref103">8</reflink>]) is based on the assessment traditions of Reading Recovery—an individualized reading intervention program for struggling readers in first grade (Clay, [<reflink idref="bib15" id="ref104">15</reflink>]). However, DRA extended the logic of the Reading Recovery assessment in three ways: (a) regular teachers, not specialists, administer the assessment, (b) all students, initially in K-3 and now in K-8, are the target of the assessment, rather than the "at risk" first graders, and (c) comprehension, in addition to oral reading, is assessed (Pearson Learning Group, [<reflink idref="bib69" id="ref105">69</reflink>]). In the administration of the DRA, teachers record reading accuracy, oral reading rates, retelling quality, and comprehension performance as students progress through a set of graded passages.</p> <p>The DRA is distinguished from the QRI and the BRI in three important aspects. First, the DRA has more levels than either of the IRIs, covering many more levels from Kindergarten through Grade 8. These levels are set as benchmarks at different points in an academic year (for example, levels 3 through 6 are benchmarks at the beginning of first grade while levels 16 and 18 are the benchmarks at the end of first grade). Second, the DRA differs from the other IRIs in text type and length; DRA's passages are authentic texts that come in the form of booklets that have the look and feel of trade books, with full color artwork. The DRA passages vary from 20 words in length for Level A to 1,914 words for Level 80.</p> <p>Finally, the DRA differs from BRI and QRI in the main method used to scale passages, privileging what the CCSS (National Governors Association [NGA], Center for Best Practices [CBP], & Council of Chief State School Officers [CCSSO], [<reflink idref="bib59" id="ref106">59</reflink>]) identify as qualitative rather than quantitative indicators (see Pearson & Hiebert, [<reflink idref="bib70" id="ref107">70</reflink>]). The majority of original K-3 assessment texts were chosen by a committee of teachers from reading materials typically used in classrooms (e.g., Scotts Foreman Reading Systems) and some passages were authored by teachers who were involved in the development of DRA. Further, following the logic of Reading Recovery, it was these teachers' professional judgments that guided the leveling of the DRA books; both linguistic (e.g., use of repetitive language) and nonlinguistic features (e.g., picture support) were taken into consideration in the leveling process (Pearson Learning Group, [<reflink idref="bib69" id="ref108">69</reflink>]). Although the manual mentions the use of the Fry readability formula, no details are provided about how it might have influenced the leveling outcomes for DRA. The leveling of the DRA assessment books were verified by larger groups of field trial teachers with a few survey questions on a Likert-scale (e.g., "The books were leveled appropriately").</p> <p>For the within-level equivalency, the manual reports that no passage effect was found at each DRA level from Levels 6 –80 based on the multivariate analysis of variance (MANOVA) with accuracy and comprehension as two dependent variables, with an exception of Level 34 (Pearson Education, Inc., [<reflink idref="bib68" id="ref109">68</reflink>]). However, it is difficult to judge the validity of these results due to the lack of details about how passages were distributed among students at different grade levels and about sample size for each MANOVA analysis (Rathvon, [<reflink idref="bib74" id="ref110">74</reflink>]).</p> <hd id="AN0124435026-14">Curriculum-based measure</hd> <p> <emph>The Dynamic Indicators of Basic Early Literacy Skills</emph> (DIBELS; Good et al., [<reflink idref="bib30" id="ref111">30</reflink>]) served as the curriculum-based measure. DIBELS is composed of sequenced subtests that assess literacy skills of students from Grades K to 6. The present investigation focused on DIBELS oral reading fluency (DORF), which is administered in winter and spring of Grade 1 and subsequently, administered three times per year (fall, winter, spring) from Grades 2 to 6. For the DORF, students read aloud three passages at a particular level for 1 minute each. The teacher records the median number of words read correctly.</p> <p>DORF passages were written and selected to meet grade-specific ranges of readability as determined by the Dynamic Measurement Group Passage Difficulty Index (Cummings, Wallin, Good, & Kaminski, [<reflink idref="bib17" id="ref112">17</reflink>]; Good et al., [<reflink idref="bib30" id="ref113">30</reflink>]). This index analyzes decoding difficulty (e.g., number of characters in word), word difficulty (e.g., proportion of rare word), and syntactic difficulty (e.g., number of syllables per sentence). To ensure within-level equivalency, triads of passages were carefully created based on student oral reading data so that the average difficulties of the triads of passages would be comparable for a particular benchmarking period. For the within-grade passage equivalency, the manual reports single-form reliability ranges from.75 to.82 while three-form reliability varies from.89 to.94 (Good et al., [<reflink idref="bib30" id="ref114">30</reflink>]).</p> <p>Table 3 summarizes the passage levels and the methods described by the four publishers of the assessments as the basis for their leveling. All four publishers claim to have used some quantitative measurement of text complexity in determining levels but none used the new, multi-factor measures of text complexity.</p> <hd id="AN0124435026-15">Selection of passages</hd> <p>The four assessment systems provided a sample of 167 passages for Grades 1 through 6. Three passages with fewer than 100 words were dropped from the sample as they were judged too short to be reliably analyzed. To maintain consistency across the four assessments, texts leveled below first grade (e.g., primer or kindergarten) and texts above sixth grade (the grade-level designation was done by the assessment developers) were eliminated. Additionally, to make the cross-assessment comparisons possible, DRA's passages were regrouped into coarser grade levels based on the information provided in DRA's technical manuals and assessment materials (Beaver & Carter, [<reflink idref="bib8" id="ref115">8</reflink>]; Pearson Education, Inc., [<reflink idref="bib68" id="ref116">68</reflink>]; Pearson Learning Group, [<reflink idref="bib69" id="ref117">69</reflink>]). A text file was created for each passage with title and headings removed. Table 4 provides the breakdown of the number of passages by grade levels and assessments.</p> <p>Table 4. Distribution of the Passages by Grade Levels and Assessments.</p> <p> <ephtml> <table><thead><tr><td>Assessment</td><td>Grade 1</td><td>Grade 2</td><td>Grade 3</td><td>Grade 4</td><td>Grade 5</td><td>Grade 6</td><td>Total</td></tr></thead><tbody><tr><td>BRI</td><td>5</td><td>5</td><td>7</td><td>7</td><td>5</td><td>6</td><td>35</td></tr><tr><td>DIBELS</td><td>6</td><td>9</td><td>9</td><td>9</td><td>9</td><td>9</td><td>51</td></tr><tr><td>DRA</td><td>10</td><td>14</td><td>12</td><td>4</td><td>4</td><td>4</td><td>48</td></tr><tr><td>QRI</td><td>4</td><td>5</td><td>5</td><td>6</td><td>6</td><td>7</td><td>33</td></tr><tr><td>Total</td><td>25</td><td>34</td><td>38</td><td>26</td><td>24</td><td>26</td><td>167</td></tr></tbody></table> </ephtml> </p> <hd id="AN0124435026-16">Analyses of texts</hd> <p>Each of the four quantitative measures of text complexity (see Table 1) were used to analyze the passages: Lexile, Flesch-Kincaid, RMM and TE. The first two—Lexile and Flesch-Kincaid—were chosen to represent two-factor, more traditional tools that have been and continue to be widely used in education. The latter two—RMM and TE—are referred as newer, multi-factor measures and were selected because they (a) employ a multi-dimensional approach to scaling text complexity, (b) provide an overall complexity score and (c) effectively predicted text complexity in the Nelson et al. ([<reflink idref="bib60" id="ref118">60</reflink>]) study. Pairwise correlations among the four analytical tools, obtained in this study, ranged from 0.75 and 0.88 (Table 5).</p> <p>Table 5. Correlations between the Four Analytical Tools of Text Complexity Used in the Study.</p> <p> <ephtml> <table><thead><tr><td>Measures</td><td>1.</td><td>2.</td><td>3.</td><td>4.</td></tr></thead><tbody><tr><td>1. Lexile</td><td>−</td><td /><td /><td /></tr><tr><td>2. Flesch Kincaid</td><td>0.88</td><td>−</td><td /><td /></tr><tr><td>3. RMM</td><td>0.84</td><td>0.79</td><td>−</td><td /></tr><tr><td>4. TextEvaluator</td><td>0.76</td><td>0.76</td><td>0.81</td><td>−</td></tr></tbody></table> </ephtml> </p> <p>Even though all the four analytic tools are available publicly online, developers of Lexile and TE offered batch analyses of the sample. For RMM analysis, we used its online beta version (<ulink href="http://www.readingmaturity.com">http://www.readingmaturity.com</ulink>). Flesch-Kincaid's overall complexity scores were obtained as part of the RMM analysis.</p> <p>All four analytical tools provided scores of text complexity for a passage, three in grade-level metrics and the fourth in Lexiles (represented with L). We should note that, while a passage was the basic unit in these analyses, we treated passages belonging to a given grade level as a set, which forms a step in the text complexity staircase, and examined summary statistics (e.g., mean, range) at passages' grade levels. This enabled the examination of across-grade progression as well as the within-grade variability in complexity of the assessment passages.</p> <hd id="AN0124435026-17">Results</hd> <p>The four research questions of this study pertain to: (a) across-grade patterns of text complexity of the four assessments, (d) within-grade equivalency in text complexity, (c) the nature of information provided by new, multi-factor text complexity tools relative to two-factor text complexity tools, and (d) the alignment of the text complexity progression of the four assessments with the CCSS expectations. Results from the analysis of the assessment passages with the two-factor text analysis tools are used to answer the first two questions. Results from all four analytical tools are used to address the third question. The last question is addressed with results from a two-factor and a multi-factor analysis tool—Lexile and RMM. Results from Flesch-Kincaid and TE are provided in the on-line supplementary materials.</p> <hd id="AN0124435026-18">Across-grade progression (Question 1)</hd> <p></p> <hd id="AN0124435026-19">Lexile framework</hd> <p>The results from the Lexile analyses are presented in Figure 1 (for actual values, see Table 6). In this figure (and subsequent figures that use a similar format), the x-axis shows the grade-level designation given by the assessment developer, while the y-axis represents the text complexity scores obtained from the text analytical tool used in this study. Each passage is shown as a circle. A line connects the grade-specific mean complexity (indicated by filled-in circles), showing the trajectory of average passage complexity from Grades 1 to 6. The upper edge of shaded area shows the progression of maximum complexity scores, while the lower edge shows the progression of minimum complexity scores. The height of the shaded area at a grade level shows the variability of text complexity from the least to most complex passage at that grade level.</p> <p>Table 6. Grade-Specific Summary Statistics for Lexile Scores by Assessment.</p> <p> <ephtml> <table><thead><tr><td /><td>BRI</td><td>DIBELS</td><td>DRA</td><td>QRI</td></tr><tr><td>Grade</td><td><italic>M</italic></td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td><italic>M</italic></td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td><italic>M</italic></td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td><italic>M</italic></td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td></tr></thead><tbody><tr><td>G1</td><td>160.0</td><td>(95.9)</td><td>60</td><td>290</td><td>5</td><td>553.3</td><td>(88.9)</td><td>400</td><td>660</td><td>6</td><td>412.0</td><td>(63.9)</td><td>280</td><td>510</td><td>10</td><td>392.5</td><td>(110.9)</td><td>310</td><td>550</td><td>4</td></tr><tr><td>G2</td><td>468.0</td><td>(61.4)</td><td>390</td><td>540</td><td>5</td><td>592.2</td><td>(41.2)</td><td>540</td><td>660</td><td>9</td><td>473.6</td><td>(102.9)</td><td>230</td><td>600</td><td>14</td><td>512.0</td><td>(70.5)</td><td>410</td><td>590</td><td>5</td></tr><tr><td>G3</td><td>577.1</td><td>(90.9)</td><td>400</td><td>640</td><td>7</td><td>782.2</td><td>(43.5)</td><td>710</td><td>850</td><td>9</td><td>555.8</td><td>(105.3)</td><td>390</td><td>760</td><td>12</td><td>732.0</td><td>(41.5)</td><td>670</td><td>770</td><td>5</td></tr><tr><td>G4</td><td>575.7</td><td>(129.2)</td><td>400</td><td>770</td><td>7</td><td>890.0</td><td>(54.3)</td><td>780</td><td>940</td><td>9</td><td>645.0</td><td>(71.9)</td><td>580</td><td>740</td><td>4</td><td>750.0</td><td>(162.6)</td><td>510</td><td>930</td><td>6</td></tr><tr><td>G5</td><td>750.0</td><td>(158.6)</td><td>600</td><td>950</td><td>5</td><td>915.6</td><td>(36.4)</td><td>860</td><td>970</td><td>9</td><td>685.0</td><td>(30.0)</td><td>660</td><td>720</td><td>4</td><td>810.0</td><td>(129.0)</td><td>650</td><td>980</td><td>6</td></tr><tr><td>G6</td><td>811.7</td><td>(91.3)</td><td>700</td><td>930</td><td>6</td><td>1012.2</td><td>(61.4)</td><td>930</td><td>1120</td><td>9</td><td>812.5</td><td>(53.2)</td><td>750</td><td>870</td><td>4</td><td>900.0</td><td>(105.7)</td><td>760</td><td>1040</td><td>7</td></tr><tr><td>Overall</td><td>566.6</td><td>(226.8)</td><td>60</td><td>950</td><td>35</td><td>804.9</td><td>(172.4)</td><td>400</td><td>1120</td><td>51</td><td>541.5</td><td>(144.1)</td><td>230</td><td>870</td><td>48</td><td>710.6</td><td>(199.2)</td><td>310</td><td>1040</td><td>33</td></tr><tr><td>Within-grade range M (SD)</td><td>Within-grade range M (SD)</td><td>Within-grade range M (SD)</td><td>Within-grade range M (SD)</td></tr><tr><td>261.67 (83.05)</td><td>163.33 (55.38)</td><td>218.33 (129.83)</td><td>258.33 (112.50)</td></tr></tbody></table> </ephtml> </p> <p>14 <emph>Note</emph>. Range was calculated by subtracting the minimum complexity score from the maximum complexity score within grade. It was then averaged across the six grades. Numbers in parenthesis are standard deviations.</p> <p>Graph: Figure 1. Distribution of complexity scores from Lexile. Each hollow circle represents a passage. Shaded areas show the range of Lexile scores within and across Grades 1–6. Black circles within the shaded area are the grade-specific mean complexity scores. The circles are connected by a line to show the progression of grade-specific means across the six grade levels.</p> <p>Three aspects of the trajectory observed in the Lexile results in Figure 1 are noteworthy. The first pattern has to do with the progression of average, minimum, and maximum complexity. For the most part, the trajectories for all four assessments show an upward trend of text complexity from Grades 1 to 6. Most grade-specific means go up from one grade to the next with one exception: BRI's mean complexity does not increase from Grades 3 to 4, staying at around 576L (see the flat line in the left most panel in Figure 1). Overall, the maximum and the minimum complexity scores also rise as the grade level of passages increases (see the edges of the shaded area climbing up in Figure 1). However, DRA's maximum and minimum and QRI's minimum values show some instances of decline in complexity as the grade level increases (e.g., QRI's minimum value show a sharp decline from Grades 3 to 4 and DRA's maximum value decreases over Grades 3 to 5).</p> <p>Second, differences in grade-to-grade changes in text complexity among the four assessments merit attention. Mean complexity generally increases with grade level across all four assessments, but differences are evident in the size of the increase from grade to grade on particular assessments. DRA's mean text complexity shows small increments of increase from Grades 1 to 5 and a large jump at the end of the progression from Grades 5 to 6 (685L to 813L). In contrast, the other three assessments tend to show larger increases at the beginning of the trajectory with the size of the increases tapering off in the later grades.</p> <p>The third pattern has to do with start and end points of the progressions of text complexity. Assessments differ in the start and end points of the progression with the BRI assessment at one extreme with a low starting range in Grade 1 (from 60L to 290L) and a low end range in Grade 6 (from 700L to 930L; recall that a range is represented as a height of the shaded area at a given grade level in Figure 1). At the other extreme is DIBELS with a high start range in Grade 1 (from 400L to 660L) and the highest end range of all assessments in Grade 6 (from 930L to 1120L). This may reflect little more than the fact that DIBELS is designed with the logic of a CBM— it attempts to measure reading performance with passages whose difficulty is set at the end-of-grade level. Consequently, DIBELS' passages are skewed toward the upper end of the difficulty band for each grade. In terms of an overall range across all grade levels from the least to the most complex passage within an assessment, BRI covers the widest span (60L to 950L), while DRA covers the narrowest (230L to 870L).</p> <p>A regression analysis using Lexile scores treated as a response variable and grade level, the assessments, and their interactions as predictor variables was conducted to determine whether grade-specific means were statistically different from one another. The regression model used White's robust estimator for standard error to adjust for heteroscedasticity (White, [<reflink idref="bib90" id="ref119">90</reflink>]). Details of this analysis are provided in Appendix (Table A1).</p> <p>Figure 2 shows the grade- and assessment-specific mean complexity scores estimated by the regression models. A vertical line extending above and below each estimated mean represents the 95% confidence interval for the mean. As can be seen in the non-overlapping confidence intervals in Figure 2, BRI's estimated mean complexity for Grade 1 is significantly lower compared to the other assessments. DIBELS' estimated means are significantly higher than those of BRI and DRA at all six grade levels. Post-hoc multiple comparisons with Šidák corrections (Šidák, [<reflink idref="bib79" id="ref120">79</reflink>]) confirmed these observations (see Figure 9 and Table A2 in Appendix).</p> <p>Graph: Figure 2. Grade- and assessment-specific means with 95% confidence intervals (a line extending vertically above and below each marker). The means and the lines connecting them are identical from Figure 1. The confidence intervals were estimated from a regression analysis, using Lexile scores as a response variable, grades, assessments, and their interactions as predictor variables. DIBELS means are distinguishably higher at all six levels than those of BRI and DRA as evidence in no overlapping confidence intervals. Similarly, BRI's Grade 1 mean complexity is distinguishably lower than that of other assessments.</p> <p>Graph: Figure 3. Distribution of complexity scores by Flesch-Kincaid Grade Level. See caption in Figure 1.</p> <p>Graph: Figure 4. Grade- and assessment-specific means with 95% confidence intervals from a regression analysis, using Flesch Kincaid Grade Level scores as a response variable, grades, assessments, and their interactions as predictor variables. See caption in Figure 2.</p> <p>Graph: Figure 5. Distribution of complexity scores by RMM. Please see caption in Figure 1.</p> <p>Graph: Figure 6. Grade- and assessment-specific means with 95% confidence intervals from a regression analysis, using RMM Grade Level scores as a response variable, grades, assessments, and their interactions as predictor variables. Please see caption in Figure 2.</p> <p>Graph: Figure 7. Distribution of complexity scores by TextEvaluator. Please see caption in Figure 1.</p> <p>Graph: Figure 8. Grade- and assessment-specific means with 95% confidence intervals from a regression analysis, using TextEvaluator Grade Level scores as a response variable, grades, assessments, and their interactions as predictor variables. Please see caption in Figure 2.</p> <p>Graph: Figure 9. Estimated differences in mean complexity between DIBELS and a comparison assessment with 95% confidence intervals (constructed with Sidak corrections). The significant differences are those for which the intervals do not intersect with the dashed zero line. Lexile and Flesch-Kincaid results have more instances of significant difference (11 and 9 out of 18 comparisons respectively) than results from RMM and TextEvaluator (each shows four instances).</p> <hd id="AN0124435026-20">Flesch-Kincaid</hd> <p>The results from the Flesch-Kincaid analyses, provided in Figures 3 and 4, reveal similar trends to those observed with Lexile (Table 7 provides values corresponding to the figures). A general trend of upward trajectory for between-grade progression was observed across the four assessments. Differences across assessments are evident at the starting point of the progression. The first-grade passages of BRI show a lower range of complexity (from −1.7 to 1 in Flesch-Kincaid grade-level unit) than the other assessments. In contrast, DIBELS' first-grade passages start out at a higher range (from −0.1 to 2.3) and its six-grade passages achieve the highest range of any assessments (from 6.6 to 9).</p> <p>Table 7. Grade-Specific Summary Statistics for Flesch-Kincaid Grade Level Scores by Assessment.</p> <p> <ephtml> <table><thead><tr><td /><td>BRI</td><td>DIBELS</td><td>DRA</td><td>QRI</td></tr><tr><td>Grade</td><td><italic>M</italic></td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td><italic>M</italic></td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td><italic>M</italic></td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td><italic>M</italic></td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td></tr></thead><tbody><tr><td>G1</td><td>−0.3</td><td>(1.0)</td><td>−1.7</td><td>1</td><td>5</td><td>1.4</td><td>(1.0)</td><td>−0.1</td><td>2.3</td><td>6</td><td>1.5</td><td>(1.3)</td><td>−0.2</td><td>4.7</td><td>10</td><td>1.1</td><td>(0.3)</td><td>0.7</td><td>1.4</td><td>4</td></tr><tr><td>G2</td><td>1.6</td><td>(1.5)</td><td>0.2</td><td>4</td><td>5</td><td>3.3</td><td>(1.1)</td><td>1.5</td><td>4.8</td><td>9</td><td>1.9</td><td>(1.0)</td><td>−0.2</td><td>3.6</td><td>14</td><td>1.9</td><td>(0.4)</td><td>1.4</td><td>2.3</td><td>5</td></tr><tr><td>G3</td><td>2.2</td><td>(1.1)</td><td>0.4</td><td>3.5</td><td>7</td><td>4.6</td><td>(0.7)</td><td>3.7</td><td>5.7</td><td>9</td><td>2.5</td><td>(0.9)</td><td>1.1</td><td>4.2</td><td>12</td><td>3.9</td><td>(0.5)</td><td>3.3</td><td>4.4</td><td>5</td></tr><tr><td>G4</td><td>3.1</td><td>(1.6)</td><td>0.4</td><td>5.6</td><td>7</td><td>5.5</td><td>(1.0)</td><td>4.4</td><td>7.7</td><td>9</td><td>3.0</td><td>(1.1)</td><td>2</td><td>4.5</td><td>4</td><td>4.6</td><td>(1.1)</td><td>3.4</td><td>6</td><td>6</td></tr><tr><td>G5</td><td>5.6</td><td>(1.7)</td><td>3.3</td><td>7.3</td><td>5</td><td>6.7</td><td>(0.7)</td><td>5.2</td><td>7.9</td><td>9</td><td>3.1</td><td>(0.7)</td><td>2.4</td><td>4</td><td>4</td><td>5.9</td><td>(1.5)</td><td>4.3</td><td>8.4</td><td>6</td></tr><tr><td>G6</td><td>5.2</td><td>(1.6)</td><td>3.5</td><td>7.1</td><td>6</td><td>7.6</td><td>(0.7)</td><td>6.6</td><td>9</td><td>9</td><td>4.7</td><td>(0.6)</td><td>4.1</td><td>5.2</td><td>4</td><td>6.9</td><td>(0.7)</td><td>5.9</td><td>7.9</td><td>7</td></tr><tr><td>Overall</td><td>2.9</td><td>(2.4)</td><td>−1.7</td><td>7.3</td><td>35</td><td>5.1</td><td>(2.1)</td><td>−0.1</td><td>9</td><td>51</td><td>2.4</td><td>(1.3)</td><td>−0.2</td><td>5.2</td><td>48</td><td>4.4</td><td>(2.2)</td><td>0.7</td><td>8.4</td><td>33</td></tr><tr><td>Within-grade range <italic>M (SD)</italic></td><td>Within-grade range <italic>M (SD)</italic></td><td>Within-grade range <italic>M (SD)</italic></td><td>Within-grade range <italic>M (SD)</italic></td></tr><tr><td>3.73 (0.86)</td><td>2.68 (0.53)</td><td>2.83 (1.41)</td><td>1.90 (2.33)</td></tr></tbody></table> </ephtml> </p> <p>15 <emph>Note</emph>. Range was calculated by subtracting the minimum complexity score from the maximum complexity score within grade. It was then averaged across the six grades. Numbers in parenthesis are standard deviations.</p> <p>One distinct difference from the Lexile results is that the regression analysis with Flesch-Kincaid results indicates that DRA's mean complexity at Grade 5 is distinctively lower than that of other three assessments (see Figure 4). The post-hoc multiple comparisons confirmed its statistical significance (see Figure 9 under Flesch-Kincaid results) of this finding.</p> <hd id="AN0124435026-21">Within-grade complexity trajectory across grade levels (Question 2)</hd> <p></p> <hd id="AN0124435026-22">Lexile framework</hd> <p>Within-grade equivalency of passages is presented as the height of the shaded area for a grade level in Figure 1. Figure 1 shows consistently narrow within-grade bands of text complexity across the grade-level groups for DIBELS, indicating a relatively high homogeneity in passage complexity within a grade level (mean within-grade range = 163.3L; <emph>SD</emph> = 55.4L; see Table 6). Other assessments show a narrow range of complexity at some grade levels but the pattern of within-grade variability is not consistent across grade levels. For example, DRA's Grade 5 passages show the smallest range of complexity overall (60L) but passages at Grades 2 and 3 have a wide range (370L). In comparison to other assessments, QRI's passages, on average, exhibit the most variability within grade levels (mean within-grade range = 285.3L; <emph>SD</emph> = 112.5L).</p> <hd id="AN0124435026-23">Flesch-Kincaid</hd> <p>Consistent with the results from Lexile, the Flesch-Kincaid results for within-grade passage equivalency indicate that DIBELS passages consistently reveal a narrow within-grade range of complexity across the six grade levels (mean within-grade range = 2.7; <emph>SD</emph> = 0.5; see Figure 3 and Table 7). BRI passages are most variable in terms of the average within-grade-level range (<emph>M </emph>= 3.7; <emph>SD </emph>= 0.9). Unlike the Lexile results, the Flesh-Kincaid analysis revealed that QRI has the smallest average within-grade range of complexity (<emph>M </emph>= 1.90; <emph>SD </emph>= 2.3) but its relatively large standard deviation shows that QRI's complexity score ranges are not consistent across grade levels.</p> <p>To summarize both the Lexile and Flesch-Kincaid results indicate a general pattern of across grade progression of mean text complexity as passages' designated grade levels increase. However, there is considerable variability among the four assessments in the size of change from grade to grade, the start and end point of the across-grade progression, and the within-grade equivalency of passage complexity.</p> <hd id="AN0124435026-24">Patterns revealed with multi-component tools of text complexity (Question 3)</hd> <p>Results from the newer analytic tools of text complexity (RMM and TE) are shown in Figures 5–6 for RMM, while the TE results are presented in Figures 7–8 (Tables 8 and 9 provide the data for these figures respectively).</p> <p>Table 8. Grade-Specific Summary Statistics for RMM Grade-Level Scores by Assessment.</p> <p> <ephtml> <table><thead><tr><td /><td>BRI</td><td>DIBELS</td><td>DRA</td><td>QRI</td></tr><tr><td>Grade</td><td>mean</td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td>mean</td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td>mean</td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td>mean</td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td></tr></thead><tbody><tr><td>G1</td><td>1.3</td><td>(0.9)</td><td>0.5</td><td>2.8</td><td>5</td><td>2.9</td><td>(1.1)</td><td>1.7</td><td>4.8</td><td>6</td><td>2.2</td><td>(0.9)</td><td>0.8</td><td>3.7</td><td>10</td><td>2.6</td><td>(0.6)</td><td>1.6</td><td>3</td><td>4</td></tr><tr><td>G2</td><td>2.1</td><td>(0.9)</td><td>1.5</td><td>3.7</td><td>5</td><td>3.6</td><td>(0.4)</td><td>3.1</td><td>4.3</td><td>9</td><td>2.6</td><td>(0.8)</td><td>1</td><td>3.8</td><td>14</td><td>2.7</td><td>(0.9)</td><td>1.5</td><td>4</td><td>5</td></tr><tr><td>G3</td><td>3.4</td><td>(1.0)</td><td>2.5</td><td>5.4</td><td>7</td><td>4.8</td><td>(1.0)</td><td>3</td><td>6</td><td>9</td><td>4.1</td><td>(0.4)</td><td>3.3</td><td>4.6</td><td>12</td><td>4.6</td><td>(0.8)</td><td>3.5</td><td>5.3</td><td>5</td></tr><tr><td>G4</td><td>3.8</td><td>(1.1)</td><td>2.5</td><td>5.1</td><td>7</td><td>5.5</td><td>(0.6)</td><td>4.4</td><td>6.2</td><td>9</td><td>4.3</td><td>(0.7)</td><td>3.5</td><td>4.9</td><td>4</td><td>4.4</td><td>(1.5)</td><td>3.3</td><td>7.1</td><td>6</td></tr><tr><td>G5</td><td>5.5</td><td>(0.9)</td><td>4.4</td><td>6.9</td><td>5</td><td>6.5</td><td>(0.7)</td><td>4.9</td><td>7.3</td><td>9</td><td>5.5</td><td>(0.7)</td><td>4.9</td><td>6.3</td><td>4</td><td>5.6</td><td>(1.0)</td><td>4.6</td><td>7.3</td><td>6</td></tr><tr><td>G6</td><td>6.1</td><td>(1.0)</td><td>4.6</td><td>7.8</td><td>6</td><td>6.8</td><td>(0.3)</td><td>6.4</td><td>7.1</td><td>9</td><td>6.5</td><td>(1.3)</td><td>4.8</td><td>7.9</td><td>4</td><td>6.1</td><td>(1.1)</td><td>4.7</td><td>8.2</td><td>7</td></tr><tr><td>Overall</td><td>3.8</td><td>(1.9)</td><td>0.5</td><td>7.8</td><td>35</td><td>5.1</td><td>(1.5)</td><td>1.7</td><td>7.3</td><td>51</td><td>3.6</td><td>(1.5)</td><td>0.8</td><td>7.9</td><td>48</td><td>4.5</td><td>(1.6)</td><td>1.5</td><td>8.2</td><td>33</td></tr><tr><td>Within-grade range <italic>M (SD)</italic></td><td>Within-grade range <italic>M (SD)</italic></td><td>Within-grade range <italic>M (SD)</italic></td><td>Within-grade range <italic>M (SD)</italic></td></tr><tr><td>2.62 (0.38)</td><td>2.03 (0.97)</td><td /><td>2.15 (0.86)</td><td /><td>2.61 (0.93)</td></tr></tbody></table> </ephtml> </p> <p>16 <emph>Note</emph>. Range was calculated by subtracting the minimum complexity score from the maximum complexity score within grade. It was then averaged across the six grades. Numbers in parenthesis are standard deviations.</p> <p>Table 9. Grade-Specific Summary Statistics for TextEvaluator Grade-Level Scores by Assessment.</p> <p> <ephtml> <table><thead><tr><td /><td>BRI</td><td>DIBELS</td><td>DRA</td><td>QRI</td></tr><tr><td>Grade</td><td>mean</td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td>mean</td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td>mean</td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td><td>mean</td><td><italic>(SD)</italic></td><td>min</td><td>max</td><td><italic>N</italic></td></tr></thead><tbody><tr><td>G1</td><td>1.2</td><td>(0.4)</td><td>1</td><td>2</td><td>5</td><td>1.3</td><td>(0.5)</td><td>1</td><td>2.1</td><td>6</td><td>1.0</td><td>(0.0)</td><td>1</td><td>1</td><td>10</td><td>1.4</td><td>(0.5)</td><td>1</td><td>2.1</td><td>4</td></tr><tr><td>G2</td><td>1.3</td><td>(0.4)</td><td>1</td><td>2</td><td>5</td><td>1.9</td><td>(1.1)</td><td>1</td><td>4.2</td><td>9</td><td>1.4</td><td>(0.6)</td><td>1</td><td>2.6</td><td>14</td><td>2.1</td><td>(1.2)</td><td>1</td><td>4</td><td>5</td></tr><tr><td>G3</td><td>2.0</td><td>(0.9)</td><td>1</td><td>3.6</td><td>7</td><td>4.5</td><td>(0.9)</td><td>3</td><td>5.5</td><td>9</td><td>2.0</td><td>(0.9)</td><td>1</td><td>3.5</td><td>12</td><td>3.0</td><td>(1.4)</td><td>1</td><td>4.6</td><td>5</td></tr><tr><td>G4</td><td>2.4</td><td>(1.5)</td><td>1</td><td>5.3</td><td>7</td><td>4.6</td><td>(1.5)</td><td>3.3</td><td>8.3</td><td>9</td><td>2.3</td><td>(1.0)</td><td>1</td><td>3.4</td><td>4</td><td>3.5</td><td>(1.0)</td><td>2.1</td><td>4.7</td><td>6</td></tr><tr><td>G5</td><td>4.2</td><td>(1.1)</td><td>2.6</td><td>5.6</td><td>5</td><td>5.2</td><td>(0.9)</td><td>3.5</td><td>6.4</td><td>9</td><td>4.7</td><td>(1.0)</td><td>3.9</td><td>6.2</td><td>4</td><td>4.3</td><td>(1.5)</td><td>2.5</td><td>6.6</td><td>6</td></tr><tr><td>G6</td><td>4.1</td><td>(1.3)</td><td>1.9</td><td>5.7</td><td>6</td><td>6.1</td><td>(0.9)</td><td>4.5</td><td>8</td><td>9</td><td>5.3</td><td>(1.5)</td><td>3.6</td><td>7.2</td><td>4</td><td>5.5</td><td>(0.4)</td><td>4.7</td><td>6</td><td>7</td></tr><tr><td>Total</td><td>2.6</td><td>(1.6)</td><td>1</td><td>5.7</td><td>35</td><td>4.1</td><td>(2.0)</td><td>1</td><td>8.3</td><td>51</td><td>2.1</td><td>(1.5)</td><td>1</td><td>7.2</td><td>48</td><td>3.5</td><td>(1.7)</td><td>1</td><td>6.6</td><td>33</td></tr><tr><td>Within-grade range <italic>M (SD)</italic></td><td>Within-grade range <italic>M (SD)</italic></td><td>Within-grade range <italic>M (SD)</italic></td><td>Within-grade range <italic>M (SD)</italic></td></tr><tr><td>2.61 (1.38)</td><td>4.75 (2.36)</td><td>2.07 (1.20)</td><td>2.62 (1.21)</td></tr></tbody></table> </ephtml> </p> <p>17 <emph>Note</emph>. Range was calculated by subtracting the minimum complexity score from the maximum complexity score within grade. It was then averaged across the six grades. Numbers in parenthesis are standard deviations.</p> <p>Patterns in Figures 5, 6, 7 and 8 suggest that the multi-component tools produce less pronounced differences in across-grade trajectories and within-grade equivalency in the four assessments than do the two-factor tools. Direct comparison of Lexile results with RMM and TE results is not possible due to the difference in the units that text complexity is scaled. However, when compared to the results from Flesch-Kincaid, the distribution of complexity scores (i.e., the shape of the shaded areas) is more similar across the four assessments in Figure 5 (for RMM) and, to a lesser degree, in Figure 7 (for TE). For example, the RMM results show the mean Grade 6 complexity scores to be similar across the four assessments (mean complexity ranges from 5.9 for QRI to 6.9 for DIBELS, all in the RMM grade-level unit) while the means are more variable in the Flesch-Kincaid results (mean complexity ranges from 4.7 for DRA to 7.6 for DIBELS, all in the Flesch-Kincaid Grade-Level unit).</p> <p>In examining within-grade equivalency of complexity, the RMM results in Figure 5 and Table 8 reveal greater similarity across the four assessments than is the case for two-factor complexity tools. Consistent with the Lexile and Flesch-Kincaid results, the RMM results show that DIBELS' average height of the shaded area across the six grades is smallest (<emph>M </emph>= 2.03; SD = 0.97) of the four assessments. However, the average within-grade variability for the other assessments is not substantially different (for example, <emph>M </emph>= 2.62; SD = 0.38 for BRI and <emph>M </emph>= 2.15; <emph>SD </emph>= 0.86 for DRA). This consistency of the within-grade equivalency can also be inferred from the smaller standard deviations for the within-grade range of complexity scores (less than 1 for all the four assessments, as evident in the bottom row in Table 8; compare this with the Flesch-Kincaid results in the bottom row in Table 7).</p> <p>Like RMM, TE's results produced fairly similar complexity score distributions across the four assessments (Figure 7). Unlike the other analytic tools, the TE analyses show limited distinctions across lower-grade texts. For example, TE gave a score of 1 (grade level) to all first-grade passages from DRA. In contrast, other analytic tools showed variability in DRA's first-grade passages, as large as 4.9 grade levels according to Flesch-Kincaid. Of the four analytical tools, TE showed little increase in mean complexity from Grades 1 to 2 (Figures 7–8). Further, the minimum complexity consistently stayed at 1 for the first several grade levels (see the flat lower edge of the shaded areas for Grades 1–3 and Grades 1–4 in Figure 7)— a pattern that was not observed by other analytical tools (See Figures 1, 3, and 5 for Lexile, Flesch-Kincaid, and RMM). These results seem to suggest that TE does not differentiate lower-level passages well.</p> <p>To examine whether the four text complexity tools consistently reveal statistically significant differences in average passage complexity at a given grade level among the four classroom assessments, we ran four separate regression models with text complexity scores from one of the four quantitative tools as the response variable in each model (see Appendix for details). The same set of predictor variables was used across the four models: grades, the assessments, and their interactions. For a given grade level, one can conduct six pairwise comparisons of mean complexity scores (e.g., BRI vs. DRA, QRI vs. DIBELS). We chose to limit our analyses to three planned comparisons using DIBELS as a reference group against which each of the remaining three assessments was compared (i.e., BRI vs. DIBELS, DRA vs. DIBELS, and QRI vs. DIBELS). We made this decision because our descriptive analyses revealed that DIBELS' complexity score distributions were distinct; that is, DIBELS passages consistently covered higher ranges of complexity across grades and revealed the least within-grade variability in complexity of the four assessments.</p> <p>Each circle in Figure 9 represents an estimated difference in mean complexity between DIBELS and one of the other assessments (i.e., BRI, DRA, or QRI) at a particular grade level. Values are typically negative, indicating that DIBELS' estimated mean complexity is higher than that of the comparison assessment. A line extending up and down from a circle is the 95% confidence interval for the mean difference estimate. When this line crosses the dashed zero line, there is no statistically significant difference in the mean complexity between DIBELS and the comparison assessment. Of the 18 comparisons conducted, Lexile revealed 11 statistically significant differences between pairs of grade-specific means (see the top panel of Figure 9 & Table A2), and Flesch-Kincaid elicited nine such cases (the second panel in Figure 9 & Table A3). In contrast, RMM and TE each revealed only four significant differences (the bottom two panels in Figure 9 & Tables A4 and A5). These results suggest that, when compared to two-factor complexity tools, the recent multidimensional tools, RMM and TE, tend to homogenize distinctions among grade-specific averages of passage complexity across the assessment systems. This finding echoes the patterns found in the descriptive analyses reported above.</p> <p>To summarize, the multi-factor complexity tools, RMM and TE, showed less pronounced cross-assessment differences than the two-factor model tools, Lexile and Flesh-Kincaid. Further, TE results suggest that the tool does not differentiate lower-level passages as well as the other three analytical tools.</p> <hd id="AN0124435026-25">Comparison against the CCSS expectations (Question 4)</hd> <p>Figures 10 and 11 compare the complexity staircases of the leveled passages from the four assessments with the CCSS expectations in Lexile and in RMM grade level respectively.[<reflink idref="bib4" id="ref121">4</reflink>] For each grade band, the top bar (in darker shade) represents the CCSS expectation, while the second bar (in lighter shade) represents the complexity of assessment passages obtained in this study. Note that the second lighter shade bar for the Grades 6–8 band in each sub-graph represents the complexity range for only Grade 6 passages from each assessment (recall that the current study examined assessment passages only through Grade 6). In contrast, the first darker shade bar represents the CCSS expectation range for Grades 6 through 8. This explains why the lighter shade bar covers only the beginning part of the darker bar or does not cover the darker bar at all.</p> <p>Graph: Figure 10. The "staircases" of text complexity as measured by Lexile: Comparisons against the CCSS Expectations (NGA, CBP, & CCSSO, [<reflink idref="bib61" id="ref122">61</reflink>]). Note. The lighter-shade bar for the Grades 6–8 band in each sub-graph represents the complexity of Grade 6 assessment passages only (recall this study examined Grades 1–6 assessment passages).</p> <p>Graph: Figure 11. The "Staircases" of Text Complexity as Measured by RMM: Comparisons against the CCSS Expectations (NGA, CBP, & CCSSO, [<reflink idref="bib61" id="ref123">61</reflink>]). See caption in Figure 10.</p> <p>Figures 10 and 11 show that DIBELS passages are most consistently aligned with the CCSS expectations in comparison to the other assessments. QRI passages cover the CCSS expectations fairly well, although its Grade 4–5 band covers substantially lower level than the expectation. BRI and DRA passages are less aligned with the CCSS expectations. According to the Lexile, BRI's Grade 6 passages and DRA's Grades 4–5 passages do not reach the lower-end of the CCSS recommendations (see Figure 10).</p> <p>The RMM results yield a portrait of more similar complexity staircases across the different assessments (see Figure 11) than the Lexile results. This pattern is similar to the pattern that was evident in the distribution of the complexity scores across the six grade levels.</p> <hd id="AN0124435026-26">Discussion</hd> <p>The present study used four analytical tools to examine the complexity of leveled passages in four classroom assessments. This investigation was motivated by four research questions. The first two questions examined (a) progressions of text difficulty across the grades and (b) the consistency of within-grade level passage complexity based on two-factor indicators of text complexity. The third question focused on whether newer, multi-factor analytical tools of text complexity yielded different information about text complexity. Finally, the fourth question examined whether the scaling of assessment passages are consistent with the CCSS expectations for text complexity, based on both two-factor and multi-factor text complexity tools.</p> <hd id="AN0124435026-27">Cross-grade progression of passage complexity</hd> <p>A general upward trajectory of complexity was observed in all assessment systems as grade levels increased. However, the Lexile Framework and Flesch-Kincaid showed differences among the four assessments in the amount of change from grade to grade and of the starting and ending points for Grades 1 to 6. Specifically, DIBELS passages represented noticeably higher levels of complexity at all six grade levels, especially compared with BRI and DRA, due mostly to its CBM logic to use the end-of-grade level passages to evaluate and monitor reading performance.</p> <p>This finding suggests that the definition of "grade-levelness" of texts differs across assessments, as measured by two widely used text analysis systems. The two excerpts of first-grade texts, one from BRI and the other from DIBELS in Table 10, differ by 350 Lexiles. In a school using the BRI (where the first-grade text is at the lower end of the range), students with borderline reading ability might do well in an oral reading task. If students moved to schools using DIBELS, whose passages are all scaled toward the end-of-grade level complexity, their oral fluency performance could be expected to fall short of the standard. Teachers and specialists who use classroom assessments for high-stakes decisions such as placement into special education services, need to be aware that different assessments employ different standards of complexity to gauge their passages even when they may be labeled as representing the same grade level.</p> <p>Table 10. Excerpts from Two Grade 1 Passages with 470L Difference.</p> <p> <ephtml> <table><thead><tr><td>At the Zoo (BRI Grade 1)</td><td>Go Fish (DIBELS Grade 1)</td></tr></thead><tbody><tr><td>Dan wanted to go to the zoo. He asked his mother. She said, "Yes." Dan had fun at the zoo. There were many animals he liked. One animal looked like it had two tails. It was an elephant. One had a nice back to ride on.</td><td>It was a cold, snowy day. Abby had invited two friends over to play the card game Go Fish. Abby's little brother, Tim, had never played and wanted to learn.</td></tr><tr><td>"I'll explain during this game," said Abby.</td></tr><tr><td>Abby showed Tim the cards in her hand, which had different numbers on them.</td></tr><tr><td>Number of words in the actual text: 100</td><td>Number of words in the actual text: 255</td></tr><tr><td>Lexile: 70L</td><td>Lexile: 540L</td></tr><tr><td>MLWF: 3.89</td><td>MLWF: 3.71</td></tr><tr><td>MSL: 5.26</td><td>MSL: 9.14</td></tr><tr><td>RMM: 0.8 (grade level)</td><td>RMM: 4.8 (grade level)</td></tr><tr><td>Flesch-Kincaid: −0.1 (grade level)</td><td>Flesch-Kincaid: 0.5 (grade level)</td></tr><tr><td>TextEvaluator: 1 (grade level)</td><td>TextEvaluator: 1 (grade level)</td></tr></tbody></table> </ephtml> </p> <p>18 <emph>Note</emph>. MLWF = Mean Log Word Frequency, MSL = Mean Sentence Length.</p> <hd id="AN0124435026-28">Within-grade variability</hd> <p>Findings of within-grade variability in passage complexity can contribute to uncertainty about students' instructional reading levels when different passages are used to compare performances across readers or the same reader over time. Even in the RMM analyses that reduced cross-assessment differences, average variability from the least to most complex passage within a grade level was around 2 grade level units for DIBELS and DRA and 2.6 for BRI and DIBELS. Such patterns leave educators uncertain as to whether differences in reading performances across time indicate valid changes in student reading levels or simply variations in text complexity.</p> <p>The findings of this study also show considerable overlap in complexity between adjacent passage levels or even across three or more levels for all of the assessments with a few exceptions (e.g., BRI's Grade 1 passages are distinctively lower than its Grade 2 passages, according to Lexile). To understand the overlaps, consider DRA's passages for Grades 3 and 4. According to TE, the third- and fourth-grade passages cover almost the same range of complexity from 1 to 3.5 grade units (see Figure 7 and Table 9). If we take out the most complex passage from each grade set, the range of complexity covered becomes from 1 to 2.6 grade units, which is exactly the range covered by DRA's Grade 2 passages. Thus, most DRA passages leveled for Grades 2 to 4 are indistinguishable in the complexity range covered, according to TE.</p> <p>A range of complexity in texts may be appropriate for instruction. After all, students within a class can vary considerably in their reading proficiencies and teachers need to accommodate students' varying needs. In assessments where passage levels serve as a reference point, such variation in text complexity at any given grade can be problematic. Considerable overlap in text complexity at neighboring levels makes it difficult to form discernable steps, each progressively more challenging, in the staircase of text complexity. Further, overlap between levels suggests that a passage might possibly be just as well assigned to two or even three adjacent levels. One possibility is that the passages in a certain grade-level group have been established to be equivalent according to some other non-quantitative criterion (e.g., presence of figurative language). If that is the case, assessment developers need to provide evidence of equivalency based on criteria other than these broad quantitative indicators (e.g., comparable conceptual difficulty or depth of linguistic processing).</p> <hd id="AN0124435026-29">Differences among text complexity tools</hd> <p>To this point in our analyses and discussion, we have evaluated the validity of the scaling of the assessments. The implicit assumption has been that the text complexity tools are valid. But the analytic frame can be reversed. That is, the leveling of the passages could be assumed to be valid and the validity of the analytic tools to produce valid, reliable indices of complexity could be questioned. From that angle, this study also uncovered similarities and differences across the text complexity tools. Differences across the four assessments according to the two-factor models of Flesch-Kincaid and Lexiles were more substantial than the differences resulting from the multi-factor tools of text complexity. For example, Lexile and Flesch-Kincaid analyses suggested that BRI's text complexity progression starts out at a substantively low level at Grade 1 relative to other assessments. The conclusion of cross-assessment differences would be modified when viewed from the perspective of the findings from the multi-factor tools, particularly RMM. Based on the RMM analyses, the BRI mean text complexity was estimated to be closer to the intended levels, although typically at the lower-end of the grade bands.</p> <p>One possible explanation for these differences may lie in how different analytical tools estimate text complexity, and particularly how they handle vocabulary or semantic difficulty. In the two-factor Lexile model, semantic difficulty is measured by the frequency of words in text. More frequent words are assumed to be easier than rare words. Similarly, the Flesch-Kincaid assumes that the number of syllables has an inverse relationship with difficulty in pronouncing and assigning meaning to words. The limitations of these two approaches to measuring vocabulary/word difficulty have been recognized (Adams, [<reflink idref="bib1" id="ref124">1</reflink>]; Stahl, [<reflink idref="bib82" id="ref125">82</reflink>]). Word frequency introduces limitations that stem from the skewed distribution of words in the English language where a small portion of words accounts for the majority of the total words in written language and the vast majority of words occur infrequently (Adams, [<reflink idref="bib1" id="ref126">1</reflink>]). The many words in the latter group receive a similar low-frequency score. When a predictor variable, in this case word frequency, takes a limited range of values, its predictability is also limited (Adams, [<reflink idref="bib1" id="ref127">1</reflink>]). Syllable count, as used in Flesch-Kincaid, is also problematic as it ignores the fact that some monosyllablic words (e.g., <emph>hue</emph>) are more difficult than some multi-syllabic words (e.g., <emph>together</emph>).</p> <p>The newer measures of text complexity provide more sophisticated measures of word difficulty. In particular, the Word Maturity variable of RMM tracks the degree to which the meaning of a word is known to typical learners at different levels of language exposure. The claim is that RMM describes a word's difficulty in relation to how its meaning changes across contexts and over time, not simply surface features such as word frequency or syllable count (Landauer et al., [<reflink idref="bib51" id="ref128">51</reflink>]). This method may have aided in evening out the distributions of text complexity across the assessment products, at least compared to two-dimensional tools.</p> <p>To establish word difficulty, TE uses over 20 word-related variables that are reduced to three principal components: Academic Vocabulary, Word Concreteness and Word Unfamiliarity. In all likelihood, the use of the multiple word-related variables and their use of human judges in estimating text complexity have contributed to the difference found in this study between TE and the more traditional measures of readability. However, our comments about RMM and TE are more in the spirit of plausible conjectures that need to be verified with empirical research. Even so, the reduction in between and within-grade variability from these multi-factor measures suggests that differences, especially for word level analyses, in estimation methods for text complexity may well be contributing to the differences observed across the analytical tools.</p> <hd id="AN0124435026-30">Comparison of CCSS expectations and grade-appropriateness of analytical tools</hd> <p>This study also uncovered differences in the degree to which the passages from different assessments were aligned with the CCSS text complexity expectations. DIBELS passages were most consistently aligned with the CCSS guidelines relative to passages from the other assessment products. When the RMM was used to scale complexity, however, the degrees of alignment with the CCSS guidelines look more alike across the four assessments.</p> <p>Our use of the CCSS staircase of text complexity for gauging the text complexity of assessment products should not be viewed as an implicit endorsement of the validity of the CCSS expectations. A debate over the appropriateness of the CCSS staircase expectations is ongoing (e.g., Gamson, Lu, & Eckert, [<reflink idref="bib28" id="ref129">28</reflink>]; Hiebert & Mesmer, [<reflink idref="bib37" id="ref130">37</reflink>]). The problems of estimating text complexity are particularly evident with texts for beginning readers, where word features such as decodability and high-frequency words are often manipulated by text designers and where repetitive syntactic structures are used to promote linguistic predictability (Hoffman, Roser, Salas, Patterson, & Pennington, [<reflink idref="bib38" id="ref131">38</reflink>]). Adding to the dilemma with K–2 texts is the fact that the analytical tools used in this study were primarily validated with Grades 3–12 texts. As shown in this study, particular analytic tools, most specifically TE, do not differentiate beginning reading texts very well. One in-progress project focuses on measuring the text complexity of early-grade texts may reverse this trend (Fitzgerald et al., [<reflink idref="bib22" id="ref132">22</reflink>]). However we note that this new measure is available for research purposes only and its efficacy has yet to be tested in the marketplace.</p> <hd id="AN0124435026-31">An alternative perspective on role of text</hd> <p>Our fundamental critique of the two types of classroom assessment tools is that they fail to provide a reasonable staircase of complexity for fulfilling their purposes—determining instructional and independent reading levels, in the case of IRIs, or measuring progress toward an ultimate performance target, in the case of CBM tasks. Each tool, as currently used, is predicated on the assumption that it possesses, at least in an ideal state, two key attributes: (a) consistent and discernable steps (risers if you will) in the staircase of complexity, progressively increasing across levels, and (b) a high degree of consistency in complexity among passages at any given level (i.e., the surface of each tread is smooth on the complexity staircase). Further, as discussed earlier, these tools are used to make decisions that assume equal intervals even though a theory- and practice-based argument may be made that such assumption is not important in the classroom assessment context. Indeed, proponents of both IRIs ad CMBs have reported that they strive to achieve both goals (expected between-grade and minimal within-grade variability) to the degree possible (Jenkins & Fuchs, [<reflink idref="bib39" id="ref133">39</reflink>]; Nilsson, [<reflink idref="bib63" id="ref134">63</reflink>]). However some proponents acknowledge the difficulty and even the low likelihood of achieving the equal intervals necessary for progression across text levels to serve as a genuine metric of growth (see Carpenter & Paris, [<reflink idref="bib11" id="ref135">11</reflink>]; Leslie & Caldwell, [<reflink idref="bib52" id="ref136">52</reflink>]). If these classroom assessments cannot meet these standards, then test users cannot easily attribute the differential performance of students on various passages to students' capacity to handle various levels of complex text. That is the essence of our argument.</p> <p>But there is another perspective on text complexity, as derived from the models of reading comprehension, developed by Kintsch ([<reflink idref="bib45" id="ref137">45</reflink>]) and championed, in elaborated form, by the RAND Reading Study Group (RAND Reading Study Group, [<reflink idref="bib72" id="ref138">72</reflink>]). This perspective holds that a text at any level can be rendered more or less difficult because of factors in the reading process in addition to text—those related to reader, task, and context. Valencia and her colleagues (Valencia, Pearson, & Wixson, [<reflink idref="bib88" id="ref139">88</reflink>]; Valencia, Wixson, & Pearson, [<reflink idref="bib89" id="ref140">89</reflink>]) have been proponents of this perspective, proposing a model called the Text-Task-Scenario (TTS). With respect to tasks, an average sixth-grade student might have more trouble unearthing the subtext of an Aesop fable judged to be at second-grade level than selecting the main idea for a tenth-grade science passage about black holes in space. Context also matters. For example, if two passages of equivalent linguistic complexity receive different instructional support in a classroom—one read independently and the second deconstructed in a small group of peers—comprehension results might vary for students across the two contexts. The picture becomes even more complex when reader factors, such as topical knowledge, linguistic sophistication, or interest are added to the mix; deep knowledge or interest in a topic might well overcome, or at least compensate for, a great deal of linguistically complex language in a text (Alexander, Kulikowich, & Jetton, [<reflink idref="bib3" id="ref141">3</reflink>]).</p> <p>As appealing as such a nuanced model is, it carries with it a substantial empirical burden: to document such a multivariate model in which all four of the key variables (reader, text, task, and sociocultural context) can vary in any number of ways would require statistical models and validation studies of massive complexity. Even so, such an undertaking would answer vexing and consequential questions about the situations in which text complexity does (and does not) wield its influence.</p> <hd id="AN0124435026-32">Conclusion</hd> <p>The current study focused on quantitative indices of text complexity, one of the three legs of complexity detailed in the CCSS definition of text complexity (National Governors Association [NGA], Center for Best Practices [CBP], & Council of Chief State School Officers [CCSSO], [<reflink idref="bib59" id="ref142">59</reflink>], Appendix A); neither the qualitative dimension nor reader-task dimension was addressed at all. Nevertheless, the study revealed that neither assessment products nor the analytical tools of text complexity are equal when it comes to estimating quantitative aspects of text difficulty. A practical implication of this study is clear: both teachers and researchers need to be aware of the immense variability among both assessments on one hand, and analytical tools of text complexity on the other, in their effort to match students to appropriate texts and support their reading development. Caution is certainly called for until and unless we have more definitive analyses of this complex array of interactions among measures of complexity and student difficulty in processing text.</p> <hd id="AN0124435026-33">Acknowledgments</hd> <p>We thank Jeff Elmore at MetaMetrics and Kathleen Sheehan at ETS, for providing the batch analysis service for the study corpus, and Angela Dancev for preparing text files for analyses.</p> <hd id="AN0124435026-34">Appendix</hd> <p></p> <hd id="AN0124435026-35">Details of regression analysis and post-hoc multiple comparisons</hd> <p>Regression analyses were conducted to examine whether there are statistically significant differences in grade-specific text complexity averages across the four classroom assessments. Four models were run, treating complexity scores from each of the four text analysis tools as a response variable, and five dummy variables representing the six grade levels, and three dummy variables representing the four assessments, and grade ×assessment interactions as explanatory variables. Specifically, text complexity for a passage was modeled as a function of grade level, assessment, and their interactions, as shown in Equation (<reflink idref="bib1" id="ref143">1</reflink>).</p> <p>(<reflink idref="bib1" id="ref144">1</reflink>)</p> <p>Graph</p> <p>In the above equation, G2<subs>i</subs> ~ G5<subs>i</subs> are dummy variables representing Grade 1 through G6 designation for a passage <emph>i</emph> made by the assessment developer. Similarly, BRI<subs>i</subs>, DRA<subs>i</subs>, and QRI<subs>i</subs> are dummy variables representing the four assessment products analyzed in this study; DIBELS was set as a reference category because the descriptive analysis revealed that its passages tended to have distinctively higher text complexity than passages from other assessments. A multiplication between a grade level and an assessment indicates an interaction between the grade level designation and the assessment.</p> <p>A coefficient for intercept () is the estimated average text complexity for DIBELS' Grade 1 passages. Regression coefficients - are the estimated average text complexity for DIBELS' passages at the remaining grade levels from Grade 2 through Grade 6 respectively. - are the estimated average text complexity for Grade 1 passages for BRI, DRA, and QRI respectively. The remaining coefficients ( - ) for the interaction terms indicate grade- and assessment-specific mean text complexity. As can be seen, this is a saturated model.</p> <p>In Model 1, Lexile scores were used for the response variable. In Model 2, Model 3 and Model 4, grade level designations by Flesch Kincaid, RMM, and TextEvaluator (TE) were used as the response variable respectively. Table A1 shows results from the four regression models.</p> <p>Based on the regression results, the following question was investigated: For a given grade level, do we see statistically significant cross-assessment differences in average text complexity?</p> <p>Three regression coefficients under "Assessment" in Table A1 partially answer this question, as the coefficients indicate the difference in average Grade 1 text complexity from DIBELS' Grade 1 average text complexity. As can be seen, Model 1 for Lexile results indicate that BRI, DRA, and QRI's average Grade 1 text complexity is significantly lower than DIBELS Grade 1 average complexity. When Flesch Kincaid and RMM were used as measures of text complexity (Models 2 and 3), it is only BRI that has significantly lower average text complexity at Grade 1 than DIBELS. TE (Model 4) did not detect any statistically significant difference in Grade 1 average text complexity against DIBELS.</p> <p>Table A2 shows results of a planned set of 18 (out of possible 276) pairwise comparisons of grade-specific means, using DIBELS as a reference group against which one of the three remaining products (BRI, QRI, and DRA) was compared. These multiple comparisons used Šidák corrections for Type I error (Šidák, [<reflink idref="bib79" id="ref145">79</reflink>]). Tables A3, B4, and B5 report results from the same analysis applied to the results from Model 2 (Flesch-Kincaid), Model 3 (RMM), and Model 4 (TE) respectively.</p> <p>As can be seen in Table A2, Lexile detected 11 statistically significant differences in grade-specific means between BRI and DIBELS at all six grades except for Grade 5 as well as between DRA and DIBELS. These are indicated in the rows with statistically significant F-statistics by an asterisk(s).</p> <p>Table A3 shows Flesch-Kincaid detected nine statistically signify differences in grade-specific means, of which three are between BRI and DIBELS at Grades 3, 4, and 6; five are between DRA and DIBELS at all six grades except for Grade 1, and one between QRI and DIBELS at Grade 2.</p> <p>Table A4 shows that RMM detected four statistically signify differences in grade-specific means, of which two were between BRI and DIBELS at Grades 2 and Grade 4; and the remaining two were between DRA and DIBELS, again at Grade 2 and Grade 4.</p> <p>Lastly, Table A5 shows that TextEvaluator detected four statistically significant differences in specific means, two of which are between BRI and DIBELS at Grade 3 and Grade 6; and the remaining two are between DRA and DIBELS at Grade 3 and Grade 4.</p> <p>These multiple comparisons of grade-specific means against those of DIBELS indicate that older, two-factor model measures of text complexity—Lexile and Flesch-Kincaid—detected a greater number of statistically significant differences in grade-specific means, mostly between BRI and DIBELS as well as DRA and DIBELS (Lexile detected 11 such differences and Flesch-Kincaid found 9). In contrast, RMM and TextEvaluator, newer, multi-dimensional measures of text complexity, fewer instances of such differences (both measures found four instances of statistically signify differences each). These findings are visualized in Figure 9.</p> <p>Table A1. Regression Results.</p> <p> <ephtml> <table><thead><tr><td /><td>Model 1</td><td>Model 2</td><td>Model 3</td><td>Model 4</td></tr><tr><td>Predictors</td><td>[Lexile]</td><td>[FKrade]</td><td>[RMM]</td><td>[TE]</td></tr><tr><td>Assessment</td><td>Coefficient</td><td>(SE)</td><td>Coefficient</td><td>(SE)</td><td>Coefficient</td><td>(SE)</td><td>Coefficient</td><td>(SE)</td></tr></thead><tbody><tr><td>BRI</td><td>−393.33***</td><td>(54.79)</td><td>−1.73**</td><td>(0.59)</td><td>−1.67**</td><td>(0.60)</td><td>−0.07</td><td>(0.27)</td></tr><tr><td>DRA</td><td>−141.33***</td><td>(41.37</td><td>0.10</td><td>(0.57)</td><td>−0.78</td><td>(0.53)</td><td>−0.27</td><td>(0.18)</td></tr><tr><td>QRI</td><td>−160.83*</td><td>(63.04)</td><td>−0.38</td><td>(0.42)</td><td>−0.38</td><td>(0.55)</td><td>0.11</td><td>(0.30)</td></tr><tr><td>Grade</td><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td> 2</td><td>38.889</td><td>(38.44)</td><td>1.87***</td><td>(0.54)</td><td>0.64</td><td>(0.47)</td><td>0.59</td><td>(0.41)</td></tr><tr><td> 3</td><td>228.89***</td><td>(38.74)</td><td>3.12***</td><td>(0.46)</td><td>1.86**</td><td>(0.56)</td><td>3.20***</td><td>(0.37)</td></tr><tr><td> 4</td><td>336.67***</td><td>(40.28)</td><td>4.09***</td><td>(0.53)</td><td>2.59***</td><td>(0.50)</td><td>3.29***</td><td>(0.54)</td></tr><tr><td> 5</td><td>362.22***</td><td>(37.89)</td><td>5.30***</td><td>(0.47)</td><td>3.52***</td><td>(0.52)</td><td>3.90***</td><td>(0.36)</td></tr><tr><td> 6</td><td>458.89***</td><td>(41.44)</td><td>6.18***</td><td>(0.47)</td><td>3.88***</td><td>(0.46)</td><td>4.83***</td><td>(0.37)</td></tr><tr><td>Assessment x Grade</td><td /><td /><td /><td /><td /><td /><td /></tr><tr><td> BRI x 2</td><td>269.11***</td><td>(62.46)</td><td>0.05</td><td>(0.95)</td><td>0.24</td><td>(0.72)</td><td>−0.49</td><td>(0.49)</td></tr><tr><td> BRI x 3</td><td>188.25**</td><td>(66.34)</td><td>−0.60</td><td>(0.74)</td><td>0.33</td><td>(0.77)</td><td>−2.41***</td><td>(0.55)</td></tr><tr><td> BRI x 4</td><td>79.05</td><td>(75.69)</td><td>−0.72</td><td>(0.92)</td><td>−0.01</td><td>(0.77)</td><td>−2.05*</td><td>(0.81)</td></tr><tr><td> BRI x 5</td><td>227.78*</td><td>(88.62)</td><td>0.56</td><td>(0.98)</td><td>0.76</td><td>(0.76)</td><td>−0.86</td><td>(0.63)</td></tr><tr><td> BRI x 6</td><td>192.78**</td><td>(69.20)</td><td>−0.69</td><td>(0.91)</td><td>0.95</td><td>(0.74)</td><td>−1.90**</td><td>(0.68)</td></tr><tr><td> DRA2 x 2</td><td>22.68</td><td>(52.22)</td><td>−1.55*</td><td>(0.73)</td><td>−0.16</td><td>(0.59)</td><td>−0.21</td><td>(0.44)</td></tr><tr><td> DRA2 x 3</td><td>−85.06</td><td>(54.03)</td><td>−2.19**</td><td>(0.67)</td><td>0.04</td><td>(0.64)</td><td>−2.18***</td><td>(0.45)</td></tr><tr><td> DRA2 x 4</td><td>−103.66</td><td>(56.42)</td><td>−2.67**</td><td>(0.84)</td><td>−0.49</td><td>(0.65)</td><td>−1.96**</td><td>(0.72)</td></tr><tr><td> DRA2 x 5</td><td>−89.22</td><td>(45.41)</td><td>−3.71***</td><td>(0.71)</td><td>−0.22</td><td>(0.67)</td><td>−0.25</td><td>(0.61)</td></tr><tr><td> DRA2 x 6</td><td>−58.39</td><td>(52.58)</td><td>−2.98***</td><td>(0.68)</td><td>0.50</td><td>(0.82)</td><td>−0.56</td><td>(0.78)</td></tr><tr><td> QRI x 2</td><td>80.61</td><td>(71.40)</td><td>−0.98</td><td>(0.59)</td><td>−0.47</td><td>(0.69)</td><td>0.10</td><td>(0.72)</td></tr><tr><td> QRI x 3</td><td>110.61</td><td>(67.19)</td><td>−0.29</td><td>(0.52)</td><td>0.15</td><td>(0.72)</td><td>−1.56*</td><td>(0.76)</td></tr><tr><td> QRI x 4</td><td>20.83</td><td>(92.75)</td><td>−0.51</td><td>(0.71)</td><td>−0.74</td><td>(0.82)</td><td>−1.16</td><td>(0.71)</td></tr><tr><td> QRI x 5</td><td>55.28</td><td>(82.62)</td><td>−0.47</td><td>(0.77)</td><td>−0.47</td><td>(0.72)</td><td>−0.94</td><td>(0.76)</td></tr><tr><td> QRI x 6</td><td>48.61</td><td>(77.50)</td><td>−0.29</td><td>(0.57)</td><td>−0.30</td><td>(0.70)</td><td>−0.71</td><td>(0.47)</td></tr><tr><td>Constant</td><td>553.33***</td><td>(35.81)</td><td>1.43***</td><td>(0.40)</td><td>2.93***</td><td>(0.46)</td><td>1.27***</td><td>(0.18)</td></tr><tr><td> R<sup>2</sup></td><td>0.85</td><td>0.82</td><td>0.79</td><td>0.76</td></tr></tbody></table> </ephtml> *<emph>p</emph> < 0.05. **<emph>p</emph> < 0.01. ***<emph>p</emph> < 0.001. <emph>Note</emph>. A bracket [] indicates the measurement for the response variable for each model. DIBELS and grade 1 were set as reference groups for the categorical predictors respectively.</p> <p>Table A2. Comparisons of Average Complexity (in Lexile) Against DIBELS.</p> <p> <ephtml> <table><thead><tr><td>Comparisons</td><td>F-stat</td><td>Difference in Mean Complexity (95% CI)</td></tr><tr><td>Grade</td><td>DIBELS vs.</td></tr></thead><tbody><tr><td>1</td><td>BRI</td><td>51.54***</td><td>−393.3</td><td>(−559.7 </td><td>–227.0)</td></tr><tr><td>2</td><td>BRI</td><td>17.15**</td><td>−124.2</td><td>(−215.3 </td><td>–33.1)</td></tr><tr><td>3</td><td>BRI</td><td>30.06***</td><td>−205.1</td><td>(−318.7 </td><td>–91.5)</td></tr><tr><td>4</td><td>BRI</td><td>36.21***</td><td>−314.3</td><td>(−472.9 </td><td>–155.7)</td></tr><tr><td>5</td><td>BRI</td><td>5.65</td><td>−165.6</td><td>(−377.1 </td><td>46.0)</td></tr><tr><td>6</td><td>BRI</td><td>22.51***</td><td>−200.6</td><td>(−328.9 </td><td>–72.2)</td></tr><tr><td>1</td><td>DRA</td><td>11.67*</td><td>−141.3</td><td>(−267.0 </td><td>–15.7)</td></tr><tr><td>2</td><td>DRA</td><td>13.87**</td><td>−118.7</td><td>(−215.4 </td><td>–21.9)</td></tr><tr><td>3</td><td>DRA</td><td>42.46***</td><td>−226.4</td><td>(−331.9 </td><td>–120.9)</td></tr><tr><td>4</td><td>DRA</td><td>40.79***</td><td>−245.0</td><td>(−361.5 </td><td>–128.5)</td></tr><tr><td>5</td><td>DRA</td><td>151.78***</td><td>−230.6</td><td>(−287.4 </td><td>–173.7)</td></tr><tr><td>6</td><td>DRA</td><td>371.87***</td><td>−199.7</td><td>(−298.3 </td><td>–101.2)</td></tr><tr><td>1</td><td>QRI</td><td>6.51</td><td>−160.8</td><td>(−352.2 </td><td>30.6)</td></tr><tr><td>2</td><td>QRI</td><td>5.73</td><td>−80.2</td><td>(−182.0 </td><td>21.6)</td></tr><tr><td>3</td><td>QRI</td><td>4.67</td><td>−50.2</td><td>(−120.8 </td><td>20.3)</td></tr><tr><td>4</td><td>QRI</td><td>4.23</td><td>−140.0</td><td>(−346.6 </td><td>66.6)</td></tr><tr><td>5</td><td>QRI</td><td>3.91</td><td>−105.6</td><td>(−267.7 </td><td>56.6)</td></tr><tr><td>6</td><td>QRI</td><td>6.20</td><td>−112.2</td><td>(−249.1 </td><td>24.6)</td></tr></tbody></table> </ephtml> *<emph>p</emph> <.05. **<emph>p</emph> <.01. ***<emph>p</emph> <.001. <emph>Note</emph>. Sidak multiple comparison corrections are used for the calculations of <emph>p</emph>-values.</p> <p>Table A3. Comparisons of Average Complexity (in FK Grade Level) against DIBELS.</p> <p> <ephtml> <table><thead><tr><td>Comparisons</td><td>F-stat</td><td>Difference in Mean Complexity (95% CI)</td></tr><tr><td>Grade</td><td>DIBELS vs.</td></tr></thead><tbody><tr><td>1</td><td>BRI</td><td>8.75</td><td>−1.73</td><td>(−3.52</td><td>0.05)</td></tr><tr><td>2</td><td>BRI</td><td>5.13</td><td>−1.68</td><td>(−3.94</td><td>0.58)</td></tr><tr><td>3</td><td>BRI</td><td>25.65***</td><td>−2.33</td><td>(−3.73</td><td>−0.93)</td></tr><tr><td>4</td><td>BRI</td><td>11.91*</td><td>−2.45</td><td>(−4.61</td><td>−0.29)</td></tr><tr><td>5</td><td>BRI</td><td>2.22</td><td>−1.17</td><td>(−3.57</td><td>1.22)</td></tr><tr><td>6</td><td>BRI</td><td>12.15*</td><td>−2.43</td><td>(−4.55</td><td>−0.31)</td></tr><tr><td>1</td><td>DRA</td><td>0.03</td><td>0.10</td><td>(−1.65</td><td>1.84)</td></tr><tr><td>2</td><td>DRA</td><td>9.98*</td><td>−1.45</td><td>(−2.85</td><td>−0.05)</td></tr><tr><td>3</td><td>DRA</td><td>35.53***</td><td>−2.09</td><td>(−3.16</td><td>−1.02)</td></tr><tr><td>4</td><td>DRA</td><td>17.29***</td><td>−2.57</td><td>(−4.46</td><td>−0.69)</td></tr><tr><td>5</td><td>DRA</td><td>74.55***</td><td>−3.61</td><td>(−4.88</td><td>−2.34)</td></tr><tr><td>6</td><td>DRA</td><td>64.15***</td><td>−2.89</td><td>(−3.98</td><td>−1.79)</td></tr><tr><td>1</td><td>QRI</td><td>0.81</td><td>−0.38</td><td>(−1.68</td><td>0.91)</td></tr><tr><td>2</td><td>QRI</td><td>11.36*</td><td>−1.36</td><td>(−2.59</td><td>−0.13)</td></tr><tr><td>3</td><td>QRI</td><td>4.98</td><td>−0.68</td><td>(−1.60</td><td>0.25)</td></tr><tr><td>4</td><td>QRI</td><td>2.43</td><td>−0.89</td><td>(−2.63</td><td>0.85)</td></tr><tr><td>5</td><td>QRI</td><td>1.73</td><td>−0.85</td><td>(−2.82</td><td>1.12)</td></tr><tr><td>6</td><td>QRI</td><td>3.22</td><td>−0.67</td><td>(−1.80</td><td>0.47)</td></tr></tbody></table> </ephtml> *<emph>p</emph> <.05. **<emph>p</emph> <.01. ***<emph>p</emph> <.001. <emph>Note</emph>. Sidak multiple comparison corrections are used for the calculations of <emph>p</emph>-values.</p> <p>Table A4. Comparisons of Average Complexity (in RMM Grade Level) Against DIBELS.</p> <p> <ephtml> <table><thead><tr><td>Comparisons</td><td>F-stat</td><td>Difference in Mean Complexity (95% CI)</td></tr><tr><td>Grade</td><td>DIBELS vs.</td></tr></thead><tbody><tr><td>1</td><td>BRI</td><td>7.81</td><td>−1.67</td><td>(−3.50</td><td>0.15)</td></tr><tr><td>2</td><td>BRI</td><td>12.54**</td><td>−1.44</td><td>(−2.67</td><td>−0.20)</td></tr><tr><td>3</td><td>BRI</td><td>7.59</td><td>−1.35</td><td>(−2.83</td><td>0.14)</td></tr><tr><td>4</td><td>BRI</td><td>12.39*</td><td>−1.68</td><td>(−3.13</td><td>−0.23)</td></tr><tr><td>5</td><td>BRI</td><td>3.82</td><td>−0.92</td><td>(−2.34</td><td>0.51)</td></tr><tr><td>6</td><td>BRI</td><td>2.87</td><td>−0.73</td><td>(−2.04</td><td>0.58)</td></tr><tr><td>1</td><td>DRA</td><td>2.16</td><td>−0.78</td><td>(−2.41</td><td>0.84)</td></tr><tr><td>2</td><td>DRA</td><td>13.15**</td><td>−0.94</td><td>(−1.73</td><td>−0.15)</td></tr><tr><td>3</td><td>DRA</td><td>4.5</td><td>−0.74</td><td>(−1.80</td><td>0.32)</td></tr><tr><td>4</td><td>DRA</td><td>11.91*</td><td>−1.27</td><td>(−2.40</td><td>−0.15)</td></tr><tr><td>5</td><td>DRA</td><td>6.34</td><td>−1.01</td><td>(−2.22</td><td>0.21)</td></tr><tr><td>6</td><td>DRA</td><td>0.21</td><td>−0.29</td><td>(−2.19</td><td>1.62)</td></tr><tr><td>1</td><td>QRI</td><td>0.49</td><td>−0.38</td><td>(−2.05</td><td>1.28)</td></tr><tr><td>2</td><td>QRI</td><td>4.08</td><td>−0.86</td><td>(−2.15</td><td>0.44)</td></tr><tr><td>3</td><td>QRI</td><td>0.24</td><td>−0.23</td><td>(−1.65</td><td>1.19)</td></tr><tr><td>4</td><td>QRI</td><td>3.3</td><td>−1.12</td><td>(−3.00</td><td>0.76)</td></tr><tr><td>5</td><td>QRI</td><td>3.37</td><td>−0.86</td><td>(−2.27</td><td>0.56)</td></tr><tr><td>6</td><td>QRI</td><td>2.46</td><td>−0.68</td><td>(−2.01</td><td>0.64)</td></tr></tbody></table> </ephtml> *<emph>p</emph> <.05. **<emph>p</emph> <.01. ***<emph>p</emph> <.001. <emph>Note</emph>. Sidak multiple comparison corrections are used for the calculations of <emph>p</emph>-values.</p> <p>Table A5. Comparisons of Average Complexity (in Text Evaluator Grade Level) Against DIBELS.</p> <p> <ephtml> <table><thead><tr><td>Comparisons</td><td>F-stat</td><td>Difference in Mean Complexity (95% CI)</td></tr><tr><td>Grade</td><td>DIBELS vs.</td></tr></thead><tbody><tr><td>1</td><td>BRI</td><td>0.06</td><td>−0.07</td><td>(−0.88</td><td>0.74)</td></tr><tr><td>2</td><td>BRI</td><td>1.78</td><td>−0.56</td><td>(−1.82</td><td>0.71)</td></tr><tr><td>3</td><td>BRI</td><td>27.07***</td><td>−2.48</td><td>(−3.93</td><td>−1.03)</td></tr><tr><td>4</td><td>BRI</td><td>7.58</td><td>−2.11</td><td>(−4.45</td><td>0.22)</td></tr><tr><td>5</td><td>BRI</td><td>2.65</td><td>−0.93</td><td>(−2.66</td><td>0.81)</td></tr><tr><td>6</td><td>BRI</td><td>9.81*</td><td>−1.97</td><td>(−3.88</td><td>−0.06)</td></tr><tr><td>1</td><td>DRA</td><td>2.12</td><td>−0.27</td><td>(−0.82</td><td>0.29)</td></tr><tr><td>2</td><td>DRA</td><td>1.41</td><td>−0.48</td><td>(−1.70</td><td>0.74)</td></tr><tr><td>3</td><td>DRA</td><td>34.95***</td><td>−2.44</td><td>(−3.70</td><td>−1.18)</td></tr><tr><td>4</td><td>DRA</td><td>10.3*</td><td>−2.23</td><td>(−4.35</td><td>−0.12)</td></tr><tr><td>5</td><td>DRA</td><td>0.8</td><td>−0.52</td><td>(−2.28</td><td>1.24)</td></tr><tr><td>6</td><td>DRA</td><td>1.17</td><td>−0.83</td><td>(−3.14</td><td>1.49)</td></tr><tr><td>1</td><td>QRI</td><td>0.13</td><td>0.11</td><td>(−0.82</td><td>1.03)</td></tr><tr><td>2</td><td>QRI</td><td>0.1</td><td>0.20</td><td>(−1.78</td><td>2.19)</td></tr><tr><td>3</td><td>QRI</td><td>4.26</td><td>−1.45</td><td>(−3.58</td><td>0.69)</td></tr><tr><td>4</td><td>QRI</td><td>2.74</td><td>−1.06</td><td>(−3.00</td><td>0.88)</td></tr><tr><td>5</td><td>QRI</td><td>1.45</td><td>−0.83</td><td>(−2.94</td><td>1.28)</td></tr><tr><td>6</td><td>QRI</td><td>2.81</td><td>−0.60</td><td>(−1.70</td><td>0.49)</td></tr></tbody></table> </ephtml> *<emph>p</emph> <.05. **<emph>p</emph> <.01. ***<emph>p</emph> <.001. <emph>Note</emph>. Sidak multiple comparison corrections are used for the calculations of <emph>p</emph>-values.</p> <ref id="AN0124435026-36"> <title> References </title> <blist> <bibl id="bib1" idref="ref14" type="bt">1</bibl> <bibtext> Adams, N. J. (2001). On the lexile framework. In National Center for Education Statistics (Ed.), Assessing the lexile framework. Results on a panel meeting NCES 2001-08 (pp. 15–21). Washington, DC: National Center for Education Statistics.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref22" type="bt">2</bibl> <bibtext> Albee, J. J., Arnold, J. M., Dennis, L., Schafer, B. J., & Olson, S. (2013). Reading assessments for screening/placement, diagnosis, and summative/outcomes: What are schools using? The Reading Professor, 35(1), 25–34.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref82" type="bt">3</bibl> <bibtext> Alexander, P. A., Kulikowich, J. M., & Jetton, T. L. (1994). The role of subject-matter knowledge and interest in the processing of linear and nonlinear texts. Review of Educational Research, 64(2), 201–252. doi:10.3102/00346543064002201</bibtext> </blist> <blist> <bibl id="bib4" idref="ref1" type="bt">4</bibl> <bibtext> Anderson, A., Schlueter, J. E., Carlson, J. F., & Geisinger, K. F. (2016). Tests in print IX: An index to tests, test reviews, and the literature on specific tests. Lincoln, NE: University of Nebraska Press.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref46" type="bt">5</bibl> <bibtext> Ardoin, S. P., Suldo, S. M., Witt, J., Aldrich, S., & Mcdonald, E. (2005). Accuracy of readability estimates' predictions of CBM p erformance. School Psychology Quarterly, 20(1), 1–22. doi:10.1521/scpq.20.1.1.64193</bibtext> </blist> <blist> <bibl id="bib6" idref="ref18" type="bt">6</bibl> <bibtext> Arthaud, J., Vasa, S. F., & Steckelberg, A. L. (2000). Reading assessment and instructional practices in special education. Assessment for Effective Intervention, 25(3), 205–227. doi:10.1177/073724770002500302</bibtext> </blist> <blist> <bibl id="bib7" idref="ref2" type="bt">7</bibl> <bibtext> Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher, 36(5), 258–267. doi:10.3102/0013189X07306523</bibtext> </blist> <blist> <bibl id="bib8" idref="ref88" type="bt">8</bibl> <bibtext> Beaver, J. M., & Carter, M. A. (2006). The developmental reading assessment–second edition (DRA2). Upper Saddle River, NJ: Pearson.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref72" type="bt">9</bibl> <bibtext> Briggs, D. C. (2013). Measuring growth with vertical scales. Journal of Educational Measurement, 50(2), 204–226. doi:10.1111/jedm.2013.50.issue-2</bibtext> </blist> <blist> <bibtext> Brookhart, S. M. (2003). Developing measurement theory for classroom assessment purposes and uses. Educational Measurement, Issues and Practices, 22, 5–12. doi:10.1111/j.1745-3992.2003.tb00139.x</bibtext> </blist> <blist> <bibtext> Carpenter, R. D., & Paris, S. G. (2005). Issues of validity and reliability in early reading assessments. In S. G. Paris & A. Stahl (Eds.), Children's reading comprehension and assessment (pp. 279–304). Mahwah, NJ: Lawrence Erlbaum.</bibtext> </blist> <blist> <bibtext> Christ, T. J., & Ardoin, S. P. (2009). Curriculum-based measurement of oral reading: Passage equivalence and probe-set development. Journal of School Psychology, 47(1), 55–75. doi:10.1016/j.jsp.2008.09.004</bibtext> </blist> <blist> <bibtext> Christ, T. J., Monaghen, B. D., Zopluoglu, C., & Van Norman, E. R. (2013). Curriculum-based measurement of oral reading evaluation of growth estimates derived with pre–post assessment methods. Assessment for Effective Intervention, 38(3), 139–153. doi:10.1177/1534508412456417</bibtext> </blist> <blist> <bibtext> Clay, M. M. (1993). An observation survey of early literacy achievement. Portsmouth, NH: Heinemann.</bibtext> </blist> <blist> <bibtext> Clay, M. M. (1994). Reading Recovery: A guidebook for teachers in training. Portsmouth, NH: Heinemann.</bibtext> </blist> <blist> <bibtext> Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58(4), 438–481. doi:10.3102/00346543058004438</bibtext> </blist> <blist> <bibtext> Cummings, K., Wallin, J., Good, R., & Kaminski, R. (2007). The DMG passage difficulty index. [Formula and computer software]. Eugene, OR: Dynamic Measurement Group.</bibtext> </blist> <blist> <bibtext> Deeney, T. A., & Shim, M. K. (2016). Teachers' and students' views of reading fluency: Issues of consequential validity in adopting one-minute reading fluency assessments. Assessment for Effective Intervention, 41(2), 1–18.</bibtext> </blist> <blist> <bibtext> Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52(3), 219–232.</bibtext> </blist> <blist> <bibtext> DIBELS Data System. (2015). UO DIBELS data system. Retrieved from https://dibels.uoregon.edu/</bibtext> </blist> <blist> <bibtext> Domingue, B. (2014). Evaluating the equal-interval hypothesis with test score scales. Psychometrika, 79(1), 1–19. doi:10.1007/s11336-013-9342-4</bibtext> </blist> <blist> <bibtext> Fitzgerald, J., Elmore, J., Koons, H., Hiebert, E. H., Bowen, K., & Stenner, A. J. (2014). Important text characteristics for early-grades text complexity. Journal of Educational Psychology, 106(3), 4–29.</bibtext> </blist> <blist> <bibtext> Ford, M. P., & Opitz, M. F. (2008). A national survey of guided reading practices: What we can learn from primary teachers. Literacy Research and Instruction, 47, 309–331. doi:10.1080/19388070802332895</bibtext> </blist> <blist> <bibtext> Fountas, I. C., & Pinnell, G. S. (1996). Guided reading: Good first teaching for all children. Portsmouth, NH: Heinemann.</bibtext> </blist> <blist> <bibtext> Fountas, I. C., & Pinnell, G. S. (2012). Guided reading: The romance and the reality. The Reading Teacher, 66(4), 268–284. doi:10.1002/trtr.2012.66.issue-4</bibtext> </blist> <blist> <bibtext> Francis, D. J., Santi, K. L., Barr, C., Fletcher, J. M., Varisco, A., & Foorman, B. R. (2008). Form effects on the estimation of students' oral reading fluency using DIBELS. Journal of School Psychology, 46(3), 315–342. doi:10.1016/j.jsp.2007.06.003</bibtext> </blist> <blist> <bibtext> Fry, E. (1977). Fry's readability graph: Clarification, validity, and extension to level 17. Journal of Reading, 21, 242–252.</bibtext> </blist> <blist> <bibtext> Gamson, D., Lu, X., & Eckert, S. (2013). Challenging the research base of the common core state standards A historical reanalysis of text complexity. Educational Researcher, 42(7), 381–391. doi:10.3102/0013189X13505684</bibtext> </blist> <blist> <bibtext> Goffreda, C. T., & DiPerna, J. C. (2010). An empirical review of psychometric evidence for the dynamic indicators of basic early literacy skills. School Psychology Review, 39(3), 463–483.</bibtext> </blist> <blist> <bibtext> Good, R. H. I., Kaminski, R. A., Dewey, E. N., Wallin, J., Powell-Smith, K. A., & Latimer, R. J. (2013). Dynamic indicators of basic early literacy skills: DIBELS Next technical manual. Retrieved from https://dibels.org/</bibtext> </blist> <blist> <bibtext> Goodman, K. (Ed.). (2006). The truth about DIBELS. Portsmouth, NH: Heinemann.</bibtext> </blist> <blist> <bibtext> Goodman, Y. M., & Burke, C. L. (1972). Reading miscue inventory. New York, NY: Robert C. Owen.</bibtext> </blist> <blist> <bibtext> Graesser, A. C., McNamara, D. S., Cai, Z., Conley, M., Li, H., & Pennebaker, J. (2014). Coh-Metrix measures text characteristics oat multiple levels of language and discourse. The Elementary School Journal, 115(2), 210–229. doi:10.1086/678293</bibtext> </blist> <blist> <bibtext> Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-metrix: Providing multilevel analyses of text characteristics. Educational Researcher, 40(5), 223–234. doi:10.3102/0013189X11413260</bibtext> </blist> <blist> <bibtext> Hall, S. L. (2006). I've DIBEL'd, now what? Designing interventions with DIBELS Data. Longmont, CO: Sopris West Educational Services.</bibtext> </blist> <blist> <bibtext> Heilman, M., Collins-Thompson, K., Callan, J., & Eskenazi, M. (2006). Classroom success of an intelligent tutoring system for lexical practice and reading comprehension. Proceedings of the Ninth International Conference on Spoken Language Processing, Pittsburgh, PA.</bibtext> </blist> <blist> <bibtext> Hiebert, E. H., & Mesmer, H. A. E. (2013). Upping the ante of text complexity in the Common Core State Standards: Examining its potential impact on young readers. Educational Researcher, 42(1), 44–51. doi:10.3102/0013189X12459802</bibtext> </blist> <blist> <bibtext> Hoffman, J., Roser, N., Salas, R., Patterson, E., & Pennington, J. (2001). Text leveling and "little books" in first-grade reading. Journal of Literacy Research, 33(3), 507–528. doi:10.1080/10862960109548121</bibtext> </blist> <blist> <bibtext> Jenkins, J. R., & Fuchs, L. S. (2012). Curriculum-based measurement: The paradigm, history, and legacy. In C. A. Epsin, K. L. McMaster, S. Rose, & M. M. Wayman (Eds.), A measure of success: The influence of curriculum-based measurement on education (pp. 7–23). Minneapolis, MN: University of Minnesota Press.</bibtext> </blist> <blist> <bibtext> Jenkins, J. R., Zumeta, R., Dupree, O., & Johnson, K. (2005). Measuring gains in reading ability with passage reading fluency. Learning Disabilities Research & Practice, 20(4), 245–253. doi:10.1111/j.1540-5826.2005.00140.x</bibtext> </blist> <blist> <bibtext> Johns, J. L. (2012). Basic reading inventory: Pre-primer through grade twelve and early literacy assessments. Dubuque, IA: Kendall Hunt.</bibtext> </blist> <blist> <bibtext> Kame'enui, E. J., Fuchs, L., Francis, D. J., Good III, R., O'Connor, R. E., Simmons, D. C., ... Torgensen, J. K. (2006). The adequacy of tools for assessing reading competence: A framework and review. Educational Researcher, 35(4), 3–11. doi:10.3102/0013189X035004003</bibtext> </blist> <blist> <bibtext> Kaminski, R. A., Good, R. H., Baker, D., Cummings, K., Dufour-Martel, C., Fleming, K., ... Wallin, J. (2007). Position paper on use of DIBELS for system-wide accountability decisions. Eugene, OR: Dynamic Measurement Group. Retrieved from https://dibels.org/papers/Appropriateness_0207.pdf</bibtext> </blist> <blist> <bibtext> Kincaid, J. P., Fishburne, L. R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Research branch report (pp. 8–75). Memphis, TN: Chief of Naval Technical Training: Naval Air Station.</bibtext> </blist> <blist> <bibtext> Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY: Cambridge University Press.</bibtext> </blist> <blist> <bibtext> Klare, G. R. (1984). Readability. In P. D. Pearson, R. Barr, & M. L. Kamil (Eds.), Handbook of Reading Research (pp. 681–744). New York, NY: Longman.</bibtext> </blist> <blist> <bibtext> Klesius, J. P., & Homan, S. P. (1985). A validity and reliability update on the informal reading inventory with suggestions for improvement. Journal of Learning Disabilities, 18, 71–76. doi:10.1177/002221948501800202</bibtext> </blist> <blist> <bibtext> Kontovourki, S. (2012). Reading leveled books in assessment-saturated classrooms : A close examination of unmarked processes. Reading Research Quarterly, 47(2), 153–171. doi:10.1002/RRQ.014</bibtext> </blist> <blist> <bibtext> Koslin, B. L., Zeno, S., & Koslin, S. (1987). The DRP: An effective measure in reading. New York, NY: College Entrance Examination Board.</bibtext> </blist> <blist> <bibtext> Kuhn, M. R., Schwanenflugel, P. J., & Meisinger, E. B. (2010). Aligning theory and assessment of reading fluency: Automaticity. Prosody, and Definitions of Fluency. Reading Research Quarterly, 45(2), 230–251. doi:10.1598/RRQ.45.2.4</bibtext> </blist> <blist> <bibtext> Landauer, T. K., Kireyev, K., & Panaccione, C. (2011). Word maturity: A new metric for word knowledge. Scientific Studies of Reading, 15(1), 92–108. doi:10.1080/10888438.2011.536130</bibtext> </blist> <blist> <bibtext> Leslie, L., & Caldwell, J. (2009). Formal and informal measures of reading comprehensiontion. In S. E. Israel & G. G. Duffy (Eds.), Handbook of research on reading comprehension (pp. 403–427). New York, NY: Routledge.</bibtext> </blist> <blist> <bibtext> Leslie, L., & Caldwell, J. S. (2010). Qualitative reading inventory (5th ed.). Upper Saddle River, NJ: Pearson.</bibtext> </blist> <blist> <bibtext> Markus, K., & Borsboom, D. (2013). Frontiers of test validity theory: Measurement, causation, and meaning. New York, NY: Routledge.</bibtext> </blist> <blist> <bibtext> Mellard, D. F., McKnight, M., & Woods, K. (2009). Learning disabilities practice response to intervention screening and progress-monitoring practices in 41 local schools. Learning Disabilities Research & Practice, 24(4), 186–195. doi:10.1111/j.1540-5826.2009.00292.x</bibtext> </blist> <blist> <bibtext> Mesmer, H. A. (2007). Tools for matching readers to texts. New York, NY: The Guilford Press.</bibtext> </blist> <blist> <bibtext> Mesmer, H. A., Cunningham, J. W., & Hiebert, E. H. (2012). Toward a theoretical model of text complexity for the early grades: Learning from the past, anticipating the future. Reading Research Quarterly, 47(3), 235–258. doi:10.1002/RRQ.019</bibtext> </blist> <blist> <bibtext> Milone, M. (2008). The development of ATOS: The Renaissance readability formula. Wisconsin Rapids, WI: Renaissance Learning, Inc.</bibtext> </blist> <blist> <bibtext> National Governors Association [NGA], Center for Best Practices [CBP], & Council of Chief State School Officers [CCSSO]. (2010). Common core state standards for English language arts & literacy in history/social studies, science, and technical subjects with appendices A–C. Retrieved from <ulink href="http://www.corestandards.org/">http://www.corestandards.org/</ulink></bibtext> </blist> <blist> <bibtext> Nelson, J., Perfetti, C., Liben, D., & Liben, M. (2012). Measures of text difficulty: Testing their predictive value of grade levels and student performance. Report to the Gates Foundation. Retrieved from <ulink href="http://www.ccsso.org/Documents/2012/Measures">http://www.ccsso.org/Documents/2012/Measures</ulink> ofText</bibtext> </blist> <blist> <bibtext> NGA, CBP, & CCSSO. (2012). Supplemental information for Appendix A of the common core state standards for English language arts and literacy: New research on text complexity. Washington, DC: Author. Retrieved from Common Core State Standards Initiative website <ulink href="http://www.corestandards.org/resources">http://www.corestandards.org/resources</ulink></bibtext> </blist> <blist> <bibtext> Nilsson, N. L. (2008). A critical analysis of eight informal reading inventories. The Reading Teacher, 61(7), 526–536. doi:10.1598/RT.61.7.2</bibtext> </blist> <blist> <bibtext> Nilsson, N. L. (2013). Introduction to using informal reading inventories in research and practice. Reading & Writing Quarterly, 29(3), 203–207. doi:10.1080/10573569.2013.789778</bibtext> </blist> <blist> <bibtext> O'Neill, J. (2006, March 12). Leaving creativity behind: Drilling for tests kills curiosity and imagination. San Francisco Chronicle. Retrieved from <ulink href="http://www.sfgate.com">http://www.sfgate.com</ulink></bibtext> </blist> <blist> <bibtext> Paris, S. G. (2002). Measuring children's reading development using leveled texts. The Reading Teacher, 56(2), 168–170.</bibtext> </blist> <blist> <bibtext> Paris, S. G., & Carpenter, R. D. (2003). FAQs about IRIs. The Reading Teacher, 56(6), 578–580.</bibtext> </blist> <blist> <bibtext> Parker, D. C., Zaslofsky, A. F., Burns, M. K., Kanive, R., Hodgson, J., Scholin, S. E., & Klingbeil, D. A. (2015). A brief report of the diagnostic accuracy of oral reading fluency and reading inventory levels for reading failure risk among second- and third- grade students. Reading & Writing Quarterly, 31, 56–67. doi:10.1080/10573569.2013.857970</bibtext> </blist> <blist> <bibtext> Pearson Education, Inc. (2011). DRA K – 8 technical manual: Developmental reading assessment (2nd ed.). Retrieved from <ulink href="http://goodrichschools.org/view/2244.pdf">http://goodrichschools.org/view/2244.pdf</ulink></bibtext> </blist> <blist> <bibtext> Pearson Learning Group. (2003). Developmental reading AssessmentTM (DRA) technical manual. Parsippany, NJ: Pearson Education, Inc.</bibtext> </blist> <blist> <bibtext> Pearson, P. D., & Hiebert, E. H. (2014). The state of the field: Qualitative analyses of text complexity. The Elementary School Journal, 115(2), 161–183. doi:10.1086/678297</bibtext> </blist> <blist> <bibtext> Pikulski, J. J., & Shanahan, T. (1982). Informal reading inventories: A critical analysis. In J. J. Pikulski & T. Shanahan (Eds.), Approaches to the informal evaluation of reading (pp. 94–116). Newark, DE: International Reading Association.</bibtext> </blist> <blist> <bibtext> RAND Reading Study Group. (2002). Reading for understanding: Toward an R&D program in reading comprehension. Santa Monica, CA: RAND Corporation.</bibtext> </blist> <blist> <bibtext> Ransford-Kaldon, C. R., Flynt, E. S., Ross, C. L., Franceschini, L., Zoblostsky, T., & Gallagher, B. (2010). Implementation of effective intervention: An empirical study to evaluate the efficacy of Fountas & Pinnell's leveled literacy intervention system (LLI) 2009-2010. Memphis, TN: Center for Research in Educational Policy, The University of Memphis.</bibtext> </blist> <blist> <bibtext> Rathvon, N. (2006). Developmental reading assessment: DRA review. Retrieved from <ulink href="http://www.natalierathvon.com/images/DRA%5fReview-08-25-2006.pdf">http://www.natalierathvon.com/images/DRA%5fReview-08-25-2006.pdf</ulink></bibtext> </blist> <blist> <bibtext> Samuels, S. J. (2007). The DIBELS tests: Is speed of barking at print what we mean by reading fluency? Reading Research Quarterly, 42(4), 563–566.</bibtext> </blist> <blist> <bibtext> Sheehan, K. M. (2014). Development of a TextEvaluator/common core concordance table. Princeton, NJ: ETS.</bibtext> </blist> <blist> <bibtext> Sheehan, K. M., Kostin, I., Napolitano, D., & Flor, M. (2014). The TextEvaluator tool: Helping teachers and test developers select texts for use in instruction and assessment. Elementary School Journal, 115(2), 184–209. doi:10.1086/678294</bibtext> </blist> <blist> <bibtext> Shelton, N. R., Altwerger, B., & Jordan, N. (2009). Does DIBELS put reading first? Literacy Research and Instruction, 48(2), 137–148. doi:10.1080/19388070802226311</bibtext> </blist> <blist> <bibtext> Šidák, Z. K. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62(318), 626–633.</bibtext> </blist> <blist> <bibtext> Smith, D. R., Stenner, A. J., Horabin, I., & Smith, M. (1989). The Lexile scale in theory and practice. Final report. Washington, DC: MetaMetrics. Retrieved from ERIC Database (ED307577).</bibtext> </blist> <blist> <bibtext> Spector, J. E. (2005). How reliable are informal reading inventories? Psychology in the Schools, 42(6), 593–603. doi:10.1002/(ISSN)1520-6807</bibtext> </blist> <blist> <bibtext> Stahl, S. (2003). Vocabulary and readability: How knowing word meanings affects comprehension. Topics in Language Disorders, 23(3), 241–247. doi:10.1097/00011363-200307000-00009</bibtext> </blist> <blist> <bibtext> Stahl, S. A., & Heubach, K. M. (2005). Fluency-oriented reading instruction. Journal of Literacy Research, 37(1), 25–60. doi:10.1207/s15548430jlr3701_2</bibtext> </blist> <blist> <bibtext> Stenner, A. J. (1996). Measuring reading comprehension with the lexile framework. Paper presented at the California Comparability Symposium. Retrieved March 26, 2017, from https://lexile.com/research/12/</bibtext> </blist> <blist> <bibtext> Stenner, A. J., Burdick, H., Sanford, E. E., & Burdick, D. S. (2006). How accurate are lexile text measures? Journal of Applied Measurement, 7(3), 307–322.</bibtext> </blist> <blist> <bibtext> Stenner, A. J., & Fisher, W. P. (2013). Metrological traceability in the social sciences: A model from reading measurement. Journal of Physics: Conference Series, 459, 1–6.</bibtext> </blist> <blist> <bibtext> Valencia, S., Smith, A., Reece, A., Li, M., Wixson, K., & Newman, H. (2010). Oral reading fluency assessment: Issues of construct, criterion, and consequential validity. Reading Research Quarterly, 45, 270–291. doi:10.1598/RRQ.45.3.1</bibtext> </blist> <blist> <bibtext> Valencia, S. W., Pearson, P. D., & Wixson, K. K. (2011). Assessing and tracking progress in reading comprehension: The search for keystone elements in college and career readiness. Princeton: Center for K-12 Assessment & Performance Management at ETS. Retrieved from <ulink href="http://www.k12center.org/publications/through%5fcourse.html">http://www.k12center.org/publications/through%5fcourse.html</ulink></bibtext> </blist> <blist> <bibtext> Valencia, S. W., Wixson, K. K., & Pearson, P. D. (2014). Putting text complexity in context: Refocusing on comprehension of complex text. Elementary School Journal, 115(2), 270–289. doi:10.1086/678296</bibtext> </blist> <blist> <bibtext> White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica: Journal of the Econometric Society, 48(4), 817–838. doi:10.2307/1912934</bibtext> </blist> </ref> <ref id="AN0124435026-37"> <title> Footnotes </title> <blist> <bibtext> We follow Mesmer, Cunningham, and Hiebert's ([57]) distinction between text complexity and text difficulty. Text complexity is indexed by inherent properties of text, which are largely linguistic and discourse features of text; most important, they can be manipulated by researchers and text designers. Text complexity indices typically serve as independent variables that predict text difficulty. Text difficulty, in contrast, is indexed directly by student reading comprehension performance or indirectly by expert judgments of the likely difficulty students will encounter. Thus difficulty is an actual or predicted performance of multiple readers on a specific reading comprehension passage/task (or in the case of expert judgment, the actual levels at which the judges placed the various texts).</bibtext> </blist> <blist> <bibtext> Logit is a unit of measurement that represents an exponential distance between the reader's ability and the text's difficulty, and one logit equals to 180L. See Stenner ([84]) for details about rescaling text's difficulty on the logit scale to the Lexile scale.</bibtext> </blist> <blist> <bibtext> Coh-Metrix (Graesser et al., [34]) was originally part the study but not part of comparative correlational analysis because this system provided only multidimensional indices. Subsequent to the study, Graesser et al. ([33]) have developed a single index.</bibtext> </blist> <blist> <bibtext> As noted before, results from Flesch-Kincaid and TE are provided in the on-line supplementary materials.</bibtext> </blist> </ref> <aug> <p>By Yukie Toyama; Elfrieda H. Hiebert and P. David Pearson</p> <p>Reported by Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib64" firstref="ref3"></nolink> <nolink nlid="nl2" bibid="bib16" firstref="ref4"></nolink> <nolink nlid="nl3" bibid="bib31" firstref="ref5"></nolink> <nolink nlid="nl4" bibid="bib10" firstref="ref6"></nolink> <nolink nlid="nl5" bibid="bib48" firstref="ref7"></nolink> <nolink nlid="nl6" bibid="bib62" firstref="ref8"></nolink> <nolink nlid="nl7" bibid="bib63" firstref="ref9"></nolink> <nolink nlid="nl8" bibid="bib66" firstref="ref10"></nolink> <nolink nlid="nl9" bibid="bib19" firstref="ref11"></nolink> <nolink nlid="nl10" bibid="bib39" firstref="ref12"></nolink> <nolink nlid="nl11" bibid="bib57" firstref="ref13"></nolink> <nolink nlid="nl12" bibid="bib14" firstref="ref15"></nolink> <nolink nlid="nl13" bibid="bib32" firstref="ref16"></nolink> <nolink nlid="nl14" bibid="bib23" firstref="ref19"></nolink> <nolink nlid="nl15" bibid="bib65" firstref="ref21"></nolink> <nolink nlid="nl16" bibid="bib67" firstref="ref23"></nolink> <nolink nlid="nl17" bibid="bib83" firstref="ref26"></nolink> <nolink nlid="nl18" bibid="bib68" firstref="ref27"></nolink> <nolink nlid="nl19" bibid="bib25" firstref="ref28"></nolink> <nolink nlid="nl20" bibid="bib73" firstref="ref29"></nolink> <nolink nlid="nl21" bibid="bib47" firstref="ref31"></nolink> <nolink nlid="nl22" bibid="bib71" firstref="ref32"></nolink> <nolink nlid="nl23" bibid="bib81" firstref="ref33"></nolink> <nolink nlid="nl24" bibid="bib30" firstref="ref34"></nolink> <nolink nlid="nl25" bibid="bib78" firstref="ref35"></nolink> <nolink nlid="nl26" bibid="bib55" firstref="ref36"></nolink> <nolink nlid="nl27" bibid="bib20" firstref="ref37"></nolink> <nolink nlid="nl28" bibid="bib75" firstref="ref39"></nolink> <nolink nlid="nl29" bibid="bib87" firstref="ref40"></nolink> <nolink nlid="nl30" bibid="bib50" firstref="ref42"></nolink> <nolink nlid="nl31" bibid="bib29" firstref="ref44"></nolink> <nolink nlid="nl32" bibid="bib42" firstref="ref45"></nolink> <nolink nlid="nl33" bibid="bib12" firstref="ref47"></nolink> <nolink nlid="nl34" bibid="bib26" firstref="ref48"></nolink> <nolink nlid="nl35" bibid="bib40" firstref="ref49"></nolink> <nolink nlid="nl36" bibid="bib43" firstref="ref51"></nolink> <nolink nlid="nl37" bibid="bib13" firstref="ref52"></nolink> <nolink nlid="nl38" bibid="bib46" firstref="ref55"></nolink> <nolink nlid="nl39" bibid="bib56" firstref="ref56"></nolink> <nolink nlid="nl40" bibid="bib44" firstref="ref57"></nolink> <nolink nlid="nl41" bibid="bib27" firstref="ref58"></nolink> <nolink nlid="nl42" bibid="bib85" firstref="ref60"></nolink> <nolink nlid="nl43" bibid="bib49" firstref="ref61"></nolink> <nolink nlid="nl44" bibid="bib59" firstref="ref62"></nolink> <nolink nlid="nl45" bibid="bib80" firstref="ref63"></nolink> <nolink nlid="nl46" bibid="bib86" firstref="ref64"></nolink> <nolink nlid="nl47" bibid="bib21" firstref="ref73"></nolink> <nolink nlid="nl48" bibid="bib54" firstref="ref74"></nolink> <nolink nlid="nl49" bibid="bib34" firstref="ref75"></nolink> <nolink nlid="nl50" bibid="bib77" firstref="ref76"></nolink> <nolink nlid="nl51" bibid="bib51" firstref="ref77"></nolink> <nolink nlid="nl52" bibid="bib60" firstref="ref81"></nolink> <nolink nlid="nl53" bibid="bib58" firstref="ref83"></nolink> <nolink nlid="nl54" bibid="bib36" firstref="ref85"></nolink> <nolink nlid="nl55" bibid="bib61" firstref="ref90"></nolink> <nolink nlid="nl56" bibid="bib76" firstref="ref92"></nolink> <nolink nlid="nl57" bibid="bib18" firstref="ref93"></nolink> <nolink nlid="nl58" bibid="bib35" firstref="ref94"></nolink> <nolink nlid="nl59" bibid="bib53" firstref="ref95"></nolink> <nolink nlid="nl60" bibid="bib41" firstref="ref101"></nolink> <nolink nlid="nl61" bibid="bib15" firstref="ref104"></nolink> <nolink nlid="nl62" bibid="bib69" firstref="ref105"></nolink> <nolink nlid="nl63" bibid="bib70" firstref="ref107"></nolink> <nolink nlid="nl64" bibid="bib74" firstref="ref110"></nolink> <nolink nlid="nl65" bibid="bib17" firstref="ref112"></nolink> <nolink nlid="nl66" bibid="bib90" firstref="ref119"></nolink> <nolink nlid="nl67" bibid="bib79" firstref="ref120"></nolink> <nolink nlid="nl68" bibid="bib82" firstref="ref125"></nolink> <nolink nlid="nl69" bibid="bib28" firstref="ref129"></nolink> <nolink nlid="nl70" bibid="bib37" firstref="ref130"></nolink> <nolink nlid="nl71" bibid="bib38" firstref="ref131"></nolink> <nolink nlid="nl72" bibid="bib22" firstref="ref132"></nolink> <nolink nlid="nl73" bibid="bib11" firstref="ref135"></nolink> <nolink nlid="nl74" bibid="bib52" firstref="ref136"></nolink> <nolink nlid="nl75" bibid="bib45" firstref="ref137"></nolink> <nolink nlid="nl76" bibid="bib72" firstref="ref138"></nolink> <nolink nlid="nl77" bibid="bib88" firstref="ref139"></nolink> <nolink nlid="nl78" bibid="bib89" firstref="ref140"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1150145
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: An Analysis of the Text Complexity of Leveled Passages in Four Popular Classroom Reading Assessments
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Toyama%2C+Yukie%22">Toyama, Yukie</searchLink><br /><searchLink fieldCode="AR" term="%22Hiebert%2C+Elfrieda+H%2E%22">Hiebert, Elfrieda H.</searchLink><br /><searchLink fieldCode="AR" term="%22Pearson%2C+P%2E+David%22">Pearson, P. David</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Educational+Assessment%22"><i>Educational Assessment</i></searchLink>. 2017 22(3):139-170.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 32
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2017
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Grade+1%22">Grade 1</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+2%22">Grade 2</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+3%22">Grade 3</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+4%22">Grade 4</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+5%22">Grade 5</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+6%22">Grade 6</searchLink><br /><searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Reading+Materials%22">Reading Materials</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Skills%22">Reading Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Evaluation%22">Student Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Instructional+Program+Divisions%22">Instructional Program Divisions</searchLink><br /><searchLink fieldCode="DE" term="%22Common+Core+State+Standards%22">Common Core State Standards</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Tests%22">Reading Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Fluency%22">Reading Fluency</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Emergent+Literacy%22">Emergent Literacy</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Readability+Formulas%22">Readability Formulas</searchLink><br /><searchLink fieldCode="DE" term="%22Readability%22">Readability</searchLink><br /><searchLink fieldCode="DE" term="%22Informal+Reading+Inventories%22">Informal Reading Inventories</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Achievement%22">Reading Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Regression+%28Statistics%29%22">Regression (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Multivariate+Analysis%22">Multivariate Analysis</searchLink>
– Name: SubjectThesaurus
  Label: Assessment and Survey Identifiers
  Group: Su
  Data: <searchLink fieldCode="SU" term="%22Dynamic+Indicators+of+Basic+Early+Literacy+Skills+%28DIBELS%29%22">Dynamic Indicators of Basic Early Literacy Skills (DIBELS)</searchLink><br /><searchLink fieldCode="SU" term="%22Flesch+Kincaid+Grade+Level+Formula%22">Flesch Kincaid Grade Level Formula</searchLink><br /><searchLink fieldCode="SU" term="%22Basic+Reading+Inventory%22">Basic Reading Inventory</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/10627197.2017.1344091
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1062-7197
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study investigated the complexity of leveled passages used in four classroom reading assessments. A total of 167 passages leveled for Grades 1-6 from these assessments were analyzed using four analytical tools of text complexity. More traditional, two-factor measures of text complexity found a general trend of fairly consistent across-grade progression of average complexity among the four assessments. However, considerable cross-assessment variability was observed in terms of the size of increase in complexity from grade to grade, the overall range of complexity, and the within-grade text complexity. These cross-assessment differences were less pronounced with newer, multi-factor analytical tools. The four assessments also differed in the extent to which their passages met the text complexity guidelines of the Common Core State Standards. The authors discuss implications of the differences found among and within the classroom assessment systems, on one hand, and among the measures of text complexity, on the other.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 90
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2017
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1150145
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1150145
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/10627197.2017.1344091
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 32
        StartPage: 139
    Subjects:
      – SubjectFull: Reading Materials
        Type: general
      – SubjectFull: Reading Skills
        Type: general
      – SubjectFull: Student Evaluation
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Instructional Program Divisions
        Type: general
      – SubjectFull: Common Core State Standards
        Type: general
      – SubjectFull: Reading Tests
        Type: general
      – SubjectFull: Reading Fluency
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Emergent Literacy
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
      – SubjectFull: Statistical Analysis
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Readability Formulas
        Type: general
      – SubjectFull: Readability
        Type: general
      – SubjectFull: Informal Reading Inventories
        Type: general
      – SubjectFull: Reading Achievement
        Type: general
      – SubjectFull: Regression (Statistics)
        Type: general
      – SubjectFull: Multivariate Analysis
        Type: general
      – SubjectFull: Dynamic Indicators of Basic Early Literacy Skills (DIBELS)
        Type: general
      – SubjectFull: Flesch Kincaid Grade Level Formula
        Type: general
      – SubjectFull: Basic Reading Inventory
        Type: general
    Titles:
      – TitleFull: An Analysis of the Text Complexity of Leveled Passages in Four Popular Classroom Reading Assessments
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Toyama, Yukie
      – PersonEntity:
          Name:
            NameFull: Hiebert, Elfrieda H.
      – PersonEntity:
          Name:
            NameFull: Pearson, P. David
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2017
          Identifiers:
            – Type: issn-print
              Value: 1062-7197
          Numbering:
            – Type: volume
              Value: 22
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Educational Assessment
              Type: main
ResultId 1