A Review of Methods and Instruments Used in State and Local School Readiness Evaluations. Issues & Answers. REL 2007-No. 004
Saved in:
| Title: | A Review of Methods and Instruments Used in State and Local School Readiness Evaluations. Issues & Answers. REL 2007-No. 004 |
|---|---|
| Language: | English |
| Authors: | Brown, Glyn, Scott-Little, Catherine, Amwake, Lynn, Wynn, Lucy |
| Source: | Regional Educational Laboratory Southeast. 2007. |
| Availability: | Regional Educational Laboratory Southeast at SERVE Center. University of North Carolina at Greensboro, 915 Northridge Street, Greensboro, NC 27403. Tel: 800-755-3277; e-mail: RELSoutheast@serve.org; Web site: http://www.serve.org |
| Peer Reviewed: | Y |
| Page Count: | 82 |
| Publication Date: | 2007 |
| Sponsoring Agency: | Institute of Education Sciences (ED), Washington, DC. |
| Document Type: | Information Analyses Reports - Evaluative |
| Education Level: | Early Childhood Education |
| Descriptors: | School Readiness, Program Evaluation, Program Effectiveness, Evaluation Methods, Aptitude Tests, Measures (Individuals), Early Childhood Education, Outcomes of Education, Performance Based Assessment, Test Reliability, Test Validity |
| Assessment and Survey Identifiers: | Battelle Developmental Inventory, Bracken Basic Concept Scale, Developmental Indicators for the Assessment Learning, Expressive One Word Picture Vocabulary Test, Oral and Written Language Scales, Peabody Picture Vocabulary Test, Preschool and Kindergarten Behavior Scales, Preschool Language Scale, Social Skills Rating System, Woodcock Johnson Tests of Achievement, Woodcock Johnson Tests of Cognitive Ability |
| Abstract: | The report provides detailed information about the methods and instruments used to evaluate school readiness initiatives, discusses important considerations in selecting instruments, and provides resources and recommendations that may be helpful to those who are designing and implementing school readiness evaluations. Study results indicate that state and local evaluators have used a variety of instruments to collect child outcome data, some that are well known and others that are not. In general, many of the better-known instruments demonstrate adequate psychometric properties (reliability and validity, which ensure that the instruments consistently measure what they were intended to measure), but a number of issues, such as the appropriateness of the measure to the study's purpose and sample, appear to present substantial challenges in evaluations of state- and locally-funded school readiness programs. Recommendations based on the data collected from this sample are provided to help school readiness programs and evaluators as they select instruments for assessing programs and implementing the evaluations include: (1) Careful selection of outcomes for assessment that match the goals of the program and address the components of children's learning and development that are linked with later success in school; (2) Clear definition of the purpose for which assessment data will be collected, and selection of instruments designed and validated for that purpose; (3) Selection of instruments that have a proven track record; (4) Selection of instruments that are culturally and linguistically appropriate; (5) Consideration of data collection (internal/external); (6) Assessment administration, including training and reliability studies; and (7) Data collection in context. Report findings highlight the challenges that evaluators face in ensuring that data are collected in a manner that yields credible, trustworthy, and meaningful information about child outcomes. The report cites a number of resources that can assist evaluators in making decisions about child assessments: resources to guide decisions about how to assess child outcomes, reviews of measures, and web sites with technical information related to measures used in large federal studies. Four appendixes include: (A) Methodology; (B) Coding Sheet and Evaluations Reviewed; (C) Evaluation Reports Reviewed, by State and Program; and (D) Descriptions of Most Commonly Used Instruments. (Contains 4 boxes, 6 figures, 4 tables, and 28 notes.) [This report was prepared for the Institute of Education Sciences (IES) by Regional Educational Laboratory Southeast administered by SERVE Center, University of North Carolina at Greensboro.] |
| Abstractor: | ERIC |
| Number of References: | 68 |
| IES Funded: | Yes |
| IES Publication: | https://ies.ed.gov/ncee/edlabs/projects/project.asp?ProjectID=64 |
| Entry Date: | 2007 |
| Accession Number: | ED497789 |
| Database: | ERIC |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED497789 Name: ERIC Full Text Category: fullText Text: Full Text from ERIC |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: ED497789 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: A Review of Methods and Instruments Used in State and Local School Readiness Evaluations. Issues & Answers. REL 2007-No. 004 – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Brown%2C+Glyn%22">Brown, Glyn</searchLink><br /><searchLink fieldCode="AR" term="%22Scott-Little%2C+Catherine%22">Scott-Little, Catherine</searchLink><br /><searchLink fieldCode="AR" term="%22Amwake%2C+Lynn%22">Amwake, Lynn</searchLink><br /><searchLink fieldCode="AR" term="%22Wynn%2C+Lucy%22">Wynn, Lucy</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="SO" term="%22Regional+Educational+Laboratory+Southeast%22"><i>Regional Educational Laboratory Southeast</i></searchLink>. 2007. – Name: Avail Label: Availability Group: Avail Data: Regional Educational Laboratory Southeast at SERVE Center. University of North Carolina at Greensboro, 915 Northridge Street, Greensboro, NC 27403. Tel: 800-755-3277; e-mail: RELSoutheast@serve.org; Web site: http://www.serve.org – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: Y – Name: Pages Label: Page Count Group: Src Data: 82 – Name: DatePubCY Label: Publication Date Group: Date Data: 2007 – Name: SourceSuprt Label: Sponsoring Agency Group: SrcSuprt Data: Institute of Education Sciences (ED), Washington, DC. – Name: TypeDocument Label: Document Type Group: TypDoc Data: Information Analyses<br />Reports - Evaluative – Name: Audience Label: Education Level Group: Audnce Data: <searchLink fieldCode="EL" term="%22Early+Childhood+Education%22">Early Childhood Education</searchLink> – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22School+Readiness%22">School Readiness</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Evaluation%22">Program Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Effectiveness%22">Program Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Aptitude+Tests%22">Aptitude Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Measures+%28Individuals%29%22">Measures (Individuals)</searchLink><br /><searchLink fieldCode="DE" term="%22Early+Childhood+Education%22">Early Childhood Education</searchLink><br /><searchLink fieldCode="DE" term="%22Outcomes+of+Education%22">Outcomes of Education</searchLink><br /><searchLink fieldCode="DE" term="%22Performance+Based+Assessment%22">Performance Based Assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink> – Name: SubjectThesaurus Label: Assessment and Survey Identifiers Group: Su Data: <searchLink fieldCode="SU" term="%22Battelle+Developmental+Inventory%22">Battelle Developmental Inventory</searchLink><br /><searchLink fieldCode="SU" term="%22Bracken+Basic+Concept+Scale%22">Bracken Basic Concept Scale</searchLink><br /><searchLink fieldCode="SU" term="%22Developmental+Indicators+for+the+Assessment+Learning%22">Developmental Indicators for the Assessment Learning</searchLink><br /><searchLink fieldCode="SU" term="%22Expressive+One+Word+Picture+Vocabulary+Test%22">Expressive One Word Picture Vocabulary Test</searchLink><br /><searchLink fieldCode="SU" term="%22Oral+and+Written+Language+Scales%22">Oral and Written Language Scales</searchLink><br /><searchLink fieldCode="SU" term="%22Peabody+Picture+Vocabulary+Test%22">Peabody Picture Vocabulary Test</searchLink><br /><searchLink fieldCode="SU" term="%22Preschool+and+Kindergarten+Behavior+Scales%22">Preschool and Kindergarten Behavior Scales</searchLink><br /><searchLink fieldCode="SU" term="%22Preschool+Language+Scale%22">Preschool Language Scale</searchLink><br /><searchLink fieldCode="SU" term="%22Social+Skills+Rating+System%22">Social Skills Rating System</searchLink><br /><searchLink fieldCode="SU" term="%22Woodcock+Johnson+Tests+of+Achievement%22">Woodcock Johnson Tests of Achievement</searchLink><br /><searchLink fieldCode="SU" term="%22Woodcock+Johnson+Tests+of+Cognitive+Ability%22">Woodcock Johnson Tests of Cognitive Ability</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: The report provides detailed information about the methods and instruments used to evaluate school readiness initiatives, discusses important considerations in selecting instruments, and provides resources and recommendations that may be helpful to those who are designing and implementing school readiness evaluations. Study results indicate that state and local evaluators have used a variety of instruments to collect child outcome data, some that are well known and others that are not. In general, many of the better-known instruments demonstrate adequate psychometric properties (reliability and validity, which ensure that the instruments consistently measure what they were intended to measure), but a number of issues, such as the appropriateness of the measure to the study's purpose and sample, appear to present substantial challenges in evaluations of state- and locally-funded school readiness programs. Recommendations based on the data collected from this sample are provided to help school readiness programs and evaluators as they select instruments for assessing programs and implementing the evaluations include: (1) Careful selection of outcomes for assessment that match the goals of the program and address the components of children's learning and development that are linked with later success in school; (2) Clear definition of the purpose for which assessment data will be collected, and selection of instruments designed and validated for that purpose; (3) Selection of instruments that have a proven track record; (4) Selection of instruments that are culturally and linguistically appropriate; (5) Consideration of data collection (internal/external); (6) Assessment administration, including training and reliability studies; and (7) Data collection in context. Report findings highlight the challenges that evaluators face in ensuring that data are collected in a manner that yields credible, trustworthy, and meaningful information about child outcomes. The report cites a number of resources that can assist evaluators in making decisions about child assessments: resources to guide decisions about how to assess child outcomes, reviews of measures, and web sites with technical information related to measures used in large federal studies. Four appendixes include: (A) Methodology; (B) Coding Sheet and Evaluations Reviewed; (C) Evaluation Reports Reviewed, by State and Program; and (D) Descriptions of Most Commonly Used Instruments. (Contains 4 boxes, 6 figures, 4 tables, and 28 notes.) [This report was prepared for the Institute of Education Sciences (IES) by Regional Educational Laboratory Southeast administered by SERVE Center, University of North Carolina at Greensboro.] – Name: AbstractInfo Label: Abstractor Group: Ab Data: ERIC – Name: Ref Label: Number of References Group: RefInfo Data: 68 – Name: CodeSource Label: IES Funded Group: SrcInfo Data: Yes – Name: CodeSource Label: IES Publication Group: SrcInfo Data: <link linkTarget="URL" linkTerm="https://ies.ed.gov/ncee/edlabs/projects/project.asp?ProjectID=64" linkWindow="_blank">http://ies.ed.gov/ncee/edlabs/projects/project.asp?ProjectID=64</link> – Name: DateEntry Label: Entry Date Group: Date Data: 2007 – Name: AN Label: Accession Number Group: ID Data: ED497789 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED497789 |
| RecordInfo | BibRecord: BibEntity: Languages: – Text: English PhysicalDescription: Pagination: PageCount: 82 Subjects: – SubjectFull: School Readiness Type: general – SubjectFull: Program Evaluation Type: general – SubjectFull: Program Effectiveness Type: general – SubjectFull: Evaluation Methods Type: general – SubjectFull: Aptitude Tests Type: general – SubjectFull: Measures (Individuals) Type: general – SubjectFull: Early Childhood Education Type: general – SubjectFull: Outcomes of Education Type: general – SubjectFull: Performance Based Assessment Type: general – SubjectFull: Test Reliability Type: general – SubjectFull: Test Validity Type: general – SubjectFull: Battelle Developmental Inventory Type: general – SubjectFull: Bracken Basic Concept Scale Type: general – SubjectFull: Developmental Indicators for the Assessment Learning Type: general – SubjectFull: Expressive One Word Picture Vocabulary Test Type: general – SubjectFull: Oral and Written Language Scales Type: general – SubjectFull: Peabody Picture Vocabulary Test Type: general – SubjectFull: Preschool and Kindergarten Behavior Scales Type: general – SubjectFull: Preschool Language Scale Type: general – SubjectFull: Social Skills Rating System Type: general – SubjectFull: Woodcock Johnson Tests of Achievement Type: general – SubjectFull: Woodcock Johnson Tests of Cognitive Ability Type: general Titles: – TitleFull: A Review of Methods and Instruments Used in State and Local School Readiness Evaluations. Issues & Answers. REL 2007-No. 004 Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Brown, Glyn – PersonEntity: Name: NameFull: Scott-Little, Catherine – PersonEntity: Name: NameFull: Amwake, Lynn – PersonEntity: Name: NameFull: Wynn, Lucy IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 08 Type: published Y: 2007 Titles: – TitleFull: Regional Educational Laboratory Southeast Type: main |
| ResultId | 1 |
