View in EDS HTML Full Text PDF Full Text

Selecting Technically Adequate Tests

Saved in:
Bibliographic Details
Title: Selecting Technically Adequate Tests
Language: English
Authors: Susan K. Johnsen (ORCID 0000-0002-5425-3498)
Source: Gifted Child Today. 2024 47(2):142-145.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 4
Publication Date: 2024
Intended Audience: Teachers
Document Type: Journal Articles
Guides - Classroom - Teacher
Descriptors: Test Selection, Academically Gifted, Gifted Education, Test Validity, Test Reliability, Test Norms, Scoring, Testing, Test Format, Talent Identification, Student Evaluation
DOI: 10.1177/10762175231222308
ISSN: 1076-2175
2162-951X
Abstract: The author provides a checklist for educators who are selecting technically adequate tests for identifying and referring students for gifted education services and programs. The checklist includes questions related to how the test was normed, reliability and validity studies as well as questions related to types of scores, administration, and format. The author concludes that selecting technically adequate tests is a critical step in developing an identification procedure that will be inclusive, reliable, and valid for finding students who will benefit from the school's gifted education services and programs.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1417376
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEuKIhjOSg6k-vma4e_JZRWAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDDj1LtFxaJd9VpAloQIBEICBm_zdd5dlDYMJVWpv-Khn70MyNY5e6VIrGnnvO9iFo8WT8x8LHqydF-v88l6HXXKmSMfSg1o0eZpDoLAykc295tsonfN9VT1pK0fzWwgi1lKxu-eOKRna-cH_VS4Hc_El4j4KwA8wuMyQ6Lk5D7h1LbT1lIVoJV0E_UCkZ6QXeTsacAfHSPhx2PhsxEh07L5qvV5A8isATGAfBK0c
Text:
  Availability: 1
  Value: <anid>AN0176065282;gct01apr.24;2025Aug11.09:20;v2.2.500</anid> <title id="AN0176065282-1">Selecting Technically Adequate Tests </title> <p>The author provides a checklist for educators who are selecting technically adequate tests for identifying and referring students for gifted education services and programs. The checklist includes questions related to how the test was normed, reliability and validity studies as well as questions related to types of scores, administration, and format. The author concludes that selecting technically adequate tests is a critical step in developing an identification procedure that will be inclusive, reliable, and valid for finding students who will benefit from the school's gifted education services and programs.</p> <p>Keywords: gifted students; identification; validity; reliability; norming; administration of tests; quantitative tests</p> <p>Schools need to examine how closely their identification procedures follow local, state, and national policies and standards."</p> <p>Establishing identification and referral procedures is an important first step in finding students who will benefit from services and programs offered by the school. Most identification procedures will require educators to select and interpret quantitative assessments. This column will address the critical characteristics needed to identify and select technically adequate tests. Subsequent columns will examine these categories in greater depth and how educators should interpret scores in determining a match between the test results and each student's strengths and needs.</p> <hd id="AN0176065282-2">Technically Adequate</hd> <p>What does technically adequate mean? A technically adequate test is one that can reliably (i.e., consistently) and accurately predict a student's performance related to the construct or content being measured on the test. For example, if a test is supposed to measure a gifted student's knowledge in math, then the items on the test relate to math content standards. Moreover, performance from one testing to the next testing on the math test, if given sequentially, should be similar or consistent; and students' higher performance on the test should predict higher performance in math classes than peers.</p> <p>Technical adequacy is also mentioned in Standard 4, Component 4.2 of the newly revised <emph>Initial Practice-Based Professional Preparation Standards for Gifted Educators</emph> (K-12):</p> <p>Candidates use norming, reliability and validity data, and information related to minimizing bias in selecting and interpreting assessments to identify and guide all students with gifts and talents, including those from traditionally underrepresented populations.</p> <p>Gifted educators need to understand the concepts of norm-referencing, reliability, and validity to understand if the instrument is sensitive for identifying gifted students for specific services and programs.</p> <hd id="AN0176065282-3">Questions to Ask to Determine if a Test is Technically Adequate</hd> <p>All quantitative tests that are standardized and norm-referenced have technical manuals. These manuals include chapters related to how the test was normed (i.e., the sample of students who were administered the test), reliability (i.e., consistency within the instrument, across scorers, and over time), and validity (i.e., the accuracy of the content or construct that the test is supposed to measure and studies showing how the test minimizes bias, how the test discriminates between different groups, and how well the test predicts future performance). In addition, the technical manual may also have standardized directions in how to administer the test, which are very important in ensuring consistency, and norming tables to interpret the student's raw scores. These scores are frequently presented as standard or index scores and percentile ranks.</p> <p>In reviewing the technical manual, a gifted educator should ask these questions to determine the test's technical adequacy and its usefulness in the identification process (see also [<reflink idref="bib1" id="ref1">1</reflink>]):</p> <hd id="AN0176065282-4">Purpose of the Assessment</hd> <p></p> <ulist> <item> 1. What do the test developers state is the purpose of the test?</item> <p></p> <item> _____ Is it to assess achievement?</item> <p></p> <item> _____ Is it to assess general intellectual ability?</item> <p></p> <item> _____ Is it to assess critical thinking?</item> <p></p> <item> _____ Is it to assess creative thinking?</item> <p></p> <item> _____ Is it to assess performance in the arts?</item> <p></p> <item> _____ Other: _________________________________</item> <p></p> <item> 2. Do the test developers state that the instrument may be used to identify gifted and talented students?</item> <p></p> <item> _____ Are gifted students included in the norming sample?</item> <p></p> <item> _____ Are gifted students included in validity studies?</item> <p></p> <item> _____ If the purpose of the test is achievement, are items above level?</item> <p></p> <item> 3. Does the test align with the school's gifted education services and programs?</item> <p></p> <item> _____ Is it aligned to the ages/grades where services are provided?</item> <p></p> <item> _____ Is the alignment to general abilities?</item> <p></p> <item> _____ Is the alignment to above-level performance in core content areas?</item> <p></p> <item> ––––– Is the alignment to leadership aptitude?</item> <p></p> <item> _____ Is the alignment to aptitude in the visual and performing arts?</item> <p></p> <item> _____ Other: _________________________________</item> </ulist> <hd id="AN0176065282-5">Normative Scores</hd> <p></p> <ulist> <item> 1. Does the manual provide characteristics of the norming group?</item> <p></p> <item> _____ Is the size of the norming group specified?</item> <p></p> <item> _____ Does the norming group contain 100 or more subjects in each chronological age interval or academic grade level?</item> <p></p> <item> _____ Are there at least 1,000 or more subjects in the total norming group?</item> <p></p> <item> _____ Are the demographic characteristics of the norming group specified?</item> <p></p> <item> _____ Are the demographic characteristics similar to the most recent national census data?</item> <p></p> <item> 2. Was the standardization completed within the last 12 years?</item> </ulist> <hd id="AN0176065282-6">Reliability</hd> <p></p> <ulist> <item> 1. Do the test developers describe the test's reliability (i.e., consistency)?</item> <p></p> <item> ____ Does the test report an internal consistency reliability (i.e., to what degree the items relate to one another) of at least.80 for most ages or grades?</item> <p></p> <item> ____ Does the test report stability reliability (i.e., test-retest comparisons) of.80 or greater?</item> <p></p> <item> ____ Does the test report inter-scorer reliability (i.e., examiner variations in scoring) of.90 or greater?</item> <p></p> <item> –––– If the test has alternate forms, do the scores relate to one another and have a reliability of.80 or greater?</item> </ulist> <hd id="AN0176065282-7">Validity</hd> <p></p> <ulist> <item> 1. Did the test developers include studies that show the test measures what it is supposed to measure?</item> <p></p> <item> _____ Is content-description validity included (e.g., how the test relates to standards, curriculum content, theoretical frameworks for intelligence, creativity constructs, and so on)?</item> <p></p> <item> _____ Did the authors provide studies showing how items discriminate (i.e., each item should relate to one another and to the rest of the items on the test)?</item> <p></p> <item> _____ Did the authors describe the difficulty of the items and include floors (i.e., the lowest standard score that assesses functioning should be the lowest 3%)?</item> <p></p> <item> _____ Did the authors describe the difficulty of the items and include ceilings (i.e., the highest standard score that assesses functioning should be the highest 3%)?</item> <p></p> <item> _____ Did the authors describe the difficulty of the items and include item difficulty gradients (i.e., the number of items a student needs to pass to change the standard score should be more than one third of a standard deviation)?</item> <p></p> <item> 2. Did the authors include studies related to test bias?</item> <p></p> <item> _____ Did the authors show that individual items were not biased for different groups (i.e., differential item functioning—items do not function or predict differently for different groups)?</item> <p></p> <item> ____ Did the authors include studies that showed that different demographic subgroups performed similarly on the test (e.g., male vs. female, White vs. Black/African American, and so on)?</item> <p></p> <item> 3. Did the authors describe studies that showed how the test predicted a student's performance on specific activities?</item> <p></p> <item> ____ Did the authors show how the test correlated to other tests that measure similar content/constructs?</item> <p></p> <item> ____ Did the authors show how the test predicted future performance (e.g., in a classroom for gifted students, on college level assessments)?</item> <p></p> <item> 4. Did the authors describe studies that showed the degree to which a test reflects the theoretical model on which the test is based?</item> <p></p> <item> ____ Did the authors show how the test correctly identified students who are gifted?</item> <p></p> <item> ____ Did the authors show how performance was related to age since reasoning and academic achievement are developmental in nature?</item> <p></p> <item> ____ Did the authors show how students who are above average aptitude performed better on the test overall and on different subtests?</item> <p></p> <item> ____ Did the authors conduct a factor analysis to show how the subtests relate to one another and to a particular model of aptitude and/or achievement?</item> <p></p> <item> ____ Other: _________________________________</item> </ulist> <hd id="AN0176065282-8">Scores</hd> <p></p> <ulist> <item> 1. Did the authors describe how the scores interpreted?</item> <p></p> <item> ____ Does the manual provide standard or index scores that are comparable to other tests?</item> <p></p> <item> ____ Does the manual provide percentile ranks that show the student's performance relative to others who took the test?</item> <p></p> <item> ____ Does the manual provide only grade equivalent or age equivalent scores, which are not comparable?</item> <p></p> <item> ____ Other scores:____________________________</item> </ulist> <p>Along with the above technical qualities, schools will need to consider administration and examiner requirements because of human and material resources. The format of the test may also be important to consider, particularly for students who speak other languages or those with disabilities.</p> <hd id="AN0176065282-9">Administration</hd> <p></p> <ulist> <item> 1. How is the test administered?</item> <p></p> <item> ____ Is it administered individually?</item> <p></p> <item> ____ Is it administered in whole group?</item> <p></p> <item> ____ Is it administered in small group?</item> <p></p> <item> 2. What is the length of the administration?</item> <p></p> <item> ____ Is it less than 15 min?</item> <p></p> <item> ____ Is it less than 30 min?</item> <p></p> <item> ____ Is it less than 60 min?</item> <p></p> <item> ____ Is it less than 60 min or longer?</item> <p></p> <item> 3. Who can administer the test?</item> <p></p> <item> ____ Can it be administered easily after reading the manual?</item> <p></p> <item> ____ Does it require special training beyond reading the manual?</item> <p></p> <item> ____ Is the test restricted to certified examiners such as psychologists, counselors, and other specialists?</item> <p></p> <item> ____ Is the test administered by computer?</item> <p></p> <item> 4. Who scores the test?</item> <p></p> <item> ____ Can it be scored easily after reading the manual?</item> <p></p> <item> ____ Does it require special training to score the test?</item> <p></p> <item> ____ Is the test scored by a computer?</item> </ulist> <hd id="AN0176065282-10">Format</hd> <p></p> <ulist> <item> 1. What does the input require the student to do?</item> <p></p> <item> _____ Listen</item> <p></p> <item> _____ Read</item> <p></p> <item> _____ Look at stimuli</item> <p></p> <item> _____ Use a computer</item> <p></p> <item> _____ Other: _________________________________</item> <p></p> <item> 2. What does the output require the student to do?</item> <p></p> <item> _____ Point</item> <p></p> <item> _____ Mark an answer on a response sheet</item> <p></p> <item> _____ Indicate an answer on a computer</item> <p></p> <item> _____ Manipulate objects</item> <p></p> <item> _____ Draw</item> <p></p> <item> _____ Speak (one word responses)</item> <p></p> <item> _____ Speak (phrases, sentences)</item> <p></p> <item> _____ Write short answers</item> <p></p> <item> _____ Write longer answers; essays</item> <p></p> <item> _____ Other required responses: _________________</item> </ulist> <hd id="AN0176065282-11">Conclusion</hd> <p>Schools need to examine how closely their identification procedures follow local, state, and national policies and standards. In following these standards, it is important that all educators understand how to select technically adequate tests for use in the identification process. Selecting tests is a critical step in developing an identification procedure that is inclusive and valid for finding students who will benefit from the school's gifted education services and programs.</p> <hd id="AN0176065282-12">ORCID iD</hd> <p>Susan K. Johnsen https://orcid.org/0000-0002-5425-3498</p> <ref id="AN0176065282-13"> <title> Reference </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Robins J. H., Jolly J. L. (2018). Technical information regarding assessment. In Johnsen S. K. (Ed.), Identifying gifted students: A practical guide (pp. 73–116). Prufrock Press/Routledge.</bibtext> </blist> </ref> <ref id="AN0176065282-14"> <title> Footnotes </title> <blist> <bibtext> The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.</bibtext> </blist> <blist> <bibl id="bib2" type="bt">2</bibl> <bibtext> The author(s) received no financial support for the research, authorship, and/or publication of this article.</bibtext> </blist> </ref> <aug> <p>By Susan K. Johnsen</p> <p>Reported by Author</p> <p></p> <p>Susan K. Johnsen, PhD, is Professor Emerita of Educational Psychology at Baylor University. She has written three tests for identifying gifted students and more than 300 publications related to gifted education.</p> </aug>
Header DbId: eric
DbLabel: ERIC
An: EJ1417376
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Selecting Technically Adequate Tests
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Susan+K%2E+Johnsen%22">Susan K. Johnsen</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-5425-3498">0000-0002-5425-3498</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Gifted+Child+Today%22"><i>Gifted Child Today</i></searchLink>. 2024 47(2):142-145.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 4
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: Audience
  Label: Intended Audience
  Group: Audnce
  Data: Teachers
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Guides - Classroom - Teacher
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Test+Selection%22">Test Selection</searchLink><br /><searchLink fieldCode="DE" term="%22Academically+Gifted%22">Academically Gifted</searchLink><br /><searchLink fieldCode="DE" term="%22Gifted+Education%22">Gifted Education</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Norms%22">Test Norms</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring%22">Scoring</searchLink><br /><searchLink fieldCode="DE" term="%22Testing%22">Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22Talent+Identification%22">Talent Identification</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Evaluation%22">Student Evaluation</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/10762175231222308
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1076-2175<br />2162-951X
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The author provides a checklist for educators who are selecting technically adequate tests for identifying and referring students for gifted education services and programs. The checklist includes questions related to how the test was normed, reliability and validity studies as well as questions related to types of scores, administration, and format. The author concludes that selecting technically adequate tests is a critical step in developing an identification procedure that will be inclusive, reliable, and valid for finding students who will benefit from the school's gifted education services and programs.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1417376
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1417376
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/10762175231222308
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 4
        StartPage: 142
    Subjects:
      – SubjectFull: Test Selection
        Type: general
      – SubjectFull: Academically Gifted
        Type: general
      – SubjectFull: Gifted Education
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Test Norms
        Type: general
      – SubjectFull: Scoring
        Type: general
      – SubjectFull: Testing
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: Talent Identification
        Type: general
      – SubjectFull: Student Evaluation
        Type: general
    Titles:
      – TitleFull: Selecting Technically Adequate Tests
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Susan K. Johnsen
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1076-2175
            – Type: issn-electronic
              Value: 2162-951X
          Numbering:
            – Type: volume
              Value: 47
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Gifted Child Today
              Type: main
ResultId 1