View in EDS

Expert Judgment vs. Psychometric Analysis: Enhancing High School Exam Development in Albania

Saved in:
Bibliographic Details
Title: Expert Judgment vs. Psychometric Analysis: Enhancing High School Exam Development in Albania
Language: English
Authors: Eralda Gjika (ORCID 0000-0003-2662-4316), Lule Basha (ORCID 0000-0003-3790-601X), Afërdita Alizoti (ORCID 0009-0005-2952-8460), João Paulo Araujo Lessa (ORCID 0000-0003-0751-7662)
Source: International Journal of Assessment Tools in Education. 2026 13(1):108-122.
Availability: International Journal of Assessment Tools in Education. Pamukkale University, Faculty of Education, Kinikli Campus, Denizli 20070, Turkey. e-mail: ijate.editor@gmail.com; Web site: https://dergipark.org.tr/en/pub/ijate
Peer Reviewed: Y
Page Count: 15
Publication Date: 2026
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
High Schools
Secondary Education
Descriptors: College Entrance Examinations, Foreign Countries, Expertise, Psychometrics, High School Students, Test Items, Difficulty Level, Multiple Choice Tests, Test Format, Test Theory
Geographic Terms: Albania
ISSN: 2148-7456
Abstract: In Albania, high school graduates undergo State Matura exams that determine university entrance through a merit-based system. Since 2006, these pencil and paper exams have included three mandatory exams: Albanian language and literature, Mathematics, and a foreign language, plus one elective exam from a list of eight. Each exam totals 60 points: 20 points for multiple-choice items and 40 points for open-response items (structured and essay-type) covering Albanian literacy and a foreign language. Historically, item difficulty has been set primarily by expert judgment, with limited psychometric validation, and student proficiency has been computed via Classical Test Theory (CTT) on a 4-10 decimal scale. This study highlights the lack of psychometric analysis in Matura exams, emphasizing the need for improved assessment methods. By focusing on the limitations of relying solely on expert judgment in an era of technological innovations, we address the challenges posed by insufficient historical item parameters. To support expert judgment, we present a ShinyApp that integrates exam data to provide fast, transparent, and replicable psychometric insights. The tool demonstrates how technology can support evidence-based decision-making and contribute to modernizing Albania's e-assessment framework.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1495773
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1495773
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1495773
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Expert Judgment vs. Psychometric Analysis: Enhancing High School Exam Development in Albania
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Eralda+Gjika%22">Eralda Gjika</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-2662-4316">0000-0003-2662-4316</externalLink>)<br /><searchLink fieldCode="AR" term="%22Lule+Basha%22">Lule Basha</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3790-601X">0000-0003-3790-601X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Afërdita+Alizoti%22">Afërdita Alizoti</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0005-2952-8460">0009-0005-2952-8460</externalLink>)<br /><searchLink fieldCode="AR" term="%22João+Paulo+Araujo+Lessa%22">João Paulo Araujo Lessa</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-0751-7662">0000-0003-0751-7662</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Assessment+Tools+in+Education%22"><i>International Journal of Assessment Tools in Education</i></searchLink>. 2026 13(1):108-122.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Journal of Assessment Tools in Education. Pamukkale University, Faculty of Education, Kinikli Campus, Denizli 20070, Turkey. e-mail: ijate.editor@gmail.com; Web site: https://dergipark.org.tr/en/pub/ijate
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 15
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22High+Schools%22">High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22College+Entrance+Examinations%22">College Entrance Examinations</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Expertise%22">Expertise</searchLink><br /><searchLink fieldCode="DE" term="%22Psychometrics%22">Psychometrics</searchLink><br /><searchLink fieldCode="DE" term="%22High+School+Students%22">High School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Theory%22">Test Theory</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Albania%22">Albania</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2148-7456
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In Albania, high school graduates undergo State Matura exams that determine university entrance through a merit-based system. Since 2006, these pencil and paper exams have included three mandatory exams: Albanian language and literature, Mathematics, and a foreign language, plus one elective exam from a list of eight. Each exam totals 60 points: 20 points for multiple-choice items and 40 points for open-response items (structured and essay-type) covering Albanian literacy and a foreign language. Historically, item difficulty has been set primarily by expert judgment, with limited psychometric validation, and student proficiency has been computed via Classical Test Theory (CTT) on a 4-10 decimal scale. This study highlights the lack of psychometric analysis in Matura exams, emphasizing the need for improved assessment methods. By focusing on the limitations of relying solely on expert judgment in an era of technological innovations, we address the challenges posed by insufficient historical item parameters. To support expert judgment, we present a ShinyApp that integrates exam data to provide fast, transparent, and replicable psychometric insights. The tool demonstrates how technology can support evidence-based decision-making and contribute to modernizing Albania's e-assessment framework.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1495773
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1495773
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 15
        StartPage: 108
    Subjects:
      – SubjectFull: College Entrance Examinations
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Expertise
        Type: general
      – SubjectFull: Psychometrics
        Type: general
      – SubjectFull: High School Students
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Multiple Choice Tests
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: Test Theory
        Type: general
      – SubjectFull: Albania
        Type: general
    Titles:
      – TitleFull: Expert Judgment vs. Psychometric Analysis: Enhancing High School Exam Development in Albania
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Eralda Gjika
      – PersonEntity:
          Name:
            NameFull: Lule Basha
      – PersonEntity:
          Name:
            NameFull: Afërdita Alizoti
      – PersonEntity:
          Name:
            NameFull: João Paulo Araujo Lessa
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-electronic
              Value: 2148-7456
          Numbering:
            – Type: volume
              Value: 13
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: International Journal of Assessment Tools in Education
              Type: main
ResultId 1