View in EDS PDF Full Text

Investigation of Coefficient of Individual Agreement in Terms of Sample Size, Random and Monotone Missing Ratio, and Number of Repeated Measures

Saved in:
Bibliographic Details
Title: Investigation of Coefficient of Individual Agreement in Terms of Sample Size, Random and Monotone Missing Ratio, and Number of Repeated Measures
Language: English
Authors: Temel, Gülhan Orekici, Erdogan, Semra, Selvi, Hüseyin, Kaya, Irem Ersöz
Source: Educational Sciences: Theory and Practice. Aug 2016 16(4):1381-1395.
Availability: Educational Consultancy, Ltd (EDAM). Kisikli Mh. Alemdag Cd. Yan Yol Sk., SBK Is Merkezi No:5 Kat:1, Uskudar-Istanbul, 34692 Turkey. Tel: +90-216-481-30-23; Fax: +90-216-481-31-36; e-mail: editor@edam.com.tr; Web site: http://www.estp.com.tr/
Peer Reviewed: Y
Page Count: 15
Publication Date: 2016
Document Type: Journal Articles
Reports - Research
Descriptors: Investigations, Sample Size, Longitudinal Studies, Interrater Reliability, Data Analysis, Error of Measurement, Experimenter Characteristics, Scoring Formulas, Correlation, Computer Simulation, Research Methodology, Test Reliability, Test Validity, Educational Practices, Foreign Countries
Geographic Terms: Turkey
ISSN: 1303-0485
Abstract: Studies based on longitudinal data focus on the change and development of the situation being investigated and allow for examining cases regarding education, individual development, cultural change, and socioeconomic improvement in time. However, as these studies require taking repeated measures in different time periods, they may include various error sources that are exclusive of the investigated variable and that change over time. In Addition, the period of observation is critical in these studies, and observant (variable) attrition or the inability to obtain measurements for various reasons (death, moving away, resignation etc.) may be observed. This study presents the coefficient of individual agreement (CIA) developed for studies in which more than one rater are needed to provide repeated scoring and examines the applications of this agreement coefficient in relation to different sample size, missing value and number of repeated measures. In this context, trial scores were generated based on different sample sizes, missing value ratios, and the number of repeated measures provided by two raters, and the expected agreement between raters was simulated as very high. Related data were replicated 1000 times. Data analysis included the mean and standard deviation of agreement ratios calculated for each combination. According to the results, independently from sample size and number of retests, the CIA takes much lower values than the expected value when random missing value rates reaches 20%. Similar conclusions can be drawn for the existence of monotone missing values except where the sample size is 30 and the number of retests are two and three. It was observed that the CIA was higher than the expected value for the data consisting of two repeated evaluations of the raters with 5% or 10% missing values when the sample size was 30. In general, the agreement coefficient did not exceed the expected level in the situation of three and four replications of measurements. As a result, it can be concluded that an increase in the replication number of measurement decreased the obtained agreement values. The obtained data showed that missing value ratio caused differentiations in the CIA. As a result, increases in missing value ratio cause inconsistencies in CIA. Therefore, it is suggested that data obtained from studies be interpreted considering missing value ratio.
Abstractor: As Provided
Number of References: 34
Entry Date: 2016
Accession Number: EJ1115072
Database: ERIC
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGDXK_IS0IgajAKF7sZAZORAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDG59IaI6HyqRTjZpNwIBEICBm3Pqj-wifbhJCT8yy_Fzw6opl2UW6OwbAvkSnEoeSpvuQe_CzKWdBIuK7LsH5O9OEYMx1qXMfeKrCYOWtHSvvYKRX83gr0oN7cZ_vC6VaoChU2Ax805zNKLq8GIiYeU8PTKQFA1mCEh3eH-iaMW713UIU9bgO1oLie6aeWhfJVCQoa38T36RvZxkXavQfkNxHGsEKWChwvzyaFPq
Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1115072
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1115072
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Investigation of Coefficient of Individual Agreement in Terms of Sample Size, Random and Monotone Missing Ratio, and Number of Repeated Measures
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Temel%2C+Gülhan+Orekici%22">Temel, Gülhan Orekici</searchLink><br /><searchLink fieldCode="AR" term="%22Erdogan%2C+Semra%22">Erdogan, Semra</searchLink><br /><searchLink fieldCode="AR" term="%22Selvi%2C+Hüseyin%22">Selvi, Hüseyin</searchLink><br /><searchLink fieldCode="AR" term="%22Kaya%2C+Irem+Ersöz%22">Kaya, Irem Ersöz</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Educational+Sciences%3A+Theory+and+Practice%22"><i>Educational Sciences: Theory and Practice</i></searchLink>. Aug 2016 16(4):1381-1395.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Educational Consultancy, Ltd (EDAM). Kisikli Mh. Alemdag Cd. Yan Yol Sk., SBK Is Merkezi No:5 Kat:1, Uskudar-Istanbul, 34692 Turkey. Tel: +90-216-481-30-23; Fax: +90-216-481-31-36; e-mail: editor@edam.com.tr; Web site: http://www.estp.com.tr/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 15
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2016
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Investigations%22">Investigations</searchLink><br /><searchLink fieldCode="DE" term="%22Sample+Size%22">Sample Size</searchLink><br /><searchLink fieldCode="DE" term="%22Longitudinal+Studies%22">Longitudinal Studies</searchLink><br /><searchLink fieldCode="DE" term="%22Interrater+Reliability%22">Interrater Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Analysis%22">Data Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Error+of+Measurement%22">Error of Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Experimenter+Characteristics%22">Experimenter Characteristics</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring+Formulas%22">Scoring Formulas</searchLink><br /><searchLink fieldCode="DE" term="%22Correlation%22">Correlation</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Simulation%22">Computer Simulation</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Methodology%22">Research Methodology</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Practices%22">Educational Practices</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Turkey%22">Turkey</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1303-0485
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Studies based on longitudinal data focus on the change and development of the situation being investigated and allow for examining cases regarding education, individual development, cultural change, and socioeconomic improvement in time. However, as these studies require taking repeated measures in different time periods, they may include various error sources that are exclusive of the investigated variable and that change over time. In Addition, the period of observation is critical in these studies, and observant (variable) attrition or the inability to obtain measurements for various reasons (death, moving away, resignation etc.) may be observed. This study presents the coefficient of individual agreement (CIA) developed for studies in which more than one rater are needed to provide repeated scoring and examines the applications of this agreement coefficient in relation to different sample size, missing value and number of repeated measures. In this context, trial scores were generated based on different sample sizes, missing value ratios, and the number of repeated measures provided by two raters, and the expected agreement between raters was simulated as very high. Related data were replicated 1000 times. Data analysis included the mean and standard deviation of agreement ratios calculated for each combination. According to the results, independently from sample size and number of retests, the CIA takes much lower values than the expected value when random missing value rates reaches 20%. Similar conclusions can be drawn for the existence of monotone missing values except where the sample size is 30 and the number of retests are two and three. It was observed that the CIA was higher than the expected value for the data consisting of two repeated evaluations of the raters with 5% or 10% missing values when the sample size was 30. In general, the agreement coefficient did not exceed the expected level in the situation of three and four replications of measurements. As a result, it can be concluded that an increase in the replication number of measurement decreased the obtained agreement values. The obtained data showed that missing value ratio caused differentiations in the CIA. As a result, increases in missing value ratio cause inconsistencies in CIA. Therefore, it is suggested that data obtained from studies be interpreted considering missing value ratio.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 34
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2016
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1115072
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1115072
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 15
        StartPage: 1381
    Subjects:
      – SubjectFull: Investigations
        Type: general
      – SubjectFull: Sample Size
        Type: general
      – SubjectFull: Longitudinal Studies
        Type: general
      – SubjectFull: Interrater Reliability
        Type: general
      – SubjectFull: Data Analysis
        Type: general
      – SubjectFull: Error of Measurement
        Type: general
      – SubjectFull: Experimenter Characteristics
        Type: general
      – SubjectFull: Scoring Formulas
        Type: general
      – SubjectFull: Correlation
        Type: general
      – SubjectFull: Computer Simulation
        Type: general
      – SubjectFull: Research Methodology
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Educational Practices
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Turkey
        Type: general
    Titles:
      – TitleFull: Investigation of Coefficient of Individual Agreement in Terms of Sample Size, Random and Monotone Missing Ratio, and Number of Repeated Measures
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Temel, Gülhan Orekici
      – PersonEntity:
          Name:
            NameFull: Erdogan, Semra
      – PersonEntity:
          Name:
            NameFull: Selvi, Hüseyin
      – PersonEntity:
          Name:
            NameFull: Kaya, Irem Ersöz
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Type: published
              Y: 2016
          Identifiers:
            – Type: issn-print
              Value: 1303-0485
          Numbering:
            – Type: volume
              Value: 16
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Educational Sciences: Theory and Practice
              Type: main
ResultId 1