View in EDS

Automated Assessment of Domain Knowledge with Online Knowledge Mapping. CSE Technical Report 692

Saved in:
Bibliographic Details
Title: Automated Assessment of Domain Knowledge with Online Knowledge Mapping. CSE Technical Report 692
Language: English
Authors: Chung, Gregory K. W. K., Baker, Eva L., Brill, David G., National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.
Source: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 2006.
Availability: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 300 Charles E Young Drive N, GSE&IS Building 3rd Floor, Mailbox 951522, Los Angeles, CA 90095-1522. Tel: 310-206-1532; Fax: 310-825-3883; Web site: http://www.cresst.org.
Peer Reviewed: N
Page Count: 23
Publication Date: 2006
Sponsoring Agency: Office of Naval Research, Washington, DC.
Document Type: Reports - Descriptive
Descriptors: Distance Education, Skill Analysis, Knowledge Level, Prior Learning, Internet, Concept Mapping, Scoring, Military Training, Test Validity, Test Construction, Evaluation Methods, Computer Software Evaluation, Computer Uses in Education, Programming, Computer Interfaces, Computer Software, Measures (Individuals)
Abstract: A critical first step in developing training systems is gathering quality information about a trainee's competency in a skill or knowledge domain. Such information includes an estimate of what the trainee knows prior to training, how much has been learned from training, how well the trainee may perform in future task situations, and whether to recommend remediation to bolster the trainee's knowledge. This paper describes the design, development, testing, and application of a Web-based tool designed to assess a trainee's understanding of a content domain in a distributed learning environment. The tool, called the CRESST Human Performance Knowledge Mapping Tool (HPKMT), enables trainees to express their understanding of a content area by creating graphical, network representations of concepts and links that define the relationships of concepts. Knowledge mappers have been used for several years, almost always as an aid for organizing information in support of problem solving or in instructional applications. To use knowledge maps as assessments there must be a reliable scoring method and there must be evidence for the validity of scores produced by the method. Further, to be practical in a distributed learning environment, the scoring should be automated. The HPKMT provides automated, reliable, and valid scoring, and its functionality and scoring method have been built from a base of empirical research. We review and evaluate alternative knowledge mapping scoring methods and online mapping systems. We then describe the overall design approach, functionality, scoring method, usability testing, and authoring capabilities of the CRESST HPKMT. The paper ends with descriptions of applications of the HPKMT to military training, limitations of the system, and next steps. (Contains 2 tables and 3 figures.)
Abstractor: Author
Number of References: 24
Entry Date: 2006
Accession Number: ED492869
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED492869
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED492869
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Automated Assessment of Domain Knowledge with Online Knowledge Mapping. CSE Technical Report 692
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Chung%2C+Gregory+K%2E+W%2E+K%2E%22">Chung, Gregory K. W. K.</searchLink><br /><searchLink fieldCode="AR" term="%22Baker%2C+Eva+L%2E%22">Baker, Eva L.</searchLink><br /><searchLink fieldCode="AR" term="%22Brill%2C+David+G%2E%22">Brill, David G.</searchLink><br /><searchLink fieldCode="AR" term="%22National+Center+for+Research+on+Evaluation%2C+Standards%2C+and+Student+Testing%2C+Los+Angeles%2C+CA%2E%22">National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22National+Center+for+Research+on+Evaluation%2C+Standards%2C+and+Student+Testing+%28CRESST%29%22"><i>National Center for Research on Evaluation, Standards, and Student Testing (CRESST)</i></searchLink>. 2006.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 300 Charles E Young Drive N, GSE&IS Building 3rd Floor, Mailbox 951522, Los Angeles, CA 90095-1522. Tel: 310-206-1532; Fax: 310-825-3883; Web site: http://www.cresst.org.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 23
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2006
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Office of Naval Research, Washington, DC.
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Descriptive
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Distance+Education%22">Distance Education</searchLink><br /><searchLink fieldCode="DE" term="%22Skill+Analysis%22">Skill Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Knowledge+Level%22">Knowledge Level</searchLink><br /><searchLink fieldCode="DE" term="%22Prior+Learning%22">Prior Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Internet%22">Internet</searchLink><br /><searchLink fieldCode="DE" term="%22Concept+Mapping%22">Concept Mapping</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring%22">Scoring</searchLink><br /><searchLink fieldCode="DE" term="%22Military+Training%22">Military Training</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software+Evaluation%22">Computer Software Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Uses+in+Education%22">Computer Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Programming%22">Programming</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Interfaces%22">Computer Interfaces</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink><br /><searchLink fieldCode="DE" term="%22Measures+%28Individuals%29%22">Measures (Individuals)</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: A critical first step in developing training systems is gathering quality information about a trainee's competency in a skill or knowledge domain. Such information includes an estimate of what the trainee knows prior to training, how much has been learned from training, how well the trainee may perform in future task situations, and whether to recommend remediation to bolster the trainee's knowledge. This paper describes the design, development, testing, and application of a Web-based tool designed to assess a trainee's understanding of a content domain in a distributed learning environment. The tool, called the CRESST Human Performance Knowledge Mapping Tool (HPKMT), enables trainees to express their understanding of a content area by creating graphical, network representations of concepts and links that define the relationships of concepts. Knowledge mappers have been used for several years, almost always as an aid for organizing information in support of problem solving or in instructional applications. To use knowledge maps as assessments there must be a reliable scoring method and there must be evidence for the validity of scores produced by the method. Further, to be practical in a distributed learning environment, the scoring should be automated. The HPKMT provides automated, reliable, and valid scoring, and its functionality and scoring method have been built from a base of empirical research. We review and evaluate alternative knowledge mapping scoring methods and online mapping systems. We then describe the overall design approach, functionality, scoring method, usability testing, and authoring capabilities of the CRESST HPKMT. The paper ends with descriptions of applications of the HPKMT to military training, limitations of the system, and next steps. (Contains 2 tables and 3 figures.)
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: Author
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 24
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2006
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED492869
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED492869
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 23
    Subjects:
      – SubjectFull: Distance Education
        Type: general
      – SubjectFull: Skill Analysis
        Type: general
      – SubjectFull: Knowledge Level
        Type: general
      – SubjectFull: Prior Learning
        Type: general
      – SubjectFull: Internet
        Type: general
      – SubjectFull: Concept Mapping
        Type: general
      – SubjectFull: Scoring
        Type: general
      – SubjectFull: Military Training
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
      – SubjectFull: Computer Software Evaluation
        Type: general
      – SubjectFull: Computer Uses in Education
        Type: general
      – SubjectFull: Programming
        Type: general
      – SubjectFull: Computer Interfaces
        Type: general
      – SubjectFull: Computer Software
        Type: general
      – SubjectFull: Measures (Individuals)
        Type: general
    Titles:
      – TitleFull: Automated Assessment of Domain Knowledge with Online Knowledge Mapping. CSE Technical Report 692
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.
      – PersonEntity:
          Name:
            NameFull: Chung, Gregory K. W. K.
      – PersonEntity:
          Name:
            NameFull: Baker, Eva L.
      – PersonEntity:
          Name:
            NameFull: Brill, David G.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Type: published
              Y: 2006
          Titles:
            – TitleFull: National Center for Research on Evaluation, Standards, and Student Testing (CRESST)
              Type: main
ResultId 1