View in EDS PDF Full Text

A Framework for Measuring Relevancy in Discovery Environments: Increasing Scalability and Reproducibility.

Saved in:
Bibliographic Details
Title: A Framework for Measuring Relevancy in Discovery Environments: Increasing Scalability and Reproducibility.
Authors: Galbreath, Blake1 blake.galbreath@wsu.edu, Merrill, Alex2 merrilla@wsu.edu, Johnson, Corey M.3 coreyj@wsu.edu
Source: Information Technology & Libraries. Jun2024, Vol. 43 Issue 2, p1-18. 18p.
Subject Terms: *World Wide Web, *Computer software, *Citation analysis, *Newspapers, *Students, *Bibliographical citations, *Books, *Bibliography, *Automation, Serial publications, Ecology, Seasons, Research evaluation, Medical research, Data analysis software
Abstract: Institutional discovery environments now serve as central resource databases for researchers in the academic environment. Over the last several decades, there have been numerous discovery layer research inquiries centering primarily on user satisfaction measures of discovery system effectiveness. This study focuses on the creation of a largely automated method for evaluating discovery layer quality, utilizing the bibliographic sources from student research projects. Building on past research, the current study replaces a semiautomated Excel Fuzzy Lookup Add-In process with a fully scripted R-based approach, which employs the stringdist R package and applies the Jaro-Winkler distance metric as the matching evaluator. The researchers consider the error rate incurred by relying solely on an automated matching metric. They also use Open Refine for normalization processes and package the tools together on an OSF site for other institutions to use. Since the R-based approach does not require special processing or time and can be reproduced with minimal effort, it will allow future studies and users of our method to capture larger sample sizes, boosting validity. While the assessment process has been streamlined and shows promise, there remain issues in establishing solid connections between research paper bibliographies and discovery layer use. Subsequent research will focus on creating alternatives to paper titles as search proxies that better resemble genuine information-seeking behavior and comparing undergraduate and graduate student interactions within discovery environments. [ABSTRACT FROM AUTHOR]
Copyright of Information Technology & Libraries is the property of American Library Association and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: ehh
DbLabel: Education Research Complete
An: 178011113
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Framework for Measuring Relevancy in Discovery Environments: Increasing Scalability and Reproducibility.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Galbreath%2C+Blake%22">Galbreath, Blake</searchLink><relatesTo>1</relatesTo><i> blake.galbreath@wsu.edu</i><br /><searchLink fieldCode="AR" term="%22Merrill%2C+Alex%22">Merrill, Alex</searchLink><relatesTo>2</relatesTo><i> merrilla@wsu.edu</i><br /><searchLink fieldCode="AR" term="%22Johnson%2C+Corey+M%2E%22">Johnson, Corey M.</searchLink><relatesTo>3</relatesTo><i> coreyj@wsu.edu</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Information+Technology+%26+Libraries%22">Information Technology & Libraries</searchLink>. Jun2024, Vol. 43 Issue 2, p1-18. 18p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22World+Wide+Web%22">World Wide Web</searchLink><br />*<searchLink fieldCode="DE" term="%22Computer+software%22">Computer software</searchLink><br />*<searchLink fieldCode="DE" term="%22Citation+analysis%22">Citation analysis</searchLink><br />*<searchLink fieldCode="DE" term="%22Newspapers%22">Newspapers</searchLink><br />*<searchLink fieldCode="DE" term="%22Students%22">Students</searchLink><br />*<searchLink fieldCode="DE" term="%22Bibliographical+citations%22">Bibliographical citations</searchLink><br />*<searchLink fieldCode="DE" term="%22Books%22">Books</searchLink><br />*<searchLink fieldCode="DE" term="%22Bibliography%22">Bibliography</searchLink><br />*<searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Serial+publications%22">Serial publications</searchLink><br /><searchLink fieldCode="DE" term="%22Ecology%22">Ecology</searchLink><br /><searchLink fieldCode="DE" term="%22Seasons%22">Seasons</searchLink><br /><searchLink fieldCode="DE" term="%22Research+evaluation%22">Research evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+research%22">Medical research</searchLink><br /><searchLink fieldCode="DE" term="%22Data+analysis+software%22">Data analysis software</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Institutional discovery environments now serve as central resource databases for researchers in the academic environment. Over the last several decades, there have been numerous discovery layer research inquiries centering primarily on user satisfaction measures of discovery system effectiveness. This study focuses on the creation of a largely automated method for evaluating discovery layer quality, utilizing the bibliographic sources from student research projects. Building on past research, the current study replaces a semiautomated Excel Fuzzy Lookup Add-In process with a fully scripted R-based approach, which employs the stringdist R package and applies the Jaro-Winkler distance metric as the matching evaluator. The researchers consider the error rate incurred by relying solely on an automated matching metric. They also use Open Refine for normalization processes and package the tools together on an OSF site for other institutions to use. Since the R-based approach does not require special processing or time and can be reproduced with minimal effort, it will allow future studies and users of our method to capture larger sample sizes, boosting validity. While the assessment process has been streamlined and shows promise, there remain issues in establishing solid connections between research paper bibliographies and discovery layer use. Subsequent research will focus on creating alternatives to paper titles as search proxies that better resemble genuine information-seeking behavior and comparing undergraduate and graduate student interactions within discovery environments. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Information Technology & Libraries is the property of American Library Association and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=178011113
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.5860/ital.v43i2.16915
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
        StartPage: 1
    Subjects:
      – SubjectFull: World Wide Web
        Type: general
      – SubjectFull: Computer software
        Type: general
      – SubjectFull: Citation analysis
        Type: general
      – SubjectFull: Newspapers
        Type: general
      – SubjectFull: Students
        Type: general
      – SubjectFull: Bibliographical citations
        Type: general
      – SubjectFull: Books
        Type: general
      – SubjectFull: Bibliography
        Type: general
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Serial publications
        Type: general
      – SubjectFull: Ecology
        Type: general
      – SubjectFull: Seasons
        Type: general
      – SubjectFull: Research evaluation
        Type: general
      – SubjectFull: Medical research
        Type: general
      – SubjectFull: Data analysis software
        Type: general
    Titles:
      – TitleFull: A Framework for Measuring Relevancy in Discovery Environments: Increasing Scalability and Reproducibility.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Galbreath, Blake
      – PersonEntity:
          Name:
            NameFull: Merrill, Alex
      – PersonEntity:
          Name:
            NameFull: Johnson, Corey M.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 06
              Text: Jun2024
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 07309295
          Numbering:
            – Type: volume
              Value: 43
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Information Technology & Libraries
              Type: main
ResultId 1