Identifying empirical studies for mixed studies reviews: The mixed filter and the automated text classifier.
Saved in:
| Title: | Identifying empirical studies for mixed studies reviews: The mixed filter and the automated text classifier. |
|---|---|
| Authors: | El Sherif, Reem1 reem.elsherif@mail.mcgill.ca, Langlois, Alexis2, Pandu, Xiao2, Nie, Jian-Yun2, Thomas, James3, Hong, Quan Nha3, Pluye, Pierre1, Granikov, Vera, Pluye, Piere |
| Source: | Education for Information. 2020, Vol. 36 Issue 1, p101-105. 5p. 1 Diagram. |
| Subject Terms: | *Algorithms, *Bibliographic databases, Empirical research, Keywords, Decision trees |
| Abstract: | Mixed studies reviews include empirical studies with diverse designs (qualitative, quantitative and mixed methods). To make the process of identifying relevant empirical studies for such reviews more efficient, we developed a mixed filter that included different keywords and subject headings for quantitative (e.g., cohort study), qualitative (e.g., focus group), and mixed methods studies. It was tested for six journals from three disciplines. We measured precision (proportion of retrieved documents being relevant), sensitivity (proportion of relevant documents retrieved), and specificity (proportion of non-relevant documents not retrieved). Records were coded before applying the filter and compared with retrieved records, and descriptive statistics were performed, suggesting the mixed filter has high sensitivity, but lower precision and specificity (close to 50%). Next, based on the success of the filter, we developed an automated text classification system that can automatically select empirical studies in order to facilitate systematic mixed studies reviews. Several algorithms were trained and validated with 8,050 database records that were previously manually categorized. Decision trees had the best results and surpassed the accuracy of the filter by 30% when using full-text documents. This algorithm was then adapted into an online format that can be used by researchers to analyze their bibliography and categorize records into "empirical" and "nonempirical". [ABSTRACT FROM AUTHOR] |
| Copyright of Education for Information is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 142635054 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Identifying empirical studies for mixed studies reviews: The mixed filter and the automated text classifier. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22El+Sherif%2C+Reem%22">El Sherif, Reem</searchLink><relatesTo>1</relatesTo><i> reem.elsherif@mail.mcgill.ca</i><br /><searchLink fieldCode="AR" term="%22Langlois%2C+Alexis%22">Langlois, Alexis</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Pandu%2C+Xiao%22">Pandu, Xiao</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Nie%2C+Jian-Yun%22">Nie, Jian-Yun</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Thomas%2C+James%22">Thomas, James</searchLink><relatesTo>3</relatesTo><br /><searchLink fieldCode="AR" term="%22Hong%2C+Quan+Nha%22">Hong, Quan Nha</searchLink><relatesTo>3</relatesTo><br /><searchLink fieldCode="AR" term="%22Pluye%2C+Pierre%22">Pluye, Pierre</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Granikov%2C+Vera%22">Granikov, Vera</searchLink><br /><searchLink fieldCode="AR" term="%22Pluye%2C+Piere%22">Pluye, Piere</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Education+for+Information%22">Education for Information</searchLink>. 2020, Vol. 36 Issue 1, p101-105. 5p. 1 Diagram. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Algorithms%22">Algorithms</searchLink><br />*<searchLink fieldCode="DE" term="%22Bibliographic+databases%22">Bibliographic databases</searchLink><br /><searchLink fieldCode="DE" term="%22Empirical+research%22">Empirical research</searchLink><br /><searchLink fieldCode="DE" term="%22Keywords%22">Keywords</searchLink><br /><searchLink fieldCode="DE" term="%22Decision+trees%22">Decision trees</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Mixed studies reviews include empirical studies with diverse designs (qualitative, quantitative and mixed methods). To make the process of identifying relevant empirical studies for such reviews more efficient, we developed a mixed filter that included different keywords and subject headings for quantitative (e.g., cohort study), qualitative (e.g., focus group), and mixed methods studies. It was tested for six journals from three disciplines. We measured precision (proportion of retrieved documents being relevant), sensitivity (proportion of relevant documents retrieved), and specificity (proportion of non-relevant documents not retrieved). Records were coded before applying the filter and compared with retrieved records, and descriptive statistics were performed, suggesting the mixed filter has high sensitivity, but lower precision and specificity (close to 50%). Next, based on the success of the filter, we developed an automated text classification system that can automatically select empirical studies in order to facilitate systematic mixed studies reviews. Several algorithms were trained and validated with 8,050 database records that were previously manually categorized. Decision trees had the best results and surpassed the accuracy of the filter by 30% when using full-text documents. This algorithm was then adapted into an online format that can be used by researchers to analyze their bibliography and categorize records into "empirical" and "nonempirical". [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Education for Information is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=142635054 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.3233/EFI-190347 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 5 StartPage: 101 Subjects: – SubjectFull: Algorithms Type: general – SubjectFull: Bibliographic databases Type: general – SubjectFull: Empirical research Type: general – SubjectFull: Keywords Type: general – SubjectFull: Decision trees Type: general Titles: – TitleFull: Identifying empirical studies for mixed studies reviews: The mixed filter and the automated text classifier. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: El Sherif, Reem – PersonEntity: Name: NameFull: Langlois, Alexis – PersonEntity: Name: NameFull: Pandu, Xiao – PersonEntity: Name: NameFull: Nie, Jian-Yun – PersonEntity: Name: NameFull: Thomas, James – PersonEntity: Name: NameFull: Hong, Quan Nha – PersonEntity: Name: NameFull: Pluye, Pierre – PersonEntity: Name: NameFull: Granikov, Vera – PersonEntity: Name: NameFull: Pluye, Piere IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Text: 2020 Type: published Y: 2020 Identifiers: – Type: issn-print Value: 01678329 Numbering: – Type: volume Value: 36 – Type: issue Value: 1 Titles: – TitleFull: Education for Information Type: main |
| ResultId | 1 |
