View in EDS

Assessing the quality of evidence in studies estimating prevalence of exposure to occupational risk factors: The QoE-SPEO approach applied in the systematic reviews from the WHO/ILO Joint Estimates of the Work-related burden of disease and Injury.

Saved in:
Bibliographic Details
Title: Assessing the quality of evidence in studies estimating prevalence of exposure to occupational risk factors: The QoE-SPEO approach applied in the systematic reviews from the WHO/ILO Joint Estimates of the Work-related burden of disease and Injury.
Authors: Pega, Frank1 (AUTHOR) pegaf@who.int, Momen, Natalie C.1 (AUTHOR), Gagliardi, Diana2 (AUTHOR), Bero, Lisa A.3,4 (AUTHOR), Boccuni, Fabio2 (AUTHOR), Chartres, Nicholas5 (AUTHOR), Descatha, Alexis6,7 (AUTHOR), Dzhambov, Angel M.8,9 (AUTHOR), Godderis, Lode10,11 (AUTHOR), Loney, Tom12 (AUTHOR), Mandrioli, Daniele13 (AUTHOR), Modenese, Alberto14 (AUTHOR), van der Molen, Henk F.15 (AUTHOR), Morgan, Rebecca L.16 (AUTHOR), Neupane, Subas17 (AUTHOR), Pachito, Daniela18,19 (AUTHOR), Paulo, Marilia S.20,21 (AUTHOR), Prakash, K.C.17 (AUTHOR), Scheepers, Paul T.J.22 (AUTHOR), Teixeira, Liliane23 (AUTHOR)
Source: Environment International. Mar2022, Vol. 161, pN.PAG-N.PAG. 1p.
Subject Terms: *Industrial hygiene, *Environmental health, Occupational exposure, Risk exposure, Wounds & injuries
Company/Entity: International Labour Organisation , World Health Organization
Abstract: [Display omitted] The World Health Organization (WHO) and the International Labour Organization (ILO) have produced the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates). For these, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors have been conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methodology is to assess the quality of evidence across studies. In this article, we present the approach applied in these WHO/ILO systematic reviews for performing such assessments on studies of prevalence of exposure. It is called the Q uality o f E vidence in St udies estimating P revalence of E xposure to O ccupational risk factors (QoE-SPEO) approach. We describe QoE-SPEO's development to date, demonstrate its feasibility reporting results from pilot testing and case studies, note its strengths and limitations, and suggest how QoE-SPEO should be tested and developed further. Following a comprehensive literature review, and using expert opinion, selected existing quality of evidence assessment approaches used in environmental and occupational health were reviewed and analysed for their relevance to prevalence studies. Relevant steps and components from the existing approaches were adopted or adapted for QoE-SPEO. New steps and components were developed. We elicited feedback from other systematic review methodologists and exposure scientists and reached consensus on the QoE-SPEO approach. Ten individual experts pilot-tested QoE-SPEO. To assess inter-rater agreement, we counted ratings of expected (actual and non-spurious) heterogeneity and quality of evidence and calculated a raw measure of agreement (P i) between individual raters and rater teams for the downgrade domains. P i ranged between 0.00 (no two pilot testers selected the same rating) and 1.00 (all pilot testers selected the same rating). Case studies were conducted of experiences of QoE-SPEO's use in two WHO/ILO systematic reviews. We found no existing quality of evidence assessment approach for occupational exposure prevalence studies. We identified three relevant, existing approaches for environmental and occupational health studies of the effect of exposures. Assessments using QoE-SPEO comprise three steps: (1) judge the level of expected heterogeneity (defined as non-spurious variability that can be expected in exposure prevalence, within or between individual persons, because exposure may change over space and/or time), (2) assess downgrade domains, and (3) reach a final rating on the quality of evidence. Assessments are conducted using the same five downgrade domains as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach: (a) risk of bias, (b) indirectness, (c) inconsistency, (d) imprecision, and (e) publication bias. For downgrade domains (c) and (d), the assessment varies depending on the level of expected heterogeneity. There are no upgrade domains. The QoE-SPEO's ratings are "very low", "low", "moderate", and "high". To arrive at a final decision on the overall quality of evidence, the assessor starts at "high" quality of evidence and for each domain downgrades by one or two levels for serious concerns or very serious concerns, respectively. In pilot tests, there was reasonable agreement in ratings for expected heterogeneity; 70% of raters selected the same rating. Inter-rater agreement ranged considerably between downgrade domains, both for individual rater pairs (range P i : 0.36–1.00) and rater teams (0.20–1.00). Sparse data prevented rigorous assessment of inter-rater agreement in quality of evidence ratings. We present QoE-SPEO as an approach for assessing quality of evidence in prevalence studies of exposure to occupational risk factors. It has been developed to its current version (as presented here), has undergone pilot testing, and was applied in the systematic reviews for the WHO/ILO Joint Estimates. While the approach requires further testing and development, it makes steps towards filling an identified gap, and progress made so far can be used to inform future work in this area. [ABSTRACT FROM AUTHOR]
Copyright of Environment International is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: GreenFILE
FullText Text:
  Availability: 0
Header DbId: 8gh
DbLabel: GreenFILE
An: 155458386
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Assessing the quality of evidence in studies estimating prevalence of exposure to occupational risk factors: The QoE-SPEO approach applied in the systematic reviews from the WHO/ILO Joint Estimates of the Work-related burden of disease and Injury.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Pega%2C+Frank%22">Pega, Frank</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> pegaf@who.int</i><br /><searchLink fieldCode="AR" term="%22Momen%2C+Natalie+C%2E%22">Momen, Natalie C.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Gagliardi%2C+Diana%22">Gagliardi, Diana</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Bero%2C+Lisa+A%2E%22">Bero, Lisa A.</searchLink><relatesTo>3,4</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Boccuni%2C+Fabio%22">Boccuni, Fabio</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Chartres%2C+Nicholas%22">Chartres, Nicholas</searchLink><relatesTo>5</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Descatha%2C+Alexis%22">Descatha, Alexis</searchLink><relatesTo>6,7</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Dzhambov%2C+Angel+M%2E%22">Dzhambov, Angel M.</searchLink><relatesTo>8,9</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Godderis%2C+Lode%22">Godderis, Lode</searchLink><relatesTo>10,11</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Loney%2C+Tom%22">Loney, Tom</searchLink><relatesTo>12</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Mandrioli%2C+Daniele%22">Mandrioli, Daniele</searchLink><relatesTo>13</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Modenese%2C+Alberto%22">Modenese, Alberto</searchLink><relatesTo>14</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22van+der+Molen%2C+Henk+F%2E%22">van der Molen, Henk F.</searchLink><relatesTo>15</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Morgan%2C+Rebecca+L%2E%22">Morgan, Rebecca L.</searchLink><relatesTo>16</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Neupane%2C+Subas%22">Neupane, Subas</searchLink><relatesTo>17</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Pachito%2C+Daniela%22">Pachito, Daniela</searchLink><relatesTo>18,19</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Paulo%2C+Marilia+S%2E%22">Paulo, Marilia S.</searchLink><relatesTo>20,21</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Prakash%2C+K%2EC%2E%22">Prakash, K.C.</searchLink><relatesTo>17</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Scheepers%2C+Paul+T%2EJ%2E%22">Scheepers, Paul T.J.</searchLink><relatesTo>22</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Teixeira%2C+Liliane%22">Teixeira, Liliane</searchLink><relatesTo>23</relatesTo> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Environment+International%22">Environment International</searchLink>. Mar2022, Vol. 161, pN.PAG-N.PAG. 1p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Industrial+hygiene%22">Industrial hygiene</searchLink><br />*<searchLink fieldCode="DE" term="%22Environmental+health%22">Environmental health</searchLink><br /><searchLink fieldCode="DE" term="%22Occupational+exposure%22">Occupational exposure</searchLink><br /><searchLink fieldCode="DE" term="%22Risk+exposure%22">Risk exposure</searchLink><br /><searchLink fieldCode="DE" term="%22Wounds+%26+injuries%22">Wounds & injuries</searchLink>
– Name: SubjectCompany
  Label: Company/Entity
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22International+Labour+Organisation%22">International Labour Organisation</searchLink> <br /><searchLink fieldCode="DE" term="%22World+Health+Organization%22">World Health Organization</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: [Display omitted] The World Health Organization (WHO) and the International Labour Organization (ILO) have produced the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates). For these, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors have been conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methodology is to assess the quality of evidence across studies. In this article, we present the approach applied in these WHO/ILO systematic reviews for performing such assessments on studies of prevalence of exposure. It is called the Q uality o f E vidence in St udies estimating P revalence of E xposure to O ccupational risk factors (QoE-SPEO) approach. We describe QoE-SPEO's development to date, demonstrate its feasibility reporting results from pilot testing and case studies, note its strengths and limitations, and suggest how QoE-SPEO should be tested and developed further. Following a comprehensive literature review, and using expert opinion, selected existing quality of evidence assessment approaches used in environmental and occupational health were reviewed and analysed for their relevance to prevalence studies. Relevant steps and components from the existing approaches were adopted or adapted for QoE-SPEO. New steps and components were developed. We elicited feedback from other systematic review methodologists and exposure scientists and reached consensus on the QoE-SPEO approach. Ten individual experts pilot-tested QoE-SPEO. To assess inter-rater agreement, we counted ratings of expected (actual and non-spurious) heterogeneity and quality of evidence and calculated a raw measure of agreement (P i) between individual raters and rater teams for the downgrade domains. P i ranged between 0.00 (no two pilot testers selected the same rating) and 1.00 (all pilot testers selected the same rating). Case studies were conducted of experiences of QoE-SPEO's use in two WHO/ILO systematic reviews. We found no existing quality of evidence assessment approach for occupational exposure prevalence studies. We identified three relevant, existing approaches for environmental and occupational health studies of the effect of exposures. Assessments using QoE-SPEO comprise three steps: (1) judge the level of expected heterogeneity (defined as non-spurious variability that can be expected in exposure prevalence, within or between individual persons, because exposure may change over space and/or time), (2) assess downgrade domains, and (3) reach a final rating on the quality of evidence. Assessments are conducted using the same five downgrade domains as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach: (a) risk of bias, (b) indirectness, (c) inconsistency, (d) imprecision, and (e) publication bias. For downgrade domains (c) and (d), the assessment varies depending on the level of expected heterogeneity. There are no upgrade domains. The QoE-SPEO's ratings are "very low", "low", "moderate", and "high". To arrive at a final decision on the overall quality of evidence, the assessor starts at "high" quality of evidence and for each domain downgrades by one or two levels for serious concerns or very serious concerns, respectively. In pilot tests, there was reasonable agreement in ratings for expected heterogeneity; 70% of raters selected the same rating. Inter-rater agreement ranged considerably between downgrade domains, both for individual rater pairs (range P i : 0.36–1.00) and rater teams (0.20–1.00). Sparse data prevented rigorous assessment of inter-rater agreement in quality of evidence ratings. We present QoE-SPEO as an approach for assessing quality of evidence in prevalence studies of exposure to occupational risk factors. It has been developed to its current version (as presented here), has undergone pilot testing, and was applied in the systematic reviews for the WHO/ILO Joint Estimates. While the approach requires further testing and development, it makes steps towards filling an identified gap, and progress made so far can be used to inform future work in this area. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Environment International is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=8gh&AN=155458386
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.envint.2022.107136
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 1
        StartPage: N.PAG
    Subjects:
      – SubjectFull: Industrial hygiene
        Type: general
      – SubjectFull: Environmental health
        Type: general
      – SubjectFull: Occupational exposure
        Type: general
      – SubjectFull: Risk exposure
        Type: general
      – SubjectFull: Wounds & injuries
        Type: general
      – SubjectFull: International Labour Organisation
        Type: general
      – SubjectFull: World Health Organization
        Type: general
    Titles:
      – TitleFull: Assessing the quality of evidence in studies estimating prevalence of exposure to occupational risk factors: The QoE-SPEO approach applied in the systematic reviews from the WHO/ILO Joint Estimates of the Work-related burden of disease and Injury.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Pega, Frank
      – PersonEntity:
          Name:
            NameFull: Momen, Natalie C.
      – PersonEntity:
          Name:
            NameFull: Gagliardi, Diana
      – PersonEntity:
          Name:
            NameFull: Bero, Lisa A.
      – PersonEntity:
          Name:
            NameFull: Boccuni, Fabio
      – PersonEntity:
          Name:
            NameFull: Chartres, Nicholas
      – PersonEntity:
          Name:
            NameFull: Descatha, Alexis
      – PersonEntity:
          Name:
            NameFull: Dzhambov, Angel M.
      – PersonEntity:
          Name:
            NameFull: Godderis, Lode
      – PersonEntity:
          Name:
            NameFull: Loney, Tom
      – PersonEntity:
          Name:
            NameFull: Mandrioli, Daniele
      – PersonEntity:
          Name:
            NameFull: Modenese, Alberto
      – PersonEntity:
          Name:
            NameFull: van der Molen, Henk F.
      – PersonEntity:
          Name:
            NameFull: Morgan, Rebecca L.
      – PersonEntity:
          Name:
            NameFull: Neupane, Subas
      – PersonEntity:
          Name:
            NameFull: Pachito, Daniela
      – PersonEntity:
          Name:
            NameFull: Paulo, Marilia S.
      – PersonEntity:
          Name:
            NameFull: Prakash, K.C.
      – PersonEntity:
          Name:
            NameFull: Scheepers, Paul T.J.
      – PersonEntity:
          Name:
            NameFull: Teixeira, Liliane
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Text: Mar2022
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 01604120
          Numbering:
            – Type: volume
              Value: 161
          Titles:
            – TitleFull: Environment International
              Type: main
ResultId 1