View in EDS

The "Free lunches" index for assessing academics: a not entirely serious proposal.

Saved in:
Bibliographic Details
Title: The "Free lunches" index for assessing academics: a not entirely serious proposal.
Authors: Scanff, Alexandre1 (AUTHOR), Mauhe, Nicolas2 (AUTHOR), Taburet, Marion1 (AUTHOR), Savourat, Pierre-Etienne3 (AUTHOR), Clément, Thomas3 (AUTHOR), Bastian, Benjamin3,4 (AUTHOR), Cristea, Ioana5 (AUTHOR), Braillon, Alain6 (AUTHOR), Carayol, Nicolas2 (AUTHOR), Naudet, Florian1,7 (AUTHOR) floriannaudet@gmail.com
Source: Scientometrics. Dec2023, Vol. 128 Issue 12, p6761-6772. 12p.
Subjects: Cie. Générale des Établissements Michelin SCA, Surgery, Physicians, Pearson correlation (Statistics), Luncheons, Medical research, Medical informatics
Abstract: Background: Evaluating academics is a challenge, and the use of indicators such as scientific impact (i.e. number of published papers and their citation rate) is complex and poorly validated. We propose a new indicator for academic medical research: the "Free lunches" index (fl-index), computed from the sum of gifts from the industry. The fl-index provides a direct and straightforward measure of industry investment consisting in regaling a clinical researcher with rewards like a leisurely meal in a Michelin-starred restaurant or a relaxing stay in a high-end resort hotel. Methods and findings: 3,936 French academics were included in this observational and satirical retrospective study using the French database registering gifts received by medical doctors and Web of Science, over the years 2014–2019. Pearson's correlation coefficients explored the associations between the fl-index and in the h-index (the maximum number of published papers h that have each been cited at least h times) increase over the period 2014–2019. The diagnostic properties and optimal thresholds of the fl-index for detecting high scientific productivity were explored. High scientific productivity was defined as ranking in the top 25% scientists in terms of increase in the h-index. To detect possible differences according to medical disciplines, subgroup analyses were performed. The correlation coefficient between the fl-index and the increase in the h-index was 0.31 (95% CI 0.29 to 0.34). The optimal threshold was 7,700 € for the fl-index, giving a sensitivity of 65% (95% CI 61 to 67%), a specificity of 59% (95% CI 57 to 61%). However, there were considerable differences across medical disciplines, with correlations ranging from 0.12 (Morphology and morphogenesis) to 0.51 (Internal medicine, geriatrics, general surgery and general medicine), and the median fl-index ranging from 37 € (Public health, environment and society) to 30 404 € (Cardiorespiratory and vascular pathologies). Importantly, the highest correlations and values for the fl-index were observed for clinical disciplines. Conclusions: Overall, the correlation between the fl-index and an increase in the h-index was modest so that the fl-index cannot be used as a surrogate for academic success as gauged by productivity-based metrics. However, future residents could use these results to complement the usual metrics in order to choose (or avoid) certain specialties, depending on whether they are more eager to produce scientific articles or to enjoy an affluent lifestyle that they consider well-deserved. Registration: osf.io/7d4bk. [ABSTRACT FROM AUTHOR]
Copyright of Scientometrics is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Background: Evaluating academics is a challenge, and the use of indicators such as scientific impact (i.e. number of published papers and their citation rate) is complex and poorly validated. We propose a new indicator for academic medical research: the "Free lunches" index (fl-index), computed from the sum of gifts from the industry. The fl-index provides a direct and straightforward measure of industry investment consisting in regaling a clinical researcher with rewards like a leisurely meal in a Michelin-starred restaurant or a relaxing stay in a high-end resort hotel. Methods and findings: 3,936 French academics were included in this observational and satirical retrospective study using the French database registering gifts received by medical doctors and Web of Science, over the years 2014–2019. Pearson's correlation coefficients explored the associations between the fl-index and in the h-index (the maximum number of published papers h that have each been cited at least h times) increase over the period 2014–2019. The diagnostic properties and optimal thresholds of the fl-index for detecting high scientific productivity were explored. High scientific productivity was defined as ranking in the top 25% scientists in terms of increase in the h-index. To detect possible differences according to medical disciplines, subgroup analyses were performed. The correlation coefficient between the fl-index and the increase in the h-index was 0.31 (95% CI 0.29 to 0.34). The optimal threshold was 7,700 € for the fl-index, giving a sensitivity of 65% (95% CI 61 to 67%), a specificity of 59% (95% CI 57 to 61%). However, there were considerable differences across medical disciplines, with correlations ranging from 0.12 (Morphology and morphogenesis) to 0.51 (Internal medicine, geriatrics, general surgery and general medicine), and the median fl-index ranging from 37 € (Public health, environment and society) to 30 404 € (Cardiorespiratory and vascular pathologies). Importantly, the highest correlations and values for the fl-index were observed for clinical disciplines. Conclusions: Overall, the correlation between the fl-index and an increase in the h-index was modest so that the fl-index cannot be used as a surrogate for academic success as gauged by productivity-based metrics. However, future residents could use these results to complement the usual metrics in order to choose (or avoid) certain specialties, depending on whether they are more eager to produce scientific articles or to enjoy an affluent lifestyle that they consider well-deserved. Registration: osf.io/7d4bk. [ABSTRACT FROM AUTHOR]
ISSN:01389130
DOI:10.1007/s11192-023-04862-8