Applying the criteria indicators and metrics evaluation method on ICT research: The HUMBOLDT project.

Saved in:

Bibliographic Details
Title:	Applying the criteria indicators and metrics evaluation method on ICT research: The HUMBOLDT project.
Authors:	Khan, Zaheer¹, Ludlow, David¹, Rix, Joachim¹
Source:	Research Evaluation. Mar2013, Vol. 22 Issue 1, p30-40. 11p.
Subjects:	Technology indicators, Information & communication technologies, Methodology, Geospatial data, Technological innovations, Evaluation methodology
Abstract:	Evaluating an Information Technology (IT) based research and development project from an end-user’s perspective is necessary in order to discover whether or not the outcomes of the project are successful. However, it is not straightforward to design and implement a generic user evaluation methodology that can be applied to a large-scale collaborative integrated research and development project. In this article, we share our experience based on the research evaluation carried out in a large-scale integrated ICT research project—The HUMBOLDT project. The project develops IT tools and services for cross-border geospatial data harmonization, which are applied to nine different application scenarios. A novel user-oriented evaluation methodology, namely—Criteria Indicators and Metrics (CIM)—is defined and utilized for the evaluation of the project research outcomes. The user evaluation results reveal the extent to which the research and development objectives of the project are achieved, and also demonstrate the benefits and limitations of the evaluation methodology. The results of this article can be usefully applied to other collaborative research projects in a variety of domains. [ABSTRACT FROM AUTHOR]
	Copyright of Research Evaluation is the property of Oxford University Press / USA and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	Evaluating an Information Technology (IT) based research and development project from an end-user’s perspective is necessary in order to discover whether or not the outcomes of the project are successful. However, it is not straightforward to design and implement a generic user evaluation methodology that can be applied to a large-scale collaborative integrated research and development project. In this article, we share our experience based on the research evaluation carried out in a large-scale integrated ICT research project—The HUMBOLDT project. The project develops IT tools and services for cross-border geospatial data harmonization, which are applied to nine different application scenarios. A novel user-oriented evaluation methodology, namely—Criteria Indicators and Metrics (CIM)—is defined and utilized for the evaluation of the project research outcomes. The user evaluation results reveal the extent to which the research and development objectives of the project are achieved, and also demonstrate the benefits and limitations of the evaluation methodology. The results of this article can be usefully applied to other collaborative research projects in a variety of domains. [ABSTRACT FROM AUTHOR]
ISSN:	09582029
DOI:	10.1093/reseval/rvs037