View in EDS HTML Full Text PDF Full Text

An Assessment of the Transparency of Contemporary Technology Education Research Employing Interview-Based Methodologies

Saved in:
Bibliographic Details
Title: An Assessment of the Transparency of Contemporary Technology Education Research Employing Interview-Based Methodologies
Language: English
Authors: Buckley, Jeffrey (ORCID 0000-0002-8292-5642), Adams, Latif, Aribilola, Ifeoluwapo, Arshad, Iram, Azeem, Muhammad, Bracken, Lauryn, Breheny, Colette, Buckley, Ciara, Chimello, Ismael, Fagan, Alison, Fitzpatrick, Daniel P., Garza Herrera, Diana, Gomes, Guilherme Daniel, Grassick, Shaun, Halligan, Elaine, Hirway, Amit, Hyland, Tomás, Imtiaz, Muhammad Babar, Khan, Muhammad Bilal, Lanzagorta Garcia, Eduardo, Lennon, Paul, Manaf, Eyman, Meng, Jing, Mohd Sufian, Mohd Sufino Zuhaily, Moraes, Adrielle, Osterwald, Katja Magdalena, Platonava, Anastasia, Reid, Clodagh, Renard, Michèle, Rodriguez-Barroso, Laura G., Simonassi-Paiva, Bianca, Singh, Maulshree, Szank, Tomasz, Tahir, Mehwish, Vijayakumar, Sowmya, Ward, Cormac, Yan, Xinyu, Zainol, Ismin, Zhang, Lin
Source: International Journal of Technology and Design Education. Sep 2022 32(4):1963-1982.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 20
Publication Date: 2022
Document Type: Journal Articles
Reports - Research
Descriptors: Technology Education, Educational Research, Research Methodology, Accountability, Qualitative Research, Interviews, Educational Researchers, Data Collection, Power Structure, Statistical Analysis
DOI: 10.1007/s10798-021-09695-1
ISSN: 0957-7572
1573-1804
Abstract: A high level of transparency in reported research is critical for several reasons, such as ensuring an acceptable level of trustworthiness and enabling replication. Transparency in qualitative research permits the identification of specific circumstances which are associated with findings and observations. Thus, transparency is important for the repeatability of original studies and for explorations of the transferability of original findings. There has been no investigation into levels of transparency in reported technology education research to date. With a position that increasing transparency would be beneficial, this article presents an analysis of levels of transparency in contemporary technology education research studies which employed interviews within their methodologies, and which were published within the "International Journal of Technology and Design Education and Design and Technology Education: An International Journal" (n = 38). The results indicate room for improvement, especially in terms of documenting researcher positionality, determinations of data saturation, and how power imbalances were managed. A discussion is presented on why it is important to improve levels of transparency in reported studies, and a guide on areas to make transparent is presented for qualitative and quantitative research.
Abstractor: As Provided
Entry Date: 2022
Accession Number: EJ1345511
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGalQu6YBD5BGkei168XwmNAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDN_Xa0I8AOh1nSf0nQIBEICBm6RAtIs5bBJ4ZTq7DnSdkG-LwhMk5wTbQSCQsBpc4PAvr2tZWhm1gfAIj8uKGbtnpBM5RS5_YJpjbnKTNRbkHRdbcow6kCehtaDNzg3CfUuW_gjh684YwbZut_GgeWOnNmzLEtNdk7aCT4zMkpkvnX3Tce238pTLVBncn_ngKtLGW1Z3uTdKryUuiBKeeJB5WfJJuMpvB1nbEO10
Text:
  Availability: 1
  Value: <anid>AN0158611141;ogv01sep.22;2022Aug22.06:07;v2.2.500</anid> <title id="AN0158611141-1">An assessment of the transparency of contemporary technology education research employing interview-based methodologies </title> <p>A high level of transparency in reported research is critical for several reasons, such as ensuring an acceptable level of trustworthiness and enabling replication. Transparency in qualitative research permits the identification of specific circumstances which are associated with findings and observations. Thus, transparency is important for the repeatability of original studies and for explorations of the transferability of original findings. There has been no investigation into levels of transparency in reported technology education research to date. With a position that increasing transparency would be beneficial, this article presents an analysis of levels of transparency in contemporary technology education research studies which employed interviews within their methodologies, and which were published within the International Journal of Technology and Design Education and Design and Technology Education: An International Journal (n = 38). The results indicate room for improvement, especially in terms of documenting researcher positionality, determinations of data saturation, and how power imbalances were managed. A discussion is presented on why it is important to improve levels of transparency in reported studies, and a guide on areas to make transparent is presented for qualitative and quantitative research.</p> <p>Keywords: Replicability; Transparency; Trustworthiness; Repeatability; Reporting practices; Qualitative research; Technology education research</p> <hd id="AN0158611141-2">Introduction</hd> <p>With a view towards making research more credible, much effort has been invested in improving open science practices within the social sciences. Among others, reasons for this include the relative ease in which researchers can engage in questionable research practices (QRP's) to obtain more easily publishable results (Simmons et al., [<reflink idref="bib39" id="ref1">39</reflink>]), the prevalence of QRP's in quantitative research (John et al., [<reflink idref="bib19" id="ref2">19</reflink>]), and that different methodological decisions can lead to significantly different results (Silberzahn et al., [<reflink idref="bib38" id="ref3">38</reflink>]). While perspectives and goals regarding open science have evolved over time, Fecher and Friesike, ([<reflink idref="bib11" id="ref4">11</reflink>]) describe five "open science schools of thought" which open science actors can subscribe to. These include the <emph>infrastructure school</emph> whose overall goal is to develop openly available platforms and tools for researchers, the <emph>public school</emph> who invest efforts into making research results accessible to lay audiences, the <emph>measurement school</emph> who aim to develop alternative metrics (altmetrics) to measure research impact, the <emph>democratic school</emph> who try to make knowledge freely available to everyone, and the <emph>pragmatic school</emph> who strive to make knowledge production more efficient and goal orientated. A strong emphasis has been placed on quantitative research within the open science movement, with particular emphasis being put on making research more replicable and reproducible (Barba, [<reflink idref="bib3" id="ref5">3</reflink>]; National Academies of Sciences Engineering & Medicine, [<reflink idref="bib33" id="ref6">33</reflink>]) in response to the replication crisis (Open Science Collaboration, [<reflink idref="bib30" id="ref7">30</reflink>]; Shrout & Rodgers, [<reflink idref="bib37" id="ref8">37</reflink>]). More recently however, and while not without contention (Aguinis & Solarino, [<reflink idref="bib1" id="ref9">1</reflink>]), there has been an increased interest in integrating open sciences practices within qualitative research (Closa, [<reflink idref="bib6" id="ref10">6</reflink>]; Haven et al., [<reflink idref="bib16" id="ref11">16</reflink>]; Pratt et al., [<reflink idref="bib32" id="ref12">32</reflink>]). In particular, there has been general consensus that improving the level of transparency in qualitative research would be a positive development, where increasing transparency can be defined as "unveiling each and every decision made during the research process, particularly in relation to data" (Closa, [<reflink idref="bib6" id="ref13">6</reflink>], p. 4). This article aims to take a first step in examining and improving the transparency of qualitative research methodologies in technology education. In this study, transparency levels in reported original qualitative research involving interview-based methodologies are examined. The motivation to concentrate exclusively on interview-based methodologies was based on their relative popularity within technology education research and as the instrument used to qualify transparency was developed and implemented initially in a case study examining transparency levels in interview-based research in the field of strategic management (Aguinis & Solarino, [<reflink idref="bib1" id="ref14">1</reflink>]). Further, while interviews are one of multiple approaches to the collection of qualitative data, the characteristics of qualitative research which require being made transparent are broadly common across data collection approaches, e.g., kind of methodology, sampling procedures, saturation, and coding and analysis. By focusing exclusively on one data collection approach this article presents a clearer and more coherent example of increasing research transparency, and while the results are bound to one type of methodology the implications of this work are more broadly applicable.</p> <p>This work relates to the open science public school of thought in that the aim is to make research processes more transparent so that it is clear under what conditions qualitative findings emerge from. However, the emphasis here is not just on making the work clearer for lay audiences, but experts as well. Specifically, this study addresses the research question: "how transparent are current reporting practices within contemporary technology education research in articles which report original studies where interviews were employed within their methodology?". Importantly, the intent of this work is not to dwell on past publication practices, but to examine published work with the agenda of identifying potential areas for future improvement. Further, there is no suggestion that any identified lack of transparency is indicative of QRP's. Instead, there is an assumption that traditional norms and practices relating to the reporting of qualitative empirical work may have resulted in a culture where certain details are not made explicit. As noted by Hiles and Čermák, ([<reflink idref="bib17" id="ref15">17</reflink>]), transparency can be easily taken for granted. This is possibly a result of assumptions that readers will be experts and may not need in-depth description.</p> <hd id="AN0158611141-3">A primer on research replicability</hd> <p>Replicable research is critical to the credibility of a field. As a concept however, it may require different interpretation for quantitative and qualitative research. In quantitative research the term replicability relates to results generated across multiple empirical studies in response to the same research question which may or may not replicate (National Academies of Sciences Engineering & Medicine, [<reflink idref="bib33" id="ref16">33</reflink>]). In other words, a research question or hypothesis is generated, and then an original investigation is conducted with the objective of providing evidence to aid in answering it in the case of a research question or testing it in the case of a hypothesis. Subsequently, a second empirical study, a "replication" study, is conducted which can be either a <emph>direct replication</emph> or a <emph>conceptual replication</emph> of the original study. A direct replication is a study which aims to be identical to an original study except with regard to the study sample (Hüffmeier et al., [<reflink idref="bib18" id="ref17">18</reflink>]), whereas a conceptual replication involves addressing the same research question or hypothesis as an original study but with different methods (Schmidt, [<reflink idref="bib36" id="ref18">36</reflink>]). Where replication studies successfully replicate the results of original studies credibility is given to the original findings and depending on the nature of the replication, the replication study may support the generalisation of original findings to broader populations. Where replication studies fail to replicate the results of original studies, it can be an indication of sampling error, measurement error, validity issues, or QRP's in the original study (Schmidt, [<reflink idref="bib35" id="ref19">35</reflink>]).</p> <p>Considering replicability as above within qualitative research has been argued to be ontologically problematic and potentially harmful (Pratt et al., [<reflink idref="bib32" id="ref20">32</reflink>]), and therefore replication in qualitative research may be more appropriately considered in terms of methodological repeatability and the transferability of original qualitative findings between contexts. Quantitative research, where replicability is most often considered, is viewed here as typical of positivist and post-positivist paradigms where the agenda is deductive theory testing and serves a different function to qualitative research. Qualitative research, in contrast, is inductive or abductive with an agenda of theory generation and is more typical of research conducted within a constructivist paradigm. An awareness of the assumptions between these paradigms is critical to appreciating the role of replication within qualitative research. Research, typically quantitative, conducted within the post-positivist paradigm is employed under an ontological assumption that phenomena of interest are objectively measurable. Qualitative research conducted within the constructivist paradigm, in contrast, has an ontological assumption that phenomena are understood differently by individuals with such understandings being socially and historically influenced (Creswell & Creswell, [<reflink idref="bib8" id="ref21">8</reflink>]; Kivunja & Kuyini, [<reflink idref="bib21" id="ref22">21</reflink>]). While a failure to replicate a quantitative result can indicate issues with validity, a qualitative replication study may lead to different yet equally valid findings because of meaningfully different qualitative circumstances, such a variance in social or cultural milieu or researcher positionality. This is not to suggest that replication is not appropriate in qualitative research, but it serves a different function than in quantitative research, that is, one of identifying the circumstances which are associated with qualitative observations.</p> <hd id="AN0158611141-4">Issues with replication and the importance of transparency in qualitative research</hd> <p>Stemming from the ontological differences underpinning qualitative and quantitative research, replication in qualitative research has been argued against for additional reasons. In some instances, such as with ethnographic methodologies, due to the role of the researcher as a research instrument, replication in how it is interpreted for quantitative research may not be meaningful (Welch & Piekkari, [<reflink idref="bib43" id="ref23">43</reflink>]) as the replication of such work involving different researchers would inevitably lead to different outcomes. Additionally, the adoption of quantitative approaches to achieving more replicable qualitative research have been criticised. For example, to advance the replicability of quantitative research, methods for pre-registering empirical studies have been developed. Pre-registration involves defining research questions, data collection plans, and data analysis plans prior to observing results (Nosek et al., [<reflink idref="bib29" id="ref24">29</reflink>]). The intent is to differentiate between prediction and "postdiction" (Nosek et al., [<reflink idref="bib29" id="ref25">29</reflink>], p. 201). Functionally, this can be achieved through pre-registering a study on a database such as the Open Science Framework. Alternatively, some journals offer the option of registered reports which see authors prepare a "Phase I" manuscript without the results or discussion sections which is subjected to peer-review and "in-principle" acceptance can be granted. Following this, the authors conduct the study and peer reviewers check whether the methodological and analytic protocols were followed and if so the work is published irrespective of the study outcome (cf. Reich, [<reflink idref="bib34" id="ref26">34</reflink>]). In theory, this prevents QRP's such as hypothesising after the results are known (HARKing: Kerr, [<reflink idref="bib20" id="ref27">20</reflink>]) and publication bias stemming from journals being less accepting of null results (Franco et al., [<reflink idref="bib12" id="ref28">12</reflink>]), and there is already evidence indicative of these benefits (Allen & Mehler, [<reflink idref="bib2" id="ref29">2</reflink>]).</p> <p>While it is possible to pre-register a qualitative study, it is questionable as to whether one should. Qualitative research is often emergent, such as in the use of grounded theory as a methodology (Charmaz, [<reflink idref="bib5" id="ref30">5</reflink>]). It is questionable how valuable it would be to pre-register such work. It is of course possible, for example, to pre-register stopping rules for data collection such as when theoretical saturation is reached, but this is markedly different to pre-registering sampling procedures in quantitative research as the imposition of such rules is still, to a degree, interpretive. Further, it is argued that the inherent uncertainty of qualitative methodologies is a strength when the goal is inductive theory generation (Weiland, [<reflink idref="bib42" id="ref31">42</reflink>]) and that good inductive work should not involve prediction (Pratt & Bonaccio, [<reflink idref="bib31" id="ref32">31</reflink>]). Pre-registration does not prevent exploratory work (it just requires authors to note where they deviated from the pre-registered protocol) nor does it have to involve prediction (just an explication of a research question), but it is clear that many believe that the nature of qualitative research does not align with current views of pre-registration. Instead, considering a primary goal of pre-registration is to increase the replicability of research, a goal which in itself has nuances to be understood from different research perspectives, qualitative scholars often argue that a more appropriate aim than increasing replicability would be to increase the transparency in reported studies (Pratt et al., [<reflink idref="bib32" id="ref33">32</reflink>]; Tuval-Mashiach, [<reflink idref="bib41" id="ref34">41</reflink>]). Doing this would increase qualitative repeatability and as a result improve capacity to determine the transferability of findings between situations and contexts.</p> <p>As mentioned, transparency is important in the reporting of qualitative studies as it enables researchers to recognise the circumstances which lead to findings (Pratt et al., [<reflink idref="bib32" id="ref35">32</reflink>]), i.e., placing a focus on the transferability of findings rather than the replicability of results. For example, an original qualitative study may lead to certain interpretive findings based on, for example, the researchers positionality along an insider-outsider continuum (Griffith, [<reflink idref="bib15" id="ref36">15</reflink>]) or the study sample. A replication study with variance in these circumstances could be perfectly valid and rigorous but yield different insight because the researcher(s) was positioned differently and thus had a different interpretive perspective, or because the sample had different prior experience or characteristics. Here, a lack of transparency reduces or prevents capacity to identify these differing circumstances as explanatory, thus hindering the function of generating a theory containing the qualitative observations.</p> <p>The importance of transparency in qualitative research further extends beyond qualitative replication and understanding circumstances associated with qualitative observation. The determination of many other characteristics of qualitative research depends on the transparent presentation of the research process. For example, determining qualitative validity, the accuracy of findings from various stakeholder perspectives, and qualitative reliability, the consistency or accuracy in implementation of qualitative methods (Gibbs, [<reflink idref="bib13" id="ref37">13</reflink>]), is only possible if the processes for doing so are made clear. Similarly, transparency in reporting is necessary to judge the dependability, confirmability, authenticity, plausibility and criticality of qualitative studies and associated findings (Lincoln & Guba, [<reflink idref="bib24" id="ref38">24</reflink>]; Locke & Golden-Biddle, [<reflink idref="bib26" id="ref39">26</reflink>]). As a synthesis of this, to consolidate the importance of transparency, the transparent reporting of qualitative research is central to enabling readers to determine the trustworthiness of qualitative studies. That is, transparency is essential to a reader's capacity to evaluate whether a researcher(s) has been honest in explaining how they conducted a study and fair in how they derived conclusions.</p> <hd id="AN0158611141-5">Method</hd> <p></p> <hd id="AN0158611141-6">Approach and method design</hd> <p>With the view that increasing transparency in reported studies would be of substantial benefit, to examine how transparent current reporting practices are for qualitative studies in technology education which employ interviews within their methodologies, this study involved rating a selection of pertinent empirical articles against criteria outlined in the behaviourally-anchored rating scales (BARS) instrument developed by Aguinis and Solarino, ([<reflink idref="bib1" id="ref40">1</reflink>]). Of importance to note, Aguinis and Solarino, ([<reflink idref="bib1" id="ref41">1</reflink>]) developed this instrument based in the field of strategic management under the ontological and epistemological perspective of qualitative positivism (Eisenhardt, [<reflink idref="bib10" id="ref42">10</reflink>]; Yin, [<reflink idref="bib44" id="ref43">44</reflink>]). That is, they viewed "social phenomena [to] exist not only in the mind, but also in the objective world" (Miles & Huberman, [<reflink idref="bib28" id="ref44">28</reflink>], p. 4). In this work their instrument is being adopted, however as the goal is to apply it as a measure of the transparency of the work of others there is no subscribed to ontological view. The only assumption being made is that the level of transparency in published work can be objectively determined and this work remains at a descriptive level.</p> <p>To develop the instrument, Aguinis and Solarino, ([<reflink idref="bib1" id="ref45">1</reflink>]) conducted a systematised literature review using the search terms "quality", "transparency", "reproducibility", "trustworthiness", and "rigor" in a search of substantive and methodological journals and books from management, business, sociology, psychology, education, nursing studies, and geography. The result was a list of 127 articles and 14 books which described methodological phases (design, measurement, analysis, reporting of results, and data availability) which were coded using an open-coding approach (Strauss & Corbin, [<reflink idref="bib40" id="ref46">40</reflink>]) until theoretical saturation was reached (Locke, [<reflink idref="bib25" id="ref47">25</reflink>]). Next, they employed theoretical coding (Charmaz, [<reflink idref="bib4" id="ref48">4</reflink>]; Strauss & Corbin, [<reflink idref="bib40" id="ref49">40</reflink>]) and identified 12 transparency criteria relating to research design, measurement, data analysis, and data disclosure which form their BARS instrument. In the implementation of the BARS instrument, an article is rated across each of these 12 criteria and given a score of either (<reflink idref="bib1" id="ref50">1</reflink>) criterion not mentioned, (<reflink idref="bib2" id="ref51">2</reflink>) criterion mentioned but not elaborated, (<reflink idref="bib3" id="ref52">3</reflink>) criterion partially met, and (<reflink idref="bib4" id="ref53">4</reflink>) criterion is met. The criteria are (Aguinis & Solarino, [<reflink idref="bib1" id="ref54">1</reflink>], pp. 1295–1296):</p> <p></p> <ulist> <item> Kind of qualitative method: The particular qualitative methodology used in the study (e.g., action research, case study, grounded theory),</item> <p></p> <item> Research setting: The physical, social, and cultural milieu of the study (e.g., firm conditions, industry, participants' social status),</item> <p></p> <item> Position of the researcher along the insider–outsider continuum: The researcher's relationship with the organisation and study participants; the closer the relationship, the more the researcher is an insider rather than an outsider,</item> <p></p> <item> Sampling procedures: The procedures used to select participants or cases for the study (e.g., convenience, purposive, theoretical),</item> <p></p> <item> Relative importance of the participants/cases: The study's sample and the relative importance of each participant of case,</item> <p></p> <item> Documenting interactions with participants: The documentation and transcription of the interviews and all other forms of observation (e.g., audio, video, notations),</item> <p></p> <item> Saturation point: It occurs when there are no new insights or themes in the process of collecting data and drawing conclusions,</item> <p></p> <item> Unexpected opportunities, challenges, and other events: Unexpected opportunities (e.g., access to additional sources of data), challenges (e.g., a firm's unit declines to participate in the last data collection stage and is replaced by a different one), and events (e.g., internal and external changes such as a new CEO or changes in the market conditions during the study) that occur during all stages of the research process,</item> <p></p> <item> Management of power imbalance: The differential exercise of control, authority, or influence during the research process,</item> <p></p> <item> Data coding and first-order codes: The process through which the data are categorised to facilitate subsequent analysis (e.g., structural coding, descriptive coding, narrative coding),</item> <p></p> <item> Data analysis and second- and higher- order codes: The classification and interpretation of linguistic or visual material to make statements about implicit and explicit dimensions and structures (Flick, 2014) and it is generally done by identifying key relationships that tie the first order codes together into a narrative or sequence (e.g., pattern coding, focused coding, axial coding), and</item> <p></p> <item> Data disclosure: Raw material includes any information collected by the research before any manipulation (i.e., analysis) (e.g., transcripts, video recordings).</item> </ulist> <p>In this study, one modification was made to the BARS instrument, henceforth referred to as the "modified BARS instrument". Based on an initial review of the selected articles by the primary author, it was apparent that in some of the articles there was no mention of second- or higher-order codes. For example, themes or other categories or groups of first-order codes were not discussed, not because of a lack of transparency, but because the process was not conducted. It could therefore be misleading to score an article as (<reflink idref="bib1" id="ref55">1</reflink>) criterion not mentioned for the "data analysis and second- and higher-order codes" criterion, as the lack of detail would not relate to a lack of transparency, but would simply reflect a lack of a methodological process to describe. Therefore, a fifth level was created solely for this criterion, which was (<reflink idref="bib5" id="ref56">5</reflink>) criterion is not relevant.</p> <hd id="AN0158611141-7">Dataset</hd> <p>For the purposes of this study, only articles which reported original research including an interview method which were published within the <emph>International Journal of Technology and Design Education</emph> (IJTDE; ISSN 0957–7572) and <emph>Design and Technology Education: An International Journal</emph> (DTEIJ; ISSN 1360–1431) were included. This criterion was not implemented to discount work published in other relevant outlets such as the <emph>Journal of Technology Education</emph> (ISSN 1045–1064) or the <emph>Australasian Journal of Technology Education</emph> (ISSN 2382–2007). The assumption was made that there is no reason to believe that publication practices will vary significantly across outlets, and the scale of the project was reflective of the number of researchers involved in coding the articles. Further, only articles which were published in print within the last two years (IJTDE issue 29.1 to 31.1; DTEIJ issue 24.1 to 29.3) were selected for inclusion in this study as the intent was to qualify how reporting practices can be improved in further articles rather than to exhaustively review previously published work. To this end, 38 articles were included in the dataset, 24 from IJTDE and 14 from DTEIJ. A complete bibliography of the included manuscripts can be found in the supplementary material (https://osf.io/aczbj/).</p> <hd id="AN0158611141-8">Coders</hd> <p>A cohort of 39 people were involved in scoring the manuscripts for their levels of transparency. One person, the primary author of this manuscript, was a lecturer and researcher from Athlone Institute of Technology, Ireland, and KTH Royal Institute of Technology, Sweden, holding a PhD in technology education and having 2.5 years post-PhD research experience. The remaining 38 coders, who are credited as co-authors as their role also involved contributing to the draft manuscript, were all research students in Athlone Institute of Technology. Of these, 36 were registered as Doctoral students within Faculties of Engineering and Informatics (<emph>n</emph> = 23), Science and Health (<emph>n</emph> = 10), and Business and Hospitality (<emph>n</emph> = 3), and two were registered as Masters students within Faculties of Engineering and Informatics (<emph>n</emph> = 1), and Business and Hospitality (<emph>n</emph> = 1). Ten of the students were in their 1<sups>st</sups> year of research studies, 15 were in their 2<sups>nd</sups> year, 11 were in their 3<sups>rd</sups> year, and two were in their 4<sups>th</sups> year. Of the students, 17 reported having prior experience with qualitative research or more specifically, research involving interview methods. All of the students volunteered to engage with this project as part of their engagement with a research module they were taking which focused on research dissemination and which was being delivered by the primary author of this manuscript during the Spring semester of the 2020/21 academic year.</p> <hd id="AN0158611141-9">Implementation</hd> <p>The primary author organised the project in advance of the delivery of the research dissemination module. This included defining the research question, the inclusion criteria for the articles which were to be scored for transparency, and deciding on the use of the BARS instrument (Aguinis & Solarino, [<reflink idref="bib1" id="ref57">1</reflink>]) and its previously described modification. In the introductory lecture for the research dissemination module, details of the project were provided to all enrolled students (<emph>N</emph> = 61). The students were required to complete a survey indicating their interest in being involved in the project. In this they were asked to self-report any additional affiliations they held, the Faculty in which they were registered as a student, the degree award they were currently registered as pursuing (Doctorate or Masters), their current year of study, and their prior experience with qualitative research and interview methods. Of the 61 students within the cohort, 38 completed the survey indicating their interest in being involved in the project.</p> <p>The first phase of the project involved all participating students engaging with three two-hour seminars delivered by the primary author during a two-week period. The first two were to ensure a required minimum level of expertise was held by all people involved, and the third was to clearly explain the project and clarify any uncertainties. The seminars included:</p> <p></p> <ulist> <item> Seminar 1: An introduction to constructivism and qualitative methodologies which provided an overview of the characteristics and assumptions of constructivism, common qualitative methodologies (phenomenology, ethnography, grounded theory, case studies, and narrative research), characteristics of qualitative research (e.g., the research setting, sampling and reflexivity), data collection, and qualitative validity, reliability, trustworthiness and credibility.</item> <p></p> <item> Seminar 2: An introduction to qualitative data analysis which focused specifically on first-cycle coding (e.g., descriptive coding, in vivo coding, structural coding, etc.) and second-cycle coding methods (pattern coding, focused coding, axial coding, theoretical coding).</item> <p></p> <item> Seminar 3: An overview of the research project which included a detailed review of the modified BARS instrument. The examples for each criteria provided by Aguinis and Solarino, ([<reflink idref="bib1" id="ref58">1</reflink>]) were reviewed as exemplars and one of the included articles from this studies dataset was also coded live during the seminar to model how to use the instrument.</item> </ulist> <p>After the seminars, the primary author assigned two articles to score from the dataset to each of the involved students. Students were designated as either "Coder 1" or "Coder 2". These designations were given for two reasons. The first was logistical as both people assigned to score an article would need to communicate with each other so the person designated as Coder 1 was asked to initiate this discussion. The second reason was to ensure that each article was scored by at least one person with relevant prior methodological experience. The primary author acted as Coder 1 for one of the articles, and for 37 of the articles students who had reported having prior experience with qualitative research were designated as Coder 1. For one article, the student who was designated as Coder 1 had not reported having prior experience with qualitative research but was in their 3<sups>rd</sups> year of study so it was acknowledged that they had sufficient experience. No two people collaborated on scoring more than one article.</p> <p>The second phase of the project involved all coders individually scoring the two articles they were assigned against the modified BARS instrument. Two weeks were given for this activity. A standardised rubric was provided to all students for this. As they completed this activity they returned their scores to the primary author. The percentages of agreement between each coder were computed as the percentages of matching scores across the 12 criteria in the modified BARS instrument. These ranged from 8.333% to 91.667% (<emph>M</emph> = 44.737%, <emph>SD</emph> = 19.703%, <emph>Med</emph> = 41.667%, <emph>MAD</emph> = 8.333%).</p> <p>The third implementation phase involved Coder 1 and Coder 2 for each article meeting to discuss their scores. As the modified BARS instrument is scored categorically, averaging scores where there was disagreement would not be valid. Therefore, in their discussion both coders were tasked with discussing and resolving any areas of disagreement to generate "agreed scores" for each transparency criterion which were the scores considered in the analysis. In this discussion, having to externalise and justify the reasons given for each score was taken as a valid approach for resolving the large levels of disagreement observed at an individual level. The primary author resolved 5 instances where two student coders were unsure on what score to assign an article against a criterion. On average, there were strong associations between the scores given by Coder 1 and the agreed scores (<emph>r</emph><subs><emph>avg</emph></subs> = 0.697) and between the scores given by Coder 2 and the agreed scores (<emph>r</emph><subs><emph>avg.</emph></subs> = 0.676). These correlations were compared using the cocor R package version 1.1–3 (Diedenhofen & Musch, [<reflink idref="bib9" id="ref59">9</reflink>]). There was not a significant difference in the strength of these correlations, <emph>z</emph> = 0.167 [–0.234, 0.280], <emph>p</emph> = 0.867.</p> <hd id="AN0158611141-10">Data analysis</hd> <p>All data were analysed in RStudio (R version 4.0.3.) and the raw data and analysis code are available in the supplementary materials (https://osf.io/aczbj/). Initially, descriptive statistics were computed. The data were then plotted to examine the descriptive question of how transparently the methods in the included manuscripts were reported, and to see if there were any criteria which were typically reported more or less transparently than others. Finally, a Spearman's correlation matrix was computed to examine the associations between transparency criteria based on the scores given against them for the examined articles. A non-parametric test was used as the data were categorical.</p> <hd id="AN0158611141-11">Results</hd> <p></p> <hd id="AN0158611141-12">Descriptive statistics</hd> <p>Descriptive statistics are presented in Table 1. As the data were categorical, mean, median and mode values are presented for each criterion to give insight into the variances in scores given. As a reminder, a score of 1 denoted that the criterion was not mentioned, a score of 2 indicated that the criterion was mentioned but not elaborated upon, a score of 3 meant that the criterion was partially met, 4 meant the criterion was met, and a score of 5 was only used in criterion 11 (Data analysis and second- and higher- order codes) if this was not relevant to the study. Observing the median and mode values in Table 1 in particular indicate which criteria tended to receive higher scores in general.</p> <p>Table 1 Descriptive statistics</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Criteria</p></th><th align="left"><p>Mean (SD)</p></th><th align="left"><p>Median (MAD)</p></th><th align="left"><p>Mode</p></th><th align="left"><p>Min</p></th><th align="left"><p>Max</p></th><th align="left"><p>Skewness</p></th><th align="left"><p>Kurtosis</p></th></tr></thead><tbody><tr><td align="left"><p>C1</p></td><td align="left"><p>3.211 (0.875)</p></td><td align="left"><p>3 (1.483)</p></td><td align="left"><p>3</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>–1.110</p></td><td char="." align="char"><p>0.692</p></td></tr><tr><td align="left"><p>C2</p></td><td align="left"><p>3.184 (0.692)</p></td><td align="left"><p>3 (0.741)</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>–0.240</p></td><td char="." align="char"><p>–0.981</p></td></tr><tr><td align="left"><p>C3</p></td><td align="left"><p>1.789 (1.044)</p></td><td align="left"><p>1 (0)</p></td><td align="left"><p>1</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>0.829</p></td><td char="." align="char"><p>–0.842</p></td></tr><tr><td align="left"><p>C4</p></td><td align="left"><p>2.658 (1.146)</p></td><td align="left"><p>3 (1.483)</p></td><td align="left"><p>4</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>–0.169</p></td><td char="." align="char"><p>–1.454</p></td></tr><tr><td align="left"><p>C5</p></td><td align="left"><p>2.605 (0.974)</p></td><td align="left"><p>3 (1.483)</p></td><td align="left"><p>3</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>–0.203</p></td><td char="." align="char"><p>–1.008</p></td></tr><tr><td align="left"><p>C6</p></td><td align="left"><p>3.158 (1.027)</p></td><td align="left"><p>4 (0)</p></td><td align="left"><p>4</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>–0.744</p></td><td char="." align="char"><p>–0.886</p></td></tr><tr><td align="left"><p>C7</p></td><td align="left"><p>1.395 (0.855)</p></td><td align="left"><p>1 (0)</p></td><td align="left"><p>1</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>2.211</p></td><td char="." align="char"><p>3.864</p></td></tr><tr><td align="left"><p>C8</p></td><td align="left"><p>1.842 (1.079)</p></td><td align="left"><p>1 (0)</p></td><td align="left"><p>1</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>0.934</p></td><td char="." align="char"><p>–0.557</p></td></tr><tr><td align="left"><p>C9</p></td><td align="left"><p>1.526 (0.862)</p></td><td align="left"><p>1 (0)</p></td><td align="left"><p>1</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>1.278</p></td><td char="." align="char"><p>0.245</p></td></tr><tr><td align="left"><p>C10</p></td><td align="left"><p>2.395 (1.054)</p></td><td align="left"><p>2 (1.483)</p></td><td align="left"><p>2</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>0.405</p></td><td char="." align="char"><p>–1.138</p></td></tr><tr><td align="left"><p>C11</p></td><td align="left"><p>2.605 (1.326)</p></td><td align="left"><p>2 (0.741)</p></td><td align="left"><p>2</p></td><td align="left"><p>1</p></td><td align="left"><p>5</p></td><td char="." align="char"><p>0.727</p></td><td char="." align="char"><p>–0.832</p></td></tr><tr><td align="left"><p>C12</p></td><td align="left"><p>2.158 (0.945)</p></td><td align="left"><p>2 (1.483)</p></td><td align="left"><p>3</p></td><td align="left"><p>1</p></td><td align="left"><p>4</p></td><td char="." align="char"><p>0.068</p></td><td char="." align="char"><p>–1.297</p></td></tr></tbody></table> </ephtml> </p> <p>Note: C1 = Kind of qualitative method. C2 = Research setting. C3 = Position of the researcher along the insider–outsider continuum. C4: Sampling procedures. C5 = Relative importance's of the participants/cases. C6 = Documenting interactions with participants. C7 = Saturation point. C8 = Unexpected opportunities, challenges, and other events. C9 = Management of power imbalance. C10 = Data coding and first-order codes. C11 = Data analysis and second- or higher-order codes. C12 = Data disclosure. SD = Standard deviation. MAD = Median absolute deviation</p> <hd id="AN0158611141-13">Transparency scores</hd> <p>To gain further insight into the levels of transparency across the examined articles, the data were plotted as a bubble plot (Fig. 1) and a bar chart (Fig. 2). The bubble plot indicates each transparency score given per included article. Reading this vertically indicates which articles tended to be more or less transparent. Reading this horizontally gives an overview of which criteria were generally scored higher and which were generally scored lower. From this, it becomes apparent that in reporting their work researchers tended to more often be clearer in the type of methodology adopted, the research setting, sampling procedures, the importance of participants/cases, documenting how they interacted with participants, and how they analysed their data. In contrast, there is a higher frequency of low scores for the criteria relating to noting the positionality of researchers, the saturation point in terms of data collection, unexpected opportunities or challenges, and how researchers managed power imbalances.</p> <p>Graph: Fig. 1 Bubble plot indicating each transparency score for each included article</p> <p>Graph: Fig. 2 Percentages of scores across each of the transparency criteria</p> <p>The bar chart (Fig. 2) provides a clearer picture of this but just from the perspective of the different criteria. It is apparent that in all cases except for the research setting there were articles which did not address certain criteria at all, but equally that all criteria were addressed adequately (a score of 4) at least once. Criteria 3, 7, and 9 were least often addressed, whilst criteria 1, 2, 4, and 6 were most often addressed sufficiently.</p> <hd id="AN0158611141-14">Associations between transparency criteria</hd> <p>A correlation analysis was conducted to determine any significant associations between transparency criteria. Ten statistically significant correlations were observed (Table 2). Positive correlations were observed between the reporting of the kind of qualitative method and the positionality of the researcher(s) (<emph>ρ</emph> = 0.34, <emph>p</emph> < 0.05), between explaining the research setting and the relative importance of participants (<emph>ρ</emph> = 0.34, <emph>p</emph> < 0.05) and in describing whether interactions were documented (<emph>ρ</emph> = 0.37, <emph>p</emph> < 0.05), between detailing sampling procedures and the relative importance of participants (<emph>ρ</emph> = 0.36, <emph>p</emph> < 0.05), between documenting the relative importance of participants and interactions with participants (<emph>ρ</emph> = 0.46, <emph>p</emph> < 0.01), unexpected opportunities and challenges (<emph>ρ</emph> = 0.55, <emph>p</emph> < 0.01), and the use of first-order coding (<emph>ρ</emph> = 0.32, <emph>p</emph> < 0.05), between reporting how interactions were documented with participants and whether there were any unexpected opportunities or challenges (<emph>ρ</emph> = 0.32, <emph>p</emph> < 0.05), and between commenting on saturation and the use of first order codes (<emph>ρ</emph> = 46, <emph>p</emph> < 0.01). Additionally, a statistically significant negative correlation was observed between providing details of researcher(s) positionality and the use of second-order coding for data analysis (<emph>ρ</emph> = -0.34, <emph>p</emph> < 0.05). Based on these correlations, there appears to be three groups of associations relating to methodology, the sample and interactions with participants, and data analysis. However, a larger scale study would be required to examine this, but as an initial insight it is helpful as note broader areas of the methodology for researchers who are thinking about how they can be more transparent.</p> <p>Table 2 Spearman's rho (ρ) correlation matrix with 95% confidence intervals</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Variable</p></th><th align="left"><p>C1</p></th><th align="left"><p>C2</p></th><th align="left"><p>C3</p></th><th align="left"><p>C4</p></th><th align="left"><p>C5</p></th><th align="left"><p>C6</p></th><th align="left"><p>C7</p></th><th align="left"><p>C8</p></th><th align="left"><p>C9</p></th><th align="left"><p>C10</p></th><th align="left"><p>C11</p></th></tr></thead><tbody><tr><td align="left"><p>C1: Kind of qualitative method</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C2: Research setting</p></td><td align="left"><p>14[–.19,.45]</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C3: Position of the researcher along the insider–outsider continuum</p></td><td align="left"><p>.34*[.02,.60]</p></td><td align="left"><p>.04[–.29,.37]</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C4: Sampling procedures</p></td><td align="left"><p>.01[–.32,.33]</p></td><td align="left"><p>.39*[.07,.64]</p></td><td align="left"><p>.12[–.22,.43]</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C5: Relative importance's of the participants/cases</p></td><td align="left"><p>.20[–.14,.49]</p></td><td align="left"><p>.34*[.01,.60]</p></td><td align="left"><p>–.06[–.38,.28]</p></td><td align="left"><p>.36*[.04,.62]</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C6: Documenting interactions with participants</p></td><td align="left"><p>.23[–.11,.52]</p></td><td align="left"><p>.37*[.05,.62]</p></td><td align="left"><p>–.18[–.48,.16]</p></td><td align="left"><p>.25[–.09,.53]</p></td><td align="left"><p>.46**[.15,.68]</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C7: Saturation point</p></td><td align="left"><p>.03[–.31,.35]</p></td><td align="left"><p>.31[–.03,.58]</p></td><td align="left"><p>.12[–.21,.43]</p></td><td align="left"><p>.19[–.15,.49]</p></td><td align="left"><p>.14[–.19,.45]</p></td><td align="left"><p>.12[–.22,.43]</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C8: Unexpected opportunities, challenges, and other events</p></td><td align="left"><p>.28[–.05,.56]</p></td><td align="left"><p>.28[–.05,.56]</p></td><td align="left"><p>.24[–.10,.53]</p></td><td align="left"><p>.14[–.20,.45]</p></td><td align="left"><p>.55**[.28,.75]</p></td><td align="left"><p>.32*[–.01,.59]</p></td><td align="left"><p>–.17[–.47,.17]</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C9: Management of power imbalance</p></td><td align="left"><p>.28[–.05,.56]</p></td><td align="left"><p>.06[–.28,.38]</p></td><td align="left"><p>.26[–.07,.54]</p></td><td align="left"><p>–.05[–.37,.28]</p></td><td align="left"><p>.24[–.10,.52]</p></td><td align="left"><p>.10[–.23,.42]</p></td><td align="left"><p>–.24[–.52,.10]</p></td><td align="left"><p>.27[–.06,.55]</p></td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C10: Data coding and first–order codes</p></td><td align="left"><p>.25[–.09,.53]</p></td><td align="left"><p>.12[–.22,.43]</p></td><td align="left"><p>.08[–.26,.39]</p></td><td align="left"><p>.03[–.30,.36]</p></td><td align="left"><p>.32*[–.01,.59]</p></td><td align="left"><p>.21[–.13,.50]</p></td><td align="left"><p>.46**[.15,.68]</p></td><td align="left"><p>.02[–.31,.34]</p></td><td align="left"><p>–.0003[–.33,.33]</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>C11: Data analysis and second– or higher–order codes</p></td><td align="left"><p>–.10[–.41,.24]</p></td><td align="left"><p>–.15[–.46,.18]</p></td><td align="left"><p>–.34*[–.60, –.01]</p></td><td align="left"><p>–.02[–.35,.31]</p></td><td align="left"><p>.01[–.32,.33]</p></td><td align="left"><p>.28[–.05,.56]</p></td><td align="left"><p>–.02[–.34,.31]</p></td><td align="left"><p>–16[–.46,.18]</p></td><td align="left"><p>–.02[–.35,.31]</p></td><td align="left"><p>.09[–.25,.41]</p></td><td align="left" /></tr><tr><td align="left"><p>C12: Data disclosure</p></td><td align="left"><p>–.01[–.34,.32]</p></td><td align="left"><p>–.09[–.40,.25]</p></td><td align="left"><p>–.27[–.55,.07]</p></td><td align="left"><p>–.03[–.35,.30]</p></td><td align="left"><p>.24[–.10,.53]</p></td><td align="left"><p>.30[–.04,.57]</p></td><td align="left"><p>.05[–.28,.37]</p></td><td align="left"><p>.24[–.09,.53]</p></td><td align="left"><p>.12[–.22,.43]</p></td><td align="left"><p>.10[–.24,.41]</p></td><td align="left"><p>.09[–.25,.41]</p></td></tr></tbody></table> </ephtml> </p> <p>Note: *Correlation is significant at the 0.05 level (two-tailed). **Correlation is significant at the 0.01 level (two-tailed). A supplementary visualisation of this analysis can be found at https://osf.io/aczbj/</p> <hd id="AN0158611141-15">ficulties with employing the modified BARS instrument</hd> <p>A final question which was of interest to examine was how easy or difficult was it to implement the modified BARS instrument for the coders. The perspective taken on this was to determine how often coders tended to agree with each other across each criterion. The average level of agreement ranged from 28.95% (C4: Sampling procedures) to 57.89% (C7: Saturation point) with a mean level of average agreement of 44.74% (SD = 8.76%). Insight gained from discussions between the primary author and a number of people designated as Coder 1 (people with prior experience of qualitative research) suggests that the reason for this low level of agreement stems from a lack of experience from many people designated as Coder 2 which resulted in misunderstanding the criteria, a limitation which is seen as partially mitigated by the process of collaboration to generate agreed codes.</p> <hd id="AN0158611141-16">Discussion</hd> <p>The results of this study indicate that there are areas for improvement in terms of increasing the transparency in reporting qualitative studies in technology education research. Many articles scored quite well, however others did not, no article in the included sample received a perfect score, and there was considerable variance across manuscripts. While authors were in general more transparent in reporting the kind of qualitative methodology adopted, the research setting, their sampling procedures, and in documenting how they interacted with participants, they tended to be less clear in their positionality within investigations, whether they reached saturation in data collection, and in how power relations were managed between themselves and participants. For readers to fully appreciate observations from a qualitative study and for qualitative replication studies to be meaningful in identifying the circumstances associated with such observations, all of this information needs to be made clear in manuscripts reporting qualitative studies. While not conclusive due to the current sample size, there appears to be relationships associated with methodology, the sampling and interacting with participants, and data analysis, such that if authors tended to be more or less transparent regarding one component of these areas they tended to be more or less transparent in general within these areas. This information could be useful for future researcher development activities such as in academic writing workshops. However, where the agenda is to improve future practices it is questionable whether a larger scale study to confirm these areas would be of much value, and with respect to the included articles from this analysis, where there is any lack of clarity communication can be made with corresponding authors to gain any needed insight into the original studies.</p> <p>Before considering how these results can be used to inform future research reporting practices, it is important to recognise that transparency relates just as much to quantitative and indeed mixed methods research as it does to qualitative research. Taking Aguinis and Solarino's, ([<reflink idref="bib1" id="ref60">1</reflink>]) BARS instrument, each of the 12 criteria are either relevant to quantitative research or have corresponding quantitative alternatives. Table 3 provides our current thinking on this. In some cases, the criteria are identical in meaning across research approaches, such as with documenting the kind of methodology, data disclosure, unexpected challenges and opportunities, the relative importance of participants or cases, and sampling procedures. In other cases, the quantitative criteria are equivalent, however the treatment or purpose would be different. For example, explaining the research setting in qualitative research is necessary for understanding the social and cultural milieu of the study, whereas in quantitative research this may relate more to explaining whether it was a clinical trial or indeed for effectiveness studies it would be important to explain the real-world study conditions. Additionally, positionality and reflexivity which are often discussed in qualitative research (e.g., Mason-Bish, [<reflink idref="bib27" id="ref61">27</reflink>]) are important for quantitative researchers, but perhaps more relevant in questioning motivational and methodological decisions such as hypothesis generation and sampling. Likewise, managing power imbalances in quantitative research may relate to participation incentives, such as the use of financial incentives where people in less financially stable situations may be more likely to engage in a study they would otherwise prefer not to, or where participants volunteer into a study for a financial reward associated with completion and do not meaningfully engage and therefore create validity issues (Goldenberg et al., [<reflink idref="bib14" id="ref62">14</reflink>]; Zutlevics, [<reflink idref="bib45" id="ref63">45</reflink>]). Finally, some of the criteria for transparency in quantitative research are uniquely quantitative but are related to similar qualitative activity, such as conducting power calculations for sample size determination (Cohen, [<reflink idref="bib7" id="ref64">7</reflink>]; Lachin, [<reflink idref="bib22" id="ref65">22</reflink>]) which could be seen as similar to identifying a saturation point as both relate to determining the quantity of data to collect.</p> <p>Table 3 Quantitative alternatives to the BARS criteria with a suggested additional criterion associated with explaining the epistemological stance of research</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>BARS criteria (Aguinis & Solarino, <xref ref-type="bibr" rid="bibr1">2019</xref>)</p></th><th align="left"><p>Quantitative alternative</p></th></tr></thead><tbody><tr><td align="left"><p>Kind of qualitative method</p></td><td align="left"><p>Kind of quantitative method</p></td></tr><tr><td align="left"><p>Research setting</p></td><td align="left"><p>Research setting</p></td></tr><tr><td align="left"><p>Position of the researcher along the insider–outsider continuum</p></td><td align="left"><p>Positionality and reflexivity</p></td></tr><tr><td align="left"><p>Sampling procedures</p></td><td align="left"><p>Sampling procedures</p></td></tr><tr><td align="left"><p>Relative importance's of the participants/cases</p></td><td align="left"><p>Relative importance's of the participants/cases</p></td></tr><tr><td align="left"><p>Documenting interactions with participants</p></td><td align="left"><p>Documenting data collection protocols</p></td></tr><tr><td align="left"><p>Saturation point</p></td><td align="left"><p>Statistical power</p></td></tr><tr><td align="left"><p>Unexpected opportunities, challenges, and other events</p></td><td align="left"><p>Unexpected opportunities, challenges, and other events</p></td></tr><tr><td align="left"><p>Management of power imbalance</p></td><td align="left"><p>Management of power imbalance</p></td></tr><tr><td align="left"><p>Data coding and first-order codes</p></td><td align="left"><p>Data cleaning, the treatment of outliers and testing statistical assumptions</p></td></tr><tr><td align="left"><p>Data analysis and second- or higher-order codes</p></td><td align="left"><p>Formal statistical analysis</p></td></tr><tr><td align="left"><p>Data disclosure</p></td><td align="left"><p>Data disclosure</p></td></tr><tr><td align="left"><p>*Epistemological stance</p></td><td align="left"><p>Epistemological stance</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>Note: *Suggested criterion additional to those presented by </emph>Aguinis and Solarino ([<reflink idref="bib1" id="ref66">1</reflink>])</p> <p>Based on this, the recommendation is for an increased level of transparency in all conducted research in technology education regardless of research approach, and Table 3 could support researchers by serving as a guide. If this were to be the case, it is recommended that a further criterion be added which is associated with the epistemological stance of authors. To give examples of why this is meaningful, and this is not comprehensive with respect to researcher epistemology, it is not uncommon to see qualitative studies report frequencies of participants which offer commentary surrounding a code, theme or category with the implication that higher frequencies give weight to interpretations of validity or importance. However, other qualitative researchers may take the stance that such frequencies have no such implication, and that a single utterance or comment can offer insight of equal value to a theme commented on with relatively high frequency. This view taken in analysing data is critical to understanding the lens taken in qualitative analysis. In quantitative analysis, similar information is important in understanding researchers view around the criteria for which a result is judged to constitute as new knowledge. For example, many quantitative researchers adhere to alpha values of 0.05 to manage family-wise error rates and compare resulting <emph>p</emph>-values against this. It would be useful from a transparency perspective to understand the researcher's justification for this or the use of other alpha values (Lakens et al., [<reflink idref="bib23" id="ref67">23</reflink>]).</p> <p>In the use of this information to aid in the increasing of reporting transparency, it is envisioned that authors and reviewers would reflect on the aspects of work they are reporting or reviewing which require explication. In doing this, it is important to note that in some cases certain criteria may not be possible to meet. In particular, disclosing data may not be ethical in qualitative research (Pratt et al., [<reflink idref="bib32" id="ref68">32</reflink>]). Instead, it is often recommended that researchers provide rich descriptions of data excerpts to aid others in understanding the nature of their insights and conclusions (Creswell & Creswell, [<reflink idref="bib8" id="ref69">8</reflink>]). Ethics should not be breached to meet such a transparency criterion. Researchers could disclose data where possible such as with certain document analysis research and make protocols such as interview guides and their evolution within studies available. Additionally, it may be difficult to adhere to criteria in cases where they are not relevant, as was the case with this study where in some articles no second-order analysis was reported. On reflection, there may have been studies where there were no unexpected opportunities or challenges so it would not necessarily be poor transparency to not document anything accordingly. As noted by Tuval-Mashiach, ([<reflink idref="bib41" id="ref70">41</reflink>]), transparency is important, but it only relates to the documentation of what did happen as opposed to what did not happen. So, not documenting any challenges for instance could mean there were none or that they are not being reported. As this would not be clear, perhaps it would be of use for authors to note that there were no challenges or unexpected opportunities arising during data collection if this was the case. To add a final comment pertaining to studies including multiple sources of data, such as mixed methods research or qualitative studies involving the triangulation of interview, observational, and document data, it is important that authors consider transparency as "unveiling each and every decision made during the research process, particularly in relation to data" (Closa, [<reflink idref="bib6" id="ref71">6</reflink>], p. 4). Given that transparency can easily be taken for granted (Hiles & Čermák, [<reflink idref="bib17" id="ref72">17</reflink>]), commenting on a transparency criterion relative to one source of data in such work may not be sufficient but in practice this could be overlooked. For example, in conducting both interviews and focus groups as part of a study, where the samples come from the same population researcher positionality may only need to be described once, but if the samples came from different populations, positionality may differ and thus power imbalances could have been managed in alternate ways. Transparent reporting warrants considering each manuscript individually and ensuring that all actions and decisions made are clear such that other researchers could theoretically repeat the investigation.</p> <hd id="AN0158611141-17">Limitations</hd> <p>There are several limitations to the current work which need to be acknowledged in interpreting the reported results. Many of the coders did not have prior experience with qualitative research or educational research. While the conducted seminars were designed to address this and agreed codes were used where one person involved had reported experience with qualitative methods, it is clear from discussions with coders that there was a level of misunderstanding amongst some people. Other limitations relate to the instrument itself. As discussed, there is an epistemological dimension which, if added, would be useful. The scoring of article transparency in this study does not involve this, and therefore the reported levels of transparency relate to transparency as defined by the adopted modified BARS instrument. Finally, the implementation of the modified BARS instrument involves some subjectivity. In particular when criteria were not reported, such as with the criterion relating to unexpected challenges or opportunities, it would be difficult for a coder to know whether the criterion was met as in reality there were no unexpected challenges or opportunities, or if it should be scored as not mentioned as meaning there was nothing to report or there was just nothing reported when there should have been.</p> <p>Acknowledging the limitation in coder expertise, there is an advantage to this coder demographic. It is possible that the authors did acknowledge certain aspects of their work when reporting it and that this was not noticed by a cohort of more novice researchers. If a more experienced cohort of researchers scored the articles in this study, perhaps the scores would have been higher and more objective, so potential error should be considered. The current scores reflect what was understood of the criteria by the cohort of coders and how they understood the manuscripts, so an advantage of these scores is that they come from a novice perspective and therefore could indicate that authors may need to be more explicit in their reporting so that novice researchers engaging with their work can more clearly understand the research process. Alternatively, it is worth considering the possibility that a more expert cohort may have scored articles higher due to a tacit understanding of the research process when the process was not in fact explicit. As such, the results in this study are perhaps best used as a catalyst for conversation concerning future improvements rather than taking the given scores as absolute.</p> <hd id="AN0158611141-18">Open science practices</hd> <p>A bibliography of coded articles, the scores given to articles by individual coders, agreed scores which were used for the analysis, and the analysis code necessary to reproduce to the analyses are available at https://osf.io/aczbj/. Authors listed after the primary author are ordered alphabetically.</p> <hd id="AN0158611141-19">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0158611141-20"> <title> References </title> <blist> <bibl id="bib1" idref="ref9" type="bt">1</bibl> <bibtext> Aguinis H, Solarino AM. Transparency and replicability in qualitative research: the case of interviews with elite informants. Strategic Management Journal. 2019; 40; 8: 1291-1315. 10.1002/smj.3015</bibtext> </blist> <blist> <bibl id="bib2" idref="ref29" type="bt">2</bibl> <bibtext> Allen, C. P. G, Mehler, D. M. (2018). Open Science challenges, benefits and tips in early career and beyond. PsyArXiv. https://doi.org/10.31234/osf.io/3czyt</bibtext> </blist> <blist> <bibl id="bib3" idref="ref5" type="bt">3</bibl> <bibtext> Barba, L. (2018). Terminologies for reproducible research. ArXiv. https://arxiv.org/pdf/1802.03311</bibtext> </blist> <blist> <bibl id="bib4" idref="ref48" type="bt">4</bibl> <bibtext> Charmaz, K. (2006). Constructing grounded theory (2nd ed.). SAGE</bibtext> </blist> <blist> <bibl id="bib5" idref="ref30" type="bt">5</bibl> <bibtext> Charmaz KHesse-Biber SN, Leavy P. Grounded theory as an emergent method. Handbook of Emergent Methods. 2008; Guilford Press: 155-172</bibtext> </blist> <blist> <bibl id="bib6" idref="ref10" type="bt">6</bibl> <bibtext> Closa C. Planning, implementing and reporting: Increasing transparency, replicability and credibility in qualitative political science research. European Political Science. 2021. 10.1057/s41304-020-00299-2</bibtext> </blist> <blist> <bibl id="bib7" idref="ref64" type="bt">7</bibl> <bibtext> Cohen J. A power primer. Psychological Bulletin. 1992; 112; 1: 155-159. 10.1037/0033-2909.112.1.155</bibtext> </blist> <blist> <bibl id="bib8" idref="ref21" type="bt">8</bibl> <bibtext> Creswell, J, & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage Publications, Inc</bibtext> </blist> <blist> <bibl id="bib9" idref="ref59" type="bt">9</bibl> <bibtext> Diedenhofen B, Musch J. cocor: A comprehensive solution for the statistical comparison of correlations. PLoS ONE. 2015; 10; 4: e0121945. 10.1371/journal.pone.0121945</bibtext> </blist> <blist> <bibtext> Eisenhardt K. Building theories from case study research. The Academiy of Management Review. 1989; 14; 4: 532-550. 10.2307/258557</bibtext> </blist> <blist> <bibtext> Fecher, B, Friesike, S. (2014). Open science: One term, five schools of thought. In S. Bartling & S. Friesike (Eds.), Opening Science: The Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing (pp. 17–47). Springer International Publishing. https://doi.org/10.1007/978-3-319-00026-8_2</bibtext> </blist> <blist> <bibtext> Franco, A, Malhotra, N, Simonovits, G. (2014). Social science. Publication bias in the social sciences: Unlocking the file drawer. Science (New York, N.Y.), 345(6203), 1502–1505. https://doi.org/10.1126/science.1255484</bibtext> </blist> <blist> <bibtext> Gibbs, G. (2007). Analyzing qualitative data. In U. Flick (Ed.), The Sage qualitative research kit. SAGE</bibtext> </blist> <blist> <bibtext> Goldenberg LR, Owens EF, Pickar JG. Recruitment of research volunteers: methods, interest, and incentives. The Journal of Chiropractic Education. 2007; 21; 1: 28-31. 10.7899/1042-5055-21.1.28</bibtext> </blist> <blist> <bibtext> Griffith AI. Insider/outsider: epistemological privilege and mothering work. Human Studies. 1998; 21; 4: 361-376. 10.1023/A:1005421211078</bibtext> </blist> <blist> <bibtext> Haven T, Errington T, Gleditsch KS, van Grootel L, Jacobs A, Kern F, Piñeiro R, Rosenblatt F, Mokkink L. Preregistering qualitative research: a delphi study. International Journal of Qualitative Methods. 2020; 19: 1-13. 10.1177/1609406920976417</bibtext> </blist> <blist> <bibtext> Hiles, D, Čermák, I. (2007). Qualitative research: Transparency and narrative oriented inquiry. 10th European Congress of Psychology, 1–8</bibtext> </blist> <blist> <bibtext> Hüffmeier J, Mazei J, Schultze T. Reconceptualizing replication as a sequence of different studies: a replication typology. Journal of Experimental Social Psychology. 2016; 66: 81-92. 10.1016/j.jesp.2015.09.009</bibtext> </blist> <blist> <bibtext> John LK, Loewenstein G, Prelec D. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science. 2012; 23; 5: 524-532. 10.1177/0956797611430953</bibtext> </blist> <blist> <bibtext> Kerr NL. HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review. 1998; 2; 3: 196-217. 10.1207/s15327957pspr0203_4</bibtext> </blist> <blist> <bibtext> Kivunja C, Kuyini AB. Understanding and applying research paradigms in educational contexts. International Journal of Higher Education. 2017; 6; 5: 26. 10.5430/ijhe.v6n5p26</bibtext> </blist> <blist> <bibtext> Lachin Jo. Introduction to sample size determination and power analysis for clinical trials. Controlled Clinical Trials. 1981; 2; 2: 93-113. 10.1016/0197-2456(81)90001-5</bibtext> </blist> <blist> <bibtext> Lakens D, Adolfi FG, Albers CJ, Anvari F, Apps MAJ, Argamon SE, Baguley T, Becker RB, Benning SD, Bradford DE, Buchanan EM, Caldwell AR, Van Calster B, Carlsson R, Chen S-C, Chung B, Colling LJ, Collins GS, Crook Z, Zwaan RA. Justify your alpha. Nature Human Behaviour. 2018; 2; 3: 168-171. 10.1038/s41562-018-0311-x</bibtext> </blist> <blist> <bibtext> Lincoln Y, Guba ELincoln Y, Guba E. Establishing trustworthiness. Naturalistic inquiry. 1985; SAGE: 289-331</bibtext> </blist> <blist> <bibtext> Locke, K. (2001). Grounded theory in management research. SAGE</bibtext> </blist> <blist> <bibtext> Locke K, Golden-Biddle K. Constructing opportunities for contribution: structuring intertextual coherence and "problematizing" in organizational studies. Academy of Management Journal. 1997; 40; 5: 1023-1062. 10.2307/256926</bibtext> </blist> <blist> <bibtext> Mason-Bish H. The elite delusion: reflexivity, identity and positionality in qualitative research. Qualitative Research. 2019; 19; 3: 263-276. 10.1177/1468794118770078</bibtext> </blist> <blist> <bibtext> Miles, M, Huberman, A. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Sage</bibtext> </blist> <blist> <bibtext> Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. Proceedings of the National Academy of Sciences. 2018; 115; 11: 2600-2606. 10.1073/pnas.1708274114</bibtext> </blist> <blist> <bibtext> Open Science Collaboration. Estimating the reproducibility of psychological science. Science. 2015; 349; 6251: 943. 10.1126/science.aac4716</bibtext> </blist> <blist> <bibtext> Pratt M, Bonaccio S. Qualitative research in I-O psychology: maps, myths, and moving forward. Industrial and Organizational Psychology. 2016; 9; 4: 693-715. 10.1017/iop.2016.92</bibtext> </blist> <blist> <bibtext> Pratt M, Kaplan S, Whittington R. Editorial essay: the tumult over transparency: decoupling transparency from replication in establishing trustworthy qualitative research. Administrative Science Quarterly. 2020; 65; 1: 1-19. 10.1177/0001839219887663</bibtext> </blist> <blist> <bibtext> National Academies of Sciences Engineering and Medicine. (2019). Reproducibility and replicability in science. The National Academies Press</bibtext> </blist> <blist> <bibtext> Reich J. Preregistration and registered reports. Educational Psychologist. 2021. 10.1080/00461520.2021.1900851</bibtext> </blist> <blist> <bibtext> Schmidt S. Shall we really do it again? the powerful concept of replication is neglected in the social sciences. Review of General Psychology. 2009; 13; 2: 90-100. 10.1037/a0015108</bibtext> </blist> <blist> <bibtext> Schmidt SMakel M, Plucker J. Replication. Toward a more perfect psychology: Improving trust, accuracy, and transparency in research. 2017; American Psychological Association: 233-253. 10.1037/0000033-015</bibtext> </blist> <blist> <bibtext> Shrout P, Rodgers J. Psychology, science, and knowledge construction: broadening perspectives from the replication crisis. Annual Review of Psychology. 2018; 69; 1: 487-510. 10.1146/annurev-psych-122216-011845</bibtext> </blist> <blist> <bibtext> Silberzahn R, Uhlmann EL, Martin DP, Anselmi P, Aust F, Awtrey E, Bahník Š, Bai F, Bannard C, Bonnier E, Carlsson R, Cheung F, Christensen G, Clay R, Craig MA, Dalla Rosa A, Dam L, Evans MH, Flores Cervantes I, Nosek BA. Many analysts, one data set: making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science. 2018; 1; 3: 337-356. 10.1177/2515245917747646</bibtext> </blist> <blist> <bibtext> Simmons J, Nelson L, Simonsohn U. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science. 2011; 22; 11: 1359-1366. 10.1177/0956797611417632</bibtext> </blist> <blist> <bibtext> Strauss, A, & Corbin, J. (1998). Basics of qualitative research. SAGE</bibtext> </blist> <blist> <bibtext> Tuval-Mashiach R. Raising the curtain: the importance of transparency in qualitative research. Qualitative Psychology. 2017; 4; 2: 126-138. 10.1037/qup0000062</bibtext> </blist> <blist> <bibtext> Weiland SJosselson R, Lieblich A, McAdams D. Writers as readers in narrative inquiry: Learning from biography. Up close and personal: The teaching and learning of narrative research. 2003; American Psychological Association: 199-214. 10.1037/10486-011</bibtext> </blist> <blist> <bibtext> Welch C, Piekkari R. How should we (not) judge the 'quality' of qualitative research? a re-assessment of current evaluative criteria in international business. Journal of World Business. 2017; 52; 5: 714-725. 10.1016/j.jwb.2017.05.007</bibtext> </blist> <blist> <bibtext> Yin, R. (2014). Case study research: Design and methods (5th ed.). SAGE</bibtext> </blist> <blist> <bibtext> Zutlevics T. Could providing financial incentives to research participants be ultimately self-defeating?. Research Ethics. 2016; 12; 3: 137-148. 10.1177/1747016115626756</bibtext> </blist> </ref> <aug> <p>By Jeffrey Buckley; Latif Adams; Ifeoluwapo Aribilola; Iram Arshad; Muhammad Azeem; Lauryn Bracken; Colette Breheny; Ciara Buckley; Ismael Chimello; Alison Fagan; Daniel P. Fitzpatrick; Diana Garza Herrera; Guilherme Daniel Gomes; Shaun Grassick; Elaine Halligan; Amit Hirway; Tomás Hyland; Muhammad Babar Imtiaz; Muhammad Bilal Khan; Eduardo Lanzagorta Garcia; Paul Lennon; Eyman Manaf; Jing Meng; Mohd Sufino Zuhaily Mohd Sufian; Adrielle Moraes; Katja Magdalena Osterwald; Anastasia Platonava; Clodagh Reid; Michèle Renard; Laura G. Rodriguez-Barroso; Bianca Simonassi-Paiva; Maulshree Singh; Tomasz Szank; Mehwish Tahir; Sowmya Vijayakumar; Cormac Ward; Xinyu Yan; Ismin Zainol and Lin Zhang</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib39" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib19" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib38" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib11" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib33" firstref="ref6"></nolink> <nolink nlid="nl6" bibid="bib30" firstref="ref7"></nolink> <nolink nlid="nl7" bibid="bib37" firstref="ref8"></nolink> <nolink nlid="nl8" bibid="bib16" firstref="ref11"></nolink> <nolink nlid="nl9" bibid="bib32" firstref="ref12"></nolink> <nolink nlid="nl10" bibid="bib17" firstref="ref15"></nolink> <nolink nlid="nl11" bibid="bib18" firstref="ref17"></nolink> <nolink nlid="nl12" bibid="bib36" firstref="ref18"></nolink> <nolink nlid="nl13" bibid="bib35" firstref="ref19"></nolink> <nolink nlid="nl14" bibid="bib21" firstref="ref22"></nolink> <nolink nlid="nl15" bibid="bib43" firstref="ref23"></nolink> <nolink nlid="nl16" bibid="bib29" firstref="ref24"></nolink> <nolink nlid="nl17" bibid="bib34" firstref="ref26"></nolink> <nolink nlid="nl18" bibid="bib20" firstref="ref27"></nolink> <nolink nlid="nl19" bibid="bib12" firstref="ref28"></nolink> <nolink nlid="nl20" bibid="bib42" firstref="ref31"></nolink> <nolink nlid="nl21" bibid="bib31" firstref="ref32"></nolink> <nolink nlid="nl22" bibid="bib41" firstref="ref34"></nolink> <nolink nlid="nl23" bibid="bib15" firstref="ref36"></nolink> <nolink nlid="nl24" bibid="bib13" firstref="ref37"></nolink> <nolink nlid="nl25" bibid="bib24" firstref="ref38"></nolink> <nolink nlid="nl26" bibid="bib26" firstref="ref39"></nolink> <nolink nlid="nl27" bibid="bib10" firstref="ref42"></nolink> <nolink nlid="nl28" bibid="bib44" firstref="ref43"></nolink> <nolink nlid="nl29" bibid="bib28" firstref="ref44"></nolink> <nolink nlid="nl30" bibid="bib40" firstref="ref46"></nolink> <nolink nlid="nl31" bibid="bib25" firstref="ref47"></nolink> <nolink nlid="nl32" bibid="bib27" firstref="ref61"></nolink> <nolink nlid="nl33" bibid="bib14" firstref="ref62"></nolink> <nolink nlid="nl34" bibid="bib45" firstref="ref63"></nolink> <nolink nlid="nl35" bibid="bib22" firstref="ref65"></nolink> <nolink nlid="nl36" bibid="bib23" firstref="ref67"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1345511
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: An Assessment of the Transparency of Contemporary Technology Education Research Employing Interview-Based Methodologies
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Buckley%2C+Jeffrey%22">Buckley, Jeffrey</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-8292-5642">0000-0002-8292-5642</externalLink>)<br /><searchLink fieldCode="AR" term="%22Adams%2C+Latif%22">Adams, Latif</searchLink><br /><searchLink fieldCode="AR" term="%22Aribilola%2C+Ifeoluwapo%22">Aribilola, Ifeoluwapo</searchLink><br /><searchLink fieldCode="AR" term="%22Arshad%2C+Iram%22">Arshad, Iram</searchLink><br /><searchLink fieldCode="AR" term="%22Azeem%2C+Muhammad%22">Azeem, Muhammad</searchLink><br /><searchLink fieldCode="AR" term="%22Bracken%2C+Lauryn%22">Bracken, Lauryn</searchLink><br /><searchLink fieldCode="AR" term="%22Breheny%2C+Colette%22">Breheny, Colette</searchLink><br /><searchLink fieldCode="AR" term="%22Buckley%2C+Ciara%22">Buckley, Ciara</searchLink><br /><searchLink fieldCode="AR" term="%22Chimello%2C+Ismael%22">Chimello, Ismael</searchLink><br /><searchLink fieldCode="AR" term="%22Fagan%2C+Alison%22">Fagan, Alison</searchLink><br /><searchLink fieldCode="AR" term="%22Fitzpatrick%2C+Daniel+P%2E%22">Fitzpatrick, Daniel P.</searchLink><br /><searchLink fieldCode="AR" term="%22Garza+Herrera%2C+Diana%22">Garza Herrera, Diana</searchLink><br /><searchLink fieldCode="AR" term="%22Gomes%2C+Guilherme+Daniel%22">Gomes, Guilherme Daniel</searchLink><br /><searchLink fieldCode="AR" term="%22Grassick%2C+Shaun%22">Grassick, Shaun</searchLink><br /><searchLink fieldCode="AR" term="%22Halligan%2C+Elaine%22">Halligan, Elaine</searchLink><br /><searchLink fieldCode="AR" term="%22Hirway%2C+Amit%22">Hirway, Amit</searchLink><br /><searchLink fieldCode="AR" term="%22Hyland%2C+Tomás%22">Hyland, Tomás</searchLink><br /><searchLink fieldCode="AR" term="%22Imtiaz%2C+Muhammad+Babar%22">Imtiaz, Muhammad Babar</searchLink><br /><searchLink fieldCode="AR" term="%22Khan%2C+Muhammad+Bilal%22">Khan, Muhammad Bilal</searchLink><br /><searchLink fieldCode="AR" term="%22Lanzagorta+Garcia%2C+Eduardo%22">Lanzagorta Garcia, Eduardo</searchLink><br /><searchLink fieldCode="AR" term="%22Lennon%2C+Paul%22">Lennon, Paul</searchLink><br /><searchLink fieldCode="AR" term="%22Manaf%2C+Eyman%22">Manaf, Eyman</searchLink><br /><searchLink fieldCode="AR" term="%22Meng%2C+Jing%22">Meng, Jing</searchLink><br /><searchLink fieldCode="AR" term="%22Mohd+Sufian%2C+Mohd+Sufino+Zuhaily%22">Mohd Sufian, Mohd Sufino Zuhaily</searchLink><br /><searchLink fieldCode="AR" term="%22Moraes%2C+Adrielle%22">Moraes, Adrielle</searchLink><br /><searchLink fieldCode="AR" term="%22Osterwald%2C+Katja+Magdalena%22">Osterwald, Katja Magdalena</searchLink><br /><searchLink fieldCode="AR" term="%22Platonava%2C+Anastasia%22">Platonava, Anastasia</searchLink><br /><searchLink fieldCode="AR" term="%22Reid%2C+Clodagh%22">Reid, Clodagh</searchLink><br /><searchLink fieldCode="AR" term="%22Renard%2C+Michèle%22">Renard, Michèle</searchLink><br /><searchLink fieldCode="AR" term="%22Rodriguez-Barroso%2C+Laura+G%2E%22">Rodriguez-Barroso, Laura G.</searchLink><br /><searchLink fieldCode="AR" term="%22Simonassi-Paiva%2C+Bianca%22">Simonassi-Paiva, Bianca</searchLink><br /><searchLink fieldCode="AR" term="%22Singh%2C+Maulshree%22">Singh, Maulshree</searchLink><br /><searchLink fieldCode="AR" term="%22Szank%2C+Tomasz%22">Szank, Tomasz</searchLink><br /><searchLink fieldCode="AR" term="%22Tahir%2C+Mehwish%22">Tahir, Mehwish</searchLink><br /><searchLink fieldCode="AR" term="%22Vijayakumar%2C+Sowmya%22">Vijayakumar, Sowmya</searchLink><br /><searchLink fieldCode="AR" term="%22Ward%2C+Cormac%22">Ward, Cormac</searchLink><br /><searchLink fieldCode="AR" term="%22Yan%2C+Xinyu%22">Yan, Xinyu</searchLink><br /><searchLink fieldCode="AR" term="%22Zainol%2C+Ismin%22">Zainol, Ismin</searchLink><br /><searchLink fieldCode="AR" term="%22Zhang%2C+Lin%22">Zhang, Lin</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Technology+and+Design+Education%22"><i>International Journal of Technology and Design Education</i></searchLink>. Sep 2022 32(4):1963-1982.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 20
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Technology+Education%22">Technology Education</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Research%22">Educational Research</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Methodology%22">Research Methodology</searchLink><br /><searchLink fieldCode="DE" term="%22Accountability%22">Accountability</searchLink><br /><searchLink fieldCode="DE" term="%22Qualitative+Research%22">Qualitative Research</searchLink><br /><searchLink fieldCode="DE" term="%22Interviews%22">Interviews</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Researchers%22">Educational Researchers</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Collection%22">Data Collection</searchLink><br /><searchLink fieldCode="DE" term="%22Power+Structure%22">Power Structure</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10798-021-09695-1
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0957-7572<br />1573-1804
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: A high level of transparency in reported research is critical for several reasons, such as ensuring an acceptable level of trustworthiness and enabling replication. Transparency in qualitative research permits the identification of specific circumstances which are associated with findings and observations. Thus, transparency is important for the repeatability of original studies and for explorations of the transferability of original findings. There has been no investigation into levels of transparency in reported technology education research to date. With a position that increasing transparency would be beneficial, this article presents an analysis of levels of transparency in contemporary technology education research studies which employed interviews within their methodologies, and which were published within the "International Journal of Technology and Design Education and Design and Technology Education: An International Journal" (n = 38). The results indicate room for improvement, especially in terms of documenting researcher positionality, determinations of data saturation, and how power imbalances were managed. A discussion is presented on why it is important to improve levels of transparency in reported studies, and a guide on areas to make transparent is presented for qualitative and quantitative research.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1345511
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1345511
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10798-021-09695-1
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 20
        StartPage: 1963
    Subjects:
      – SubjectFull: Technology Education
        Type: general
      – SubjectFull: Educational Research
        Type: general
      – SubjectFull: Research Methodology
        Type: general
      – SubjectFull: Accountability
        Type: general
      – SubjectFull: Qualitative Research
        Type: general
      – SubjectFull: Interviews
        Type: general
      – SubjectFull: Educational Researchers
        Type: general
      – SubjectFull: Data Collection
        Type: general
      – SubjectFull: Power Structure
        Type: general
      – SubjectFull: Statistical Analysis
        Type: general
    Titles:
      – TitleFull: An Assessment of the Transparency of Contemporary Technology Education Research Employing Interview-Based Methodologies
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Buckley, Jeffrey
      – PersonEntity:
          Name:
            NameFull: Adams, Latif
      – PersonEntity:
          Name:
            NameFull: Aribilola, Ifeoluwapo
      – PersonEntity:
          Name:
            NameFull: Arshad, Iram
      – PersonEntity:
          Name:
            NameFull: Azeem, Muhammad
      – PersonEntity:
          Name:
            NameFull: Bracken, Lauryn
      – PersonEntity:
          Name:
            NameFull: Breheny, Colette
      – PersonEntity:
          Name:
            NameFull: Buckley, Ciara
      – PersonEntity:
          Name:
            NameFull: Chimello, Ismael
      – PersonEntity:
          Name:
            NameFull: Fagan, Alison
      – PersonEntity:
          Name:
            NameFull: Fitzpatrick, Daniel P.
      – PersonEntity:
          Name:
            NameFull: Garza Herrera, Diana
      – PersonEntity:
          Name:
            NameFull: Gomes, Guilherme Daniel
      – PersonEntity:
          Name:
            NameFull: Grassick, Shaun
      – PersonEntity:
          Name:
            NameFull: Halligan, Elaine
      – PersonEntity:
          Name:
            NameFull: Hirway, Amit
      – PersonEntity:
          Name:
            NameFull: Hyland, Tomás
      – PersonEntity:
          Name:
            NameFull: Imtiaz, Muhammad Babar
      – PersonEntity:
          Name:
            NameFull: Khan, Muhammad Bilal
      – PersonEntity:
          Name:
            NameFull: Lanzagorta Garcia, Eduardo
      – PersonEntity:
          Name:
            NameFull: Lennon, Paul
      – PersonEntity:
          Name:
            NameFull: Manaf, Eyman
      – PersonEntity:
          Name:
            NameFull: Meng, Jing
      – PersonEntity:
          Name:
            NameFull: Mohd Sufian, Mohd Sufino Zuhaily
      – PersonEntity:
          Name:
            NameFull: Moraes, Adrielle
      – PersonEntity:
          Name:
            NameFull: Osterwald, Katja Magdalena
      – PersonEntity:
          Name:
            NameFull: Platonava, Anastasia
      – PersonEntity:
          Name:
            NameFull: Reid, Clodagh
      – PersonEntity:
          Name:
            NameFull: Renard, Michèle
      – PersonEntity:
          Name:
            NameFull: Rodriguez-Barroso, Laura G.
      – PersonEntity:
          Name:
            NameFull: Simonassi-Paiva, Bianca
      – PersonEntity:
          Name:
            NameFull: Singh, Maulshree
      – PersonEntity:
          Name:
            NameFull: Szank, Tomasz
      – PersonEntity:
          Name:
            NameFull: Tahir, Mehwish
      – PersonEntity:
          Name:
            NameFull: Vijayakumar, Sowmya
      – PersonEntity:
          Name:
            NameFull: Ward, Cormac
      – PersonEntity:
          Name:
            NameFull: Yan, Xinyu
      – PersonEntity:
          Name:
            NameFull: Zainol, Ismin
      – PersonEntity:
          Name:
            NameFull: Zhang, Lin
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 09
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 0957-7572
            – Type: issn-electronic
              Value: 1573-1804
          Numbering:
            – Type: volume
              Value: 32
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: International Journal of Technology and Design Education
              Type: main
ResultId 1