View in EDS HTML Full Text PDF Full Text

Evaluating the Quality of E-Learning at the Degree Level in the Student Experience of Blended Learning

Saved in:
Bibliographic Details
Title: Evaluating the Quality of E-Learning at the Degree Level in the Student Experience of Blended Learning
Language: English
Authors: Ginns, Paul, Ellis, Rob A.
Source: British Journal of Educational Technology. Jul 2009 40(4):652-663.
Availability: Blackwell Publishing. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8599; Fax: 781-388-8232; e-mail: customerservices@blackwellpublishing.com; Web site: http://www.blackwellpublishing.com/jnl_default.asp
Peer Reviewed: Y
Physical Description: PDF
Page Count: 12
Publication Date: 2009
Document Type: Journal Articles
Reports - Evaluative
Education Level: Higher Education
Postsecondary Education
Descriptors: Undergraduate Students, Quality Control, Psychometrics, Student Experience, Learning Experience, Test Construction, Rating Scales, Course Evaluation, Test Validity, Test Reliability, Educational Technology, Technology Integration, Computer Software, Electronic Learning, Teaching Methods, Evaluation Methods
DOI: 10.1111/j.1467-8535.2008.00861.x
ISSN: 0007-1013
Abstract: This paper reports on the development of a scale for determining the quality of the student e-learning experience at the degree level when the student learning context is predominately a campus-based experience. Rapid developments in the use of information and communication technologies (ICT) in higher education require methods for evaluating the contribution of such tools to student learning, especially when they are complementing a face-to-face experience. We examine the psychometric functioning of a proposed e-learning scale in relation to a well-validated degree-level teaching evaluation instrument, the Student Course Experience Questionnaire. The e-learning scale has suitable reliability and validity in the present sample of undergraduate students for quality assurance activities around ICT.
Abstractor: As Provided
Number of References: 33
Entry Date: 2009
Accession Number: EJ843502
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFuiyknEAOUqwKiAk9iwuAqAAAA4TCB3gYJKoZIhvcNAQcGoIHQMIHNAgEAMIHHBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDEN_FrNnKMdSdDqIlgIBEICBmVyuG5pn4ilCtsTKtUYR3X42G4IWuo0qPvC3LhgH1gU2AXRpBOwzrAh1_hIW3D2YceSpmsJkuc8mZ7d-XdjD46NZt__NkkcnE_i8k9aSqKpg71cOy9ucbXSH8Apd-OeFR_Ce4qoC0HhN10y0QC9ON8mHO41LkkhQ-SZeuwAunJwiyf5_PbfZdB6lucFFC1LYnMjzlAMHGupx1w==
Text:
  Availability: 1
  Value: <anid>AN0041435973;58i01jul.09;2019May28.12:54;v2.2.500</anid> <title id="AN0041435973-1">Evaluating the quality of e-learning at the degree level in the student experience of blended learning. </title> <p>This paper reports on the development of a scale for determining the quality of the student e‐learning experience at the degree level when the student learning context is predominately a campus‐based experience. Rapid developments in the use of information and communication technologies (ICT) in higher education require methods for evaluating the contribution of such tools to student learning, especially when they are complementing a face‐to‐face experience. We examine the psychometric functioning of a proposed e‐learning scale in relation to a well‐validated degree‐level teaching evaluation instrument, the Student Course Experience Questionnaire. The e‐learning scale has suitable reliability and validity in the present sample of undergraduate students for quality assurance activities around ICT.</p> <p>In the past 10 years, largely driven by the increasing pervasiveness of the Internet, universities around the world have perceived a growing need to incorporate information and communication technologies (ICTs) into their offerings ([<reflink idref="bib4" id="ref1">4</reflink>]; [<reflink idref="bib25" id="ref2">25</reflink>]). However, following previous cycles of enthusiasm and discontent with educational technologies—including radio and educational television ([<reflink idref="bib13" id="ref3">13</reflink>])—the benefits from investments in many cases have not been forthcoming. One of the most recent of such failures, the United Kingdom's 'UKeU' (UK eUniversities Worldwide Limited) e‐learning project which began in 2000, was wound up after attracting only 900 students, despite attracting £62 million in public funding.</p> <p>The experience with UKeU has prompted the Higher Education Funding Council for England (HEFCE) to adumbrate its intended strategy for supporting the development of e‐learning in English higher education institutions over the next 10 years ([<reflink idref="bib14" id="ref4">14</reflink>]). A core aim of the strategy will be to 'support the [Higher Education] sector as it moves towards embedding e‐learning appropriately, using technology to transform higher education into a more student‐focused and flexible system' ([<reflink idref="bib14" id="ref5">14</reflink>], p. 5). From HEFCE's strategy, we adopt a definition of e‐learning as '... any learning that uses ICT... to support students and improve the management of learning' (p. 5). Moreover, HEFCE puts the emphasis on 'learning research, innovation and development that begins with a focus on student learning rather than on developments in technology per se, enabling students to learn through and be supported by technology' ([<reflink idref="bib14" id="ref6">14</reflink>], p. 6). The key concept underpinning these aims is the complementary, elaborating and enhancing role conceptualised for e‐learning in the university student experience of learning.</p> <p>Seeking to understand a complementary role of e‐learning in university experiences is by no means a goal restricted to Great Britain. Given the affordances provided by e‐learning and its ubiquitous appearance in the experiences of university students in most countries, the challenge is to identify ways to evaluate its appropriate place and contribution to the development of student understanding when used across different disciplines.</p> <p>The UK example is emblematic of how, worldwide, higher education providers are coming to understand the necessary relations between student learning and ICTs (see also [<reflink idref="bib4" id="ref7">4</reflink>]'Introduction'). [<reflink idref="bib23" id="ref8">23</reflink>]), in discussing the notion of 'integrated e‐learning', argue that there is a need for 'a variety of coherent measures at the pedagogical, organisational and technical levels for the successful implementation of e‐learning in combination with more conventional methods' (p. 5). The issue of coherence in evaluating the success of blended learning is especially germane, as the overall goal of a blended learning experience is to provide a mix of both online and face‐to‐face experiences which support each other in achieving desired learning outcomes. As such, we argue that evaluating the quality of blended learning in a coherent manner requires relating the 'part' of the online learning context (eg, online activities, discussions, etc.), to the 'whole' of the student experience (which for the purposes of this paper, shall be the totality of both online and face‐to‐face learning experiences).</p> <p>In this paper, we describe the development of a student evaluation of teaching scale which focuses on the quality of university‐level e‐learning. There have been many such questionnaires developed for gathering feedback at the subject level, either for wholly online subjects (eg, [<reflink idref="bib1" id="ref9">1</reflink>]) or those involving a combination of face‐to‐face and online teaching and learning (eg, [<reflink idref="bib18" id="ref10">18</reflink>]). The e‐learning scale described below is an emerging construct showing promise in unpacking the structure of the student experience of e‐learning at the <emph>whole degree</emph> level which is comprised of both face‐to‐face experiences as well as e‐learning. We examine the suitability of the proposed scale in relation to an existing teaching evaluation instrument, the <emph>Student Course Experience Questionnaire </emph>([<reflink idref="bib19" id="ref11">19</reflink>]) and its underlying theoretical framework.</p> <hd id="AN0041435973-2">Student Learning Theory and the student course experience questionnaire</hd> <p>Student Learning Theory (eg, [<reflink idref="bib5" id="ref12">5</reflink>]; [<reflink idref="bib16" id="ref13">16</reflink>]; [<reflink idref="bib28" id="ref14">28</reflink>]; [<reflink idref="bib30" id="ref15">30</reflink>]) is a theory of learning which has had a substantial impact on how learning at post‐secondary levels can be understood and improved. It focuses strongly on students' experiences of core aspects of the teaching and learning environment, as these experiences are often intimately related to students' approaches to learning ([<reflink idref="bib6" id="ref16">6</reflink>]; [<reflink idref="bib15" id="ref17">15</reflink>]), and subsequent performance. [<reflink idref="bib29" id="ref18">29</reflink>]) and [<reflink idref="bib35" id="ref19">35</reflink>]) developed the <emph>Course Experience Questionnaire </emph>(CEQ) as a teaching performance indicator at the degree level, in order to provide reliable and valid measures of core aspects of the teaching and learning environment related to approaches to learning.</p> <p>The psychometric properties of the CEQ have been investigated across a wide variety of settings; for a recent review of this research literature, see [<reflink idref="bib32" id="ref20">32</reflink>]). However, as a survey which is sent to Australian graduates, the CEQ is a 'lag indicator'. To supplement the CEQ with more timely data, the University of Sydney created a modified form of the CEQ, for use with currently enrolled students, called the <emph>Student Course Experience Questionnaire</emph> (SCEQ). [<reflink idref="bib19" id="ref21">19</reflink>]) discussed the psychometric functioning of the core scale of the SCEQ, finding evidence for suitable levels of reliability and validity. The SCEQ forms an important part of the university's quality assurance of teaching programmes, as well as providing an evidence base for the effectiveness of strategic projects and curriculum change (eg, [<reflink idref="bib3" id="ref22">3</reflink>]; [<reflink idref="bib2" id="ref23">2</reflink>]; [<reflink idref="bib8" id="ref24">8</reflink>]).</p> <p>A key goal of this paper, then, is to investigate the suitability of a short (5‐item) e‐learning scale as an addition to an existing validated teaching evaluation instrument, the SCEQ. Such a scale could play an important role in assessing the quality of university learning through the use of ICTs, and provide an evidence base for ongoing benchmarking and evaluation of the effects of curriculum change. A broader goal is to use the example of the scales as a way of raising the benefits and challenges in trying to evaluate a part (learning supported by ICTs) of a whole (the student learning experience) in a relational way. We will argue that such an approach allows for an interpretation of the results that can practically assist those responsible for the part to inform their approach to its provision.</p> <hd id="AN0041435973-3">Method</hd> <p></p> <hd id="AN0041435973-4">Participants</hd> <p>In this study, we examine the psychometric properties of responses to the SCEQ scales, including the proposed e‐learning scale, using the 2005 undergraduate dataset. The data set consisted of responses from 3602 students who provided a complete set of responses to all scale items examined in the following analyses.</p> <hd id="AN0041435973-5">Materials</hd> <p>The SCEQ is a biennial survey of a random stratified sample of coursework students at the University of Sydney. Students are questioned about their perceptions of their degree experience, and respond using a 5‐point Likert scale marked 'Strongly Disagree', 'Disagree', 'Neutral', 'Agree' and 'Strongly Agree'. Table 1 gives the core scales and items as well as the proposed e‐learning scale items. The SCEQ is administered through a multiple‐mode survey, including paper, online and phone surveying. For details of the development of the SCEQ, see [<reflink idref="bib19" id="ref25">19</reflink>]).</p> <p>1 Completely standardised factor loadings for SCEQ items</p> <p> <ephtml> <table><thead valign="bottom"><tr><th /><th><italic>Factor loading</italic></th></tr><tr><th><italic>GTS</italic></th><th><italic>CGS</italic></th><th><italic>AAS</italic></th><th><italic>AWS</italic></th><th><italic>E‐LS</italic></th><th><italic>GSS</italic></th></tr></thead><tbody valign="top"><tr><td>Good teaching scale (GTS)</td></tr><tr><td> 3. The teaching staff normally give me helpful feedback on how I am going</td><td>0.69</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 8. The teaching staff of this degree course motivate me to do my best work</td><td>0.74</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 16. The staff make a real effort to understand difficulties I may be having with my work</td><td>0.68</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 19. My lecturers are extremely good at explaining things</td><td>0.63</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 21. The teaching staff work hard to make their subjects interesting</td><td>0.63</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 27. The staff put a lot of time into commenting on my work</td><td>0.61</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Clear goals and standards scale (CGS)</td></tr><tr><td> 6. I have usually had a clear idea of where I am going and what is expected of me in this degree course</td><td>0</td><td>0.68</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 12. It is always easy to know the standard of work expected</td><td>0</td><td>0.71</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 25. The staff made it clear right from the start what they expected from students</td><td>0</td><td>0.73</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 29. It has often been hard to discover what is expected of me in this degree course*</td><td>0</td><td>0.69</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Appropriate assessment scale (AAS)</td></tr><tr><td> 13. The staff seem more interested in testing what I have memorised than what I have understood*</td><td>0</td><td>0</td><td>0.73</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 20. Too many staff ask me questions just about facts*</td><td>0</td><td>0</td><td>0.61</td><td>0</td><td>0</td><td>0</td></tr><tr><td> 26. To do well in this degree all you really need is a good memory*</td><td>0</td><td>0</td><td>0.67</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Appropriate workload scale (AWS)</td></tr><tr><td> 2. There is a lot of pressure on me as a student in this degree course*</td><td>0</td><td>0</td><td>0</td><td>0.60</td><td>0</td><td>0</td></tr><tr><td> 4. The workload is too heavy*</td><td>0</td><td>0</td><td>0</td><td>0.71</td><td>0</td><td>0</td></tr><tr><td> 15. I am generally given enough time to understand the things I have to learn</td><td>0</td><td>0</td><td>0</td><td>0.65</td><td>0</td><td>0</td></tr><tr><td> 24. The sheer volume of work to be got through in this degree means it can't all be thoroughly comprehended*</td><td>0</td><td>0</td><td>0</td><td>0.71</td><td>0</td><td>0</td></tr><tr><td>e‐Learning scale (E‐LS)</td></tr><tr><td> 14. Where it was used, information technology helped me to learn</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.56</td><td>0</td></tr><tr><td> 38. Resources on University of Sydney websites (eg, WebCT, Blackboard, degree course sites, faculty sites, etc.) supported my learning</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.62</td><td>0</td></tr><tr><td> 41. Communicating online with students and staff helped my learning</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.64</td><td>0</td></tr><tr><td> 43. The online learning experiences of my degree course were well‐integrated with my face‐to‐face learning</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.77</td><td>0</td></tr><tr><td> 45. My online experiences helped me engage actively in my learning</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.83</td><td>0</td></tr><tr><td>Generic skills scale (GSS)</td></tr><tr><td> 5. The degree course has helped me develop my ability to work as a team member</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.43</td></tr><tr><td> 9. The degree course has sharpened my analytic skills</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.73</td></tr><tr><td> 10. As a result of my degree course, I feel confident about tackling unfamiliar problems</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.73</td></tr><tr><td> 18. The degree course has developed my problem‐solving skills</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.74</td></tr><tr><td> 22. The degree course has improved my skills in written communication</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.51</td></tr><tr><td> 23. My degree course has helped me to develop the ability to plan my own work</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.56</td></tr><tr><td>Factor correlations</td></tr><tr><td> Good teaching scale</td><td>1</td><td /><td /><td /><td /><td /></tr><tr><td> Clear goals and standards scale</td><td>0.64</td><td>1</td><td /><td /><td /><td /></tr><tr><td> Appropriate assessment scale</td><td>0.34</td><td>0.38</td><td>1</td><td /><td /><td /></tr><tr><td> Appropriate workload scale</td><td>0.38</td><td>0.22</td><td>0.45</td><td>1</td><td /><td /></tr><tr><td> e‐Learning scale</td><td>0.22</td><td>0.23</td><td>0.01</td><td>–0.05</td><td>1</td><td /></tr><tr><td> Generic skills scale</td><td>0.58</td><td>0.38</td><td>0.15</td><td>0.27</td><td>0.34</td><td>1</td></tr><tr><td>Correlations with overall satisfaction with degree quality item</td><td>0.61</td><td>0.47</td><td>0.26</td><td>0.27</td><td>0.27</td><td>0.53</td></tr></tbody></table> </ephtml> </p> <p>1 * Reversed item. Factor loadings of zero are fixed according to the hypothesised measurement model.</p> <ulist> <item>2 SCEQ, Student Course Experience Questionnaire.</item> <item>3 [Corrections added after online publication 25 May 2009; Corrections made to correlation coefficients in columns <emph>E‐LS</emph> and <emph>GSS</emph> for the overall satisfaction with degree quality item in Table 1.]</item> </ulist> <p>The items of the e‐learning scale were developed in the following way. A pool of potential items was developed through discussion with Subject‐Matter Experts in e‐learning across the university, drawing on aspects of the Student Learning research literature, which has focused on distance, online and blended learning (eg, [<reflink idref="bib20" id="ref26">20</reflink>]; [<reflink idref="bib26" id="ref27">26</reflink>]; [<reflink idref="bib27" id="ref28">27</reflink>]). Student‐focused items were developed reflecting degree of assistance of ICTs in learning, quality of online resources, staff and student interaction, integration of online and face‐to‐face learning experiences, and active engagement. Item wording was refined through an iterative process and checked for meaningfulness with student representatives before inclusion in the SCEQ.</p> <hd id="AN0041435973-6">Analyses</hd> <p>As discussed above, the SCEQ is derived from the CEQ, and psychometric studies of both instruments have typically found a clear factor structure. On this basis, we used confirmatory factor analyses to examine the properties of the proposed e‐learning scale and its relation to the core SCEQ scales, using LISREL 8.54 ([<reflink idref="bib24" id="ref29">24</reflink>]). A key assumption of standard confirmatory factor analysis is multivariate normality ([<reflink idref="bib7" id="ref30">7</reflink>]; [<reflink idref="bib31" id="ref31">31</reflink>]). We calculated univariate and multivariate normality statistics using PRELIS 2.54, and found statistically significant levels of univariate and multivariate skewness and kurtosis in the data. To take account of this violation of assumptions, we used robust maximum likelihood estimation, which is less sensitive than other estimation methods to violations of the normality assumption with large models ([<reflink idref="bib7" id="ref32">7</reflink>]).</p> <p>We tested the validity and reliability of the new scale in the following ways.</p> <p>To test the construct validity of the new scale in relation to the existing scales, we fitted a highly restricted six‐factor model, with no error covariances, to the data. For the six‐factor model, we report the following statistics, following [<reflink idref="bib34" id="ref33">34</reflink>]) recommendations for suitable values of appropriate models. The comparative fit index (CFI) and the Relative Fit Index (RFI) are measures of fit which compare the proposed model with the independence model, in which all variables are uncorrelated; values greater than 0.90 are held to indicate good fit. The standardised root mean square residual (SRMR) is the mean difference, based on standardised residuals, between the predicted and observed variances and covariances in the model; values below 0.05 indicate good fit. The root mean square error of approximation (RMSEA) gives an estimate per degree of freedom of the discrepancy between the model and population covariance matrices. Values below 0.05 are held to indicate good fit, and values up to 0.08 indicate moderately good fit ([<reflink idref="bib34" id="ref34">34</reflink>]). The RMSEA is presented with a 90% confidence interval. We tested another form of validity—criterion validity—by correlating the core SCEQ scales and the e‐learning scale scores with the 'Overall Satisfaction with Degree Quality' item ([<reflink idref="bib35" id="ref35">35</reflink>]), and with the 'Generic Skills' outcome scale. Constructs that form an important part of the student experience can be expected to correlate to a moderate to strong degree with such measures; for instance, in enrolled student samples, [<reflink idref="bib35" id="ref36">35</reflink>]) reported correlations of CEQ scale scores with the Overall Satisfaction Item ranging from 0.64 (Good Teaching) to 0.33 (Appropriate Workload).</p> <p>We calculated the internal consistency of each scale using [<reflink idref="bib12" id="ref37">12</reflink>]) estimate of the reliability coefficient, including 95% confidence intervals for each estimate of coefficient alpha ([<reflink idref="bib17" id="ref38">17</reflink>]). Lastly, we calculated inter‐rater agreement for the e‐learning scale for each faculty. This analysis evaluates the degree to which multiple raters, using typical Likert scale formats, evaluate aspects of an organisation's culture or climate similarly ([<reflink idref="bib10" id="ref39">10</reflink>]). Suitable levels of inter‐rater agreement support arguments for the aggregation of ratings taken at a lower level (eg, quality of e‐learning at the degree level) to represent levels of a construct at a higher level (eg, quality of e‐learning at the faculty level). Within the university, individual faculties have substantial responsibilities for assuring and improving the quality of all their degree programmes; hence, it is useful to have teaching performance indicators taken at the degree level which 'scale up' to measure constructs meaningfully across multiple degree programmes within a faculty. As the e‐learning scale consists of multiple items, we calculated a form of inter‐rater agreement, <emph>AD</emph><subs>M(<emph>j</emph>)</subs>, using [<reflink idref="bib9" id="ref40">9</reflink>]) recommended method. The <emph>AD </emph>(Average Deviation) index is calculated by '... determining the extent to which each item rating differs from the mean (or median) rating, summing the absolute values of these deviations... and dividing the sum by the number of deviations' ([<reflink idref="bib9" id="ref41">9</reflink>], p. 160). Burke and Dunlap set the upper cut‐off limit for acceptable inter‐rater agreement as the number of response options divided by six. The Likert scale used in the e‐learning scale had five options, making the recommended cut‐off level for aggregation of individual results to the faculty level equal to 0.83.</p> <hd id="AN0041435973-7">Results</hd> <p></p> <hd id="AN0041435973-8">Six‐factor measurement model</hd> <p>We tested the construct validity of a six‐factor model including items from the five core SCEQ scales—Good Teaching, Clear Goals and Standards, Appropriate Assessment, Appropriate Workload, and an outcome scale, Generic Skills—plus the five items from the e‐learning scale. The hypothesised six‐factor measurement model had good fit to the data by conventional criteria ([<reflink idref="bib34" id="ref42">34</reflink>]), CFI = 0.96, RFI = 0.95, SRMR = 0.050, RMSEA = 0.047 (90% confidence interval 0.046–0.049). Table 1 gives the completely standardised factor loadings from the six‐factor measurement model. Factor loadings for the e‐learning scale ranged from 0.83 ('My online experiences helped me engage actively in my learning') to 0.56 ('Where it was used, information technology helped me to learn').</p> <hd id="AN0041435973-9">Criterion validity of e‐learning scale ratings</hd> <p>We tested the criterion validity of the SCEQ scales, including the e‐learning scale, by calculating Pearson product‐moment correlations between the scale scores and the Overall Satisfaction Item ('Overall, I was satisfied with the quality of this degree course'), and with students' Generic Skills Scale scores. These correlations are given in the final rows of Table 1. The correlations of e‐learning scale scores with overall ratings of degree quality and ratings of Generic Skills development were 0.27 and 0.34, respectively, which can be considered as moderate effect according to [<reflink idref="bib11" id="ref43">11</reflink>]) suggested benchmarks.</p> <hd id="AN0041435973-10">Reliability analyses</hd> <p>We tested the internal consistency of the SCEQ scales, including the e‐learning scale, using [<reflink idref="bib12" id="ref44">12</reflink>]) coefficient alpha. Table 2 gives estimates of Cronbach's alpha and 95% confidence intervals around these estimates. All scales had a range of plausible estimates of scale reliability acceptable according to typical psychometric conventions (eg, [<reflink idref="bib33" id="ref45">33</reflink>]).</p> <p>2 Reliability estimates of SCEQ scales (Cronbach's α and 95% confidence intervals)</p> <p> <ephtml> <table><thead valign="bottom"><tr><th><italic>Scale</italic></th><th><italic>Cronbach's coefficient alpha</italic></th></tr></thead><tbody valign="top"><tr><td>Good teaching scale</td><td>0.83 (95% CI 0.816–0.834)</td></tr><tr><td>Clear goals & standards scale</td><td>0.80 (95% CI 0.784–0.806)</td></tr><tr><td>Appropriate assessment scale</td><td>0.71 (95% CI 0.683–0.717)</td></tr><tr><td>Appropriate workload scale</td><td>0.76 (95% CI 0.748–0.774)</td></tr><tr><td>e‐Learning scale</td><td>0.81 (95% CI 0.801–0.821)</td></tr><tr><td>Generic skills scale</td><td>0.78 (95% CI 0.760–0.783)</td></tr></tbody></table> </ephtml> </p> <p>4 SCEQ, Student Course Experience Questionnaire.</p> <hd id="AN0041435973-11">Inter‐rater agreement analyses</hd> <p>Table 3 gives Average Deviation estimates of inter‐rater agreement at the faculty level for the e‐learning scale. Estimates ranged from 0.60 to 0.81, ie, all estimates fell under [<reflink idref="bib9" id="ref46">9</reflink>]) recommended cut‐off of 0.83. These results support the aggregation of students' ratings of the quality of e‐learning in their degrees up to the level of faculty and the use of such aggregated results for internal benchmarking within our university.</p> <p>3 Inter‐Rater Agreement Faculty Estimates for SCEQ scales using average deviation metric</p> <p> <ephtml> <table><thead valign="bottom"><tr><th><italic>Faculty</italic></th><th><italic>e‐Learning Scale</italic></th></tr></thead><tbody valign="top"><tr><td>Agriculture</td><td>0.71</td></tr><tr><td>Architecture</td><td>0.80</td></tr><tr><td>Arts</td><td>0.78</td></tr><tr><td>Conservatorium of Music</td><td>0.81</td></tr><tr><td>Dentistry</td><td>0.75</td></tr><tr><td>Economics and Business</td><td>0.69</td></tr><tr><td>Education and Social Work</td><td>0.77</td></tr><tr><td>Engineering</td><td>0.74</td></tr><tr><td>Health Sciences</td><td>0.75</td></tr><tr><td>Law</td><td>0.72</td></tr><tr><td>Medicine</td><td>0.60</td></tr><tr><td>Nursing & Midwifery</td><td>0.69</td></tr><tr><td>Pharmacy</td><td>0.64</td></tr><tr><td>Science</td><td>0.69</td></tr><tr><td>Sydney College of the Arts</td><td>0.75</td></tr><tr><td>Veterinary Science</td><td>0.66</td></tr></tbody></table> </ephtml> </p> <p>5 SCEQ, Student Course Experience Questionnaire.</p> <hd id="AN0041435973-12">Discussion</hd> <p>Our goal for the present study was to discuss the development of an emerging construct to help measure the quality of e‐learning at the university degree level for use in a learning context in which online activities are used to complement face‐to‐face experiences. A secondary goal is to use this experience to discuss the benefits and challenges of evaluating parts in relation to wholes in ways that inform the providers of the part. We drew on previous research on students' experiences of distance and blended learning (eg, [<reflink idref="bib20" id="ref47">20</reflink>]; [<reflink idref="bib26" id="ref48">26</reflink>]; [<reflink idref="bib27" id="ref49">27</reflink>]) to develop a brief e‐learning scale. Using confirmatory factor analysis, we examined whether the proposed items of e‐learning scale formed a scale which was distinct from other core scales of a degree‐level teaching evaluation instrument already in use within our university. The model fit statistics indicated that the proposed scale captures a distinct aspect of the overall student experience, with the items having good levels of internal consistency in the sample we examined. The e‐learning scale scores correlated with important criteria used in previous examinations of the CEQ (cf. [<reflink idref="bib35" id="ref50">35</reflink>]), Overall Satisfaction with Degree Quality and self‐ratings of Generic Skills development, providing evidence of the criterion validity of the scale. Lastly, inter‐rater agreement analyses indicated the suitability of aggregating e‐learning scale scores up to the faculty level from the degree level, an important finding given the responsibilities of faculties within our university for improving the student experience.</p> <p>Several limitations of the present study should be acknowledged. While the results of this study suggest the e‐learning scale has suitable psychometric properties for use as a teaching performance indicator at the degree level, replications in other institutions are needed to determine if this suitability extends past the current setting. While the validity of the e‐learning scale was examined through correlations with overall ratings of degree quality and self‐ratings of Generic Skills development, future studies might examine other criteria, such as approaches to study or Grade Point Average (cf. [<reflink idref="bib35" id="ref51">35</reflink>]).</p> <p>More broadly, one of the difficulties for evaluating e‐learning in a part–whole relationship with the student learning experience is abstracting the key aspects of the e‐learning part sufficiently well that their meaning transcends contextual variation among universities who seek to use such a scale to evaluate e‐learning. E‐learning as a part of a broader experience of university student learning is still in a stage of rapid growth internationally. One only needs to look at raw statistics provided by national bodies on e‐learning uptake to understand that we are still in the early stages of integrating e‐learning across higher education sectors (eg, [<reflink idref="bib21" id="ref52">21</reflink>], [<reflink idref="bib22" id="ref53">22</reflink>]; [<reflink idref="bib36" id="ref54">36</reflink>]). Until e‐learning is more deeply and meaningfully integrated into the student experience of learning in all disciplines across higher education sectors, the structure of a theoretical construct of e‐learning is likely to continue to shift.</p> <p>Stakeholders who seek to act on the results of analyses from an e‐learning scale, such as the one proposed in this paper, would do well to take care in its interpretation. The more e‐learning is integrated into the fabric of academic enterprise and university infrastructure, the more challenging it may become to identify which parts of the university correlate to the students' perception when answering one of the items in the scale. The items that make up the e‐learning scale in this study include information technology, online resources, online communicating and active learning, If we take information technology as an example, the success of its provision in the student learning experience can be dependent on (<reflink idref="bib1" id="ref55">1</reflink>) the university IT unit providing a robust and stable platform, (<reflink idref="bib2" id="ref56">2</reflink>) the university e‐learning unit providing timely and sufficiently informed email and phone support to resolve student queries, and (<reflink idref="bib3" id="ref57">3</reflink>) the teacher meaningfully integrating a use of the e‐learning activities into the curriculum which relate to the students' outcomes. If any of these areas fail to support the student experience in the provision of e‐learning appropriately, then a student could be legitimately expected to give this item a poor rating.</p> <p>The observation of a dependency on a single questionnaire item on many parts of the university performing well raises the question of how we might simplify the data collection and interpretation cycle. One way might be to provide items which pull apart some of the embedded issues in single items more precisely into separate ones. The difficulty with this is that it may not always be clear to the student which part of the university is responsible for a single learning activity supported with ICTs. Another difficulty is that there is a limited space for systematic evaluation at the end of a course. A third challenge for a university seeking to systematically evaluate e‐learning across its faculties is to have a sufficiently broad use of e‐learning for its presence to be picked up in whole‐of‐institution questionnaires, such as the one discussed in this paper.</p> <p>It is clear that e‐learning is a growing and important <emph>part</emph> of student experiences of learning at a university internationally. It is also clear that we need sustainable and meaningful ways of evaluating its contribution to student understanding that help identify its appropriate place and role in a holistic experience involving face‐to‐face interaction. The e‐learning scale discussed in this paper offers an emerging pathway for conceptualising how we might achieve these now core goals of the Academy.</p> <ref id="AN0041435973-13"> <title> References </title> <blist> <bibl id="bib1" idref="ref9" type="bt">1</bibl> <bibtext> Bangert, A. W. (2004). The seven principles of good practice: a framework for evaluating on‐line teaching. Internet and Higher Education, 7, 217 – 232.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref23" type="bt">2</bibl> <bibtext> Barrie, S. & Ginns, P. (2007). The linking of institutional performance indicators to improvements in teaching and learning in classrooms. Quality in Higher Education, 13, 275 – 286.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref22" type="bt">3</bibl> <bibtext> Barrie, S., Ginns, P. & Prosser, M. (2005). Early impact and outcomes of an institutionally aligned, student focused learning perspective on teaching quality assurance. Assessment and Evaluation in Higher Education, 30, 641 – 656.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref1" type="bt">4</bibl> <bibtext> Bell, M., Bush, D., Nicholson, P., O'Brien, D. & Tran, T. (1999). Universities online: a survey of online education and services in Australia. Canberra, Australia: DEST.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref12" type="bt">5</bibl> <bibtext> Biggs, J. (2003). Teaching for quality learning at university. Maidenhead: Open University Press.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref16" type="bt">6</bibl> <bibtext> Biggs, J., Kember, D. & Leung, D. Y. P. (2001). The revised two‐factor Study Process Questionnaire: R‐SPQ‐2F. British Journal of Educational Psychology, 71, 133 – 149.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref30" type="bt">7</bibl> <bibtext> Boomsma, A. & Hoogland, J. J. (2001). The robustness of LISREL modeling revisited. In R. Cudeck, S. Du Toit & D. Sörbom (Eds), Structural equation modeling: Present and future (pp. 139 – 168). Lincolnwood, IL: Scientific Software International.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref24" type="bt">8</bibl> <bibtext> Brew, A. & Ginns, P. (in press). The relationship between engagement in the scholarship of teaching and learning and students' course experiences. Assessment and Evaluation. Higher Education.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref40" type="bt">9</bibl> <bibtext> Burke, M. J. & Dunlap, W. P. (2002). Estimating interrater agreement with the average deviation index: a user's guide. Organizational Research Methods, 5, 159 – 172.</bibtext> </blist> <blist> <bibtext> Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: a typology of composition models. Journal of Applied Psychology, 83, 234 – 246.</bibtext> </blist> <blist> <bibtext> Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.</bibtext> </blist> <blist> <bibtext> Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 35, 297 – 334.</bibtext> </blist> <blist> <bibtext> Cuban, L. (1986). Teachers and machines: the classroom use of technology since 1920. New York: Teachers College Press.</bibtext> </blist> <blist> <bibtext> HEFCE (Higher Education Funding Council of England) (2005). HEFCE strategy for e‐learning. Retrieved October 26, 2007, from <ulink href="http://www.hefce.ac.uk/pubs/hefce/2005/05%5f12/">http://www.hefce.ac.uk/pubs/hefce/2005/05%5f12/</ulink></bibtext> </blist> <blist> <bibtext> Entwistle, N. J. & McCune, V. (2004). The conceptual bases of study strategy inventories, Educational Psychology Review, 16, 325 – 345.</bibtext> </blist> <blist> <bibtext> Entwistle, N. J. & Ramsden, P. (1983). Understanding student learning. London: Croom Helm.</bibtext> </blist> <blist> <bibtext> Fan, X. & Thompson, B. (2001). Confidence intervals around score reliability coefficients, please: an EPM guidelines editorial. Educational & Psychological Measurement, 61, 517 – 531.</bibtext> </blist> <blist> <bibtext> Ginns, P. & Ellis, R. (2007). Quality in blended learning: Exploring the relations between on‐line and face‐to‐face teaching and learning. Internet and Higher Education, 10, 53 – 64.</bibtext> </blist> <blist> <bibtext> Ginns, P., Prosser, M. & Barrie, S. (2007). Students' perceptions of teaching quality in higher education: the perspective of currently enrolled students. Studies in Higher Education, 32, 603 – 615.</bibtext> </blist> <blist> <bibtext> Goodyear, P., Jones, C., Asensio, M., Hodgson, V. & Steeples, C. (2005). Networked learning in higher education: students' expectations and experiences. Higher Education, 50, 473 – 508.</bibtext> </blist> <blist> <bibtext> Hawkins, B. & Rudy, J. A. (2004). EDUCAUSE Core Data Service 2003 Summary Report. Retrieved April 23, 2008, from <ulink href="http://connect.educause.edu/library/abstract/EDUCAUSECoreDataServ/43038">http://connect.educause.edu/library/abstract/EDUCAUSECoreDataServ/43038</ulink></bibtext> </blist> <blist> <bibtext> Hawkins, B. & Rudy, J. A. (2007). EDUCAUSE Core Data Service: Fiscal Year 2006 Summary Report. Retrieved April 23, 2008, from <ulink href="http://connect.educause.edu/library/abstract/EDUCAUSECoreDataServ/45236">http://connect.educause.edu/library/abstract/EDUCAUSECoreDataServ/45236</ulink></bibtext> </blist> <blist> <bibtext> Jochems, W., Van Merriënboer, J. & Koper, R. (2004). An introduction to integrated e‐learning. In W. Jochems, J. Van Merriënboer & R. Koper (Eds), Integrated e‐learning: Implications for pedagogy, technology and organization (pp. 1 – 12). London: RoutledgeFalmer.</bibtext> </blist> <blist> <bibtext> Jöreskog, K. G. & Sörbom, D. (2001). LISREL 8 user's reference guide. Lincolnwood, IL: Scientific Software International.</bibtext> </blist> <blist> <bibtext> Katz, R. N. (1999). Dancing with the devil: information technology and the new competition in higher education. San Francisco, CA: Jossey Bass.</bibtext> </blist> <blist> <bibtext> Laurillard, D. (2002). Rethinking university teaching: a framework for the effective use of educational technology (2nd ed.). London: Routledge.</bibtext> </blist> <blist> <bibtext> Lawless, C. J. & Richardson, J. T. E. (2002). Approaches to study and perceptions of academic quality in distance education. Higher Education, 44, 257 – 282.</bibtext> </blist> <blist> <bibtext> Prosser, M. & Trigwell, K. (1999). Understanding learning and teaching: the experience in higher education. Buckingham: Open University Press.</bibtext> </blist> <blist> <bibtext> Ramsden, P. (1991). A performance indicator of teaching quality in higher education: the Course Experience Questionnaire. Studies in Higher Education, 16, 129 – 150.</bibtext> </blist> <blist> <bibtext> Ramsden, P. (2003). Learning to teach in higher education. London: Routledge.</bibtext> </blist> <blist> <bibtext> Raykov, T., & Marcoulides, G. A. (2000). A first course in structural equation modelling. Mahwah, NJ: Lawrence Erlbaum Associates.</bibtext> </blist> <blist> <bibtext> Richardson, J. T. E. (2005). Instruments for obtaining student feedback: a review of the literature. Assessment & Evaluation in Higher Education, 30, 387 – 415.</bibtext> </blist> <blist> <bibtext> Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350 – 353.</bibtext> </blist> <blist> <bibtext> Spector, P. (2001). Research methods in industrial and organisational psychology: data collection and data analysis with special consideration to international issues. In N. Anderson, D. S. Ones, H. K. Sinangil & C. Viswesvaran (Eds), Handbook of industrial, work and organizational psychology (pp. 10 – 26). London: Sage.</bibtext> </blist> <blist> <bibtext> Wilson, K. L., Lizzio, A. & Ramsden, P. (1997). The development, validation and application of the Course Experience Questionnaire. Studies in Higher Education, 22, 33 – 53.</bibtext> </blist> <blist> <bibtext> Zastrocky, M., Harris, M. & Lowendahl, J. (2007). E‐learning for higher education: course management systems. Stanford, CA: Gartner Research.</bibtext> </blist> </ref> <aug> <p>By Paul Ginns and Rob A. Ellis</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib25" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib13" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib14" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib23" firstref="ref8"></nolink> <nolink nlid="nl5" bibid="bib18" firstref="ref10"></nolink> <nolink nlid="nl6" bibid="bib19" firstref="ref11"></nolink> <nolink nlid="nl7" bibid="bib16" firstref="ref13"></nolink> <nolink nlid="nl8" bibid="bib28" firstref="ref14"></nolink> <nolink nlid="nl9" bibid="bib30" firstref="ref15"></nolink> <nolink nlid="nl10" bibid="bib15" firstref="ref17"></nolink> <nolink nlid="nl11" bibid="bib29" firstref="ref18"></nolink> <nolink nlid="nl12" bibid="bib35" firstref="ref19"></nolink> <nolink nlid="nl13" bibid="bib32" firstref="ref20"></nolink> <nolink nlid="nl14" bibid="bib20" firstref="ref26"></nolink> <nolink nlid="nl15" bibid="bib26" firstref="ref27"></nolink> <nolink nlid="nl16" bibid="bib27" firstref="ref28"></nolink> <nolink nlid="nl17" bibid="bib24" firstref="ref29"></nolink> <nolink nlid="nl18" bibid="bib31" firstref="ref31"></nolink> <nolink nlid="nl19" bibid="bib34" firstref="ref33"></nolink> <nolink nlid="nl20" bibid="bib12" firstref="ref37"></nolink> <nolink nlid="nl21" bibid="bib17" firstref="ref38"></nolink> <nolink nlid="nl22" bibid="bib10" firstref="ref39"></nolink> <nolink nlid="nl23" bibid="bib11" firstref="ref43"></nolink> <nolink nlid="nl24" bibid="bib33" firstref="ref45"></nolink> <nolink nlid="nl25" bibid="bib21" firstref="ref52"></nolink> <nolink nlid="nl26" bibid="bib22" firstref="ref53"></nolink> <nolink nlid="nl27" bibid="bib36" firstref="ref54"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ843502
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Evaluating the Quality of E-Learning at the Degree Level in the Student Experience of Blended Learning
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Ginns%2C+Paul%22">Ginns, Paul</searchLink><br /><searchLink fieldCode="AR" term="%22Ellis%2C+Rob+A%2E%22">Ellis, Rob A.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22British+Journal+of+Educational+Technology%22"><i>British Journal of Educational Technology</i></searchLink>. Jul 2009 40(4):652-663.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Blackwell Publishing. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8599; Fax: 781-388-8232; e-mail: customerservices@blackwellpublishing.com; Web site: http://www.blackwellpublishing.com/jnl_default.asp
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: PhysDesc
  Label: Physical Description
  Group: PhysDesc
  Data: PDF
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 12
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2009
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Evaluative
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Quality+Control%22">Quality Control</searchLink><br /><searchLink fieldCode="DE" term="%22Psychometrics%22">Psychometrics</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Experience%22">Student Experience</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Experience%22">Learning Experience</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Rating+Scales%22">Rating Scales</searchLink><br /><searchLink fieldCode="DE" term="%22Course+Evaluation%22">Course Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Technology%22">Educational Technology</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Integration%22">Technology Integration</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+Learning%22">Electronic Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Teaching+Methods%22">Teaching Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/j.1467-8535.2008.00861.x
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0007-1013
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This paper reports on the development of a scale for determining the quality of the student e-learning experience at the degree level when the student learning context is predominately a campus-based experience. Rapid developments in the use of information and communication technologies (ICT) in higher education require methods for evaluating the contribution of such tools to student learning, especially when they are complementing a face-to-face experience. We examine the psychometric functioning of a proposed e-learning scale in relation to a well-validated degree-level teaching evaluation instrument, the Student Course Experience Questionnaire. The e-learning scale has suitable reliability and validity in the present sample of undergraduate students for quality assurance activities around ICT.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 33
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2009
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ843502
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ843502
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/j.1467-8535.2008.00861.x
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 652
    Subjects:
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Quality Control
        Type: general
      – SubjectFull: Psychometrics
        Type: general
      – SubjectFull: Student Experience
        Type: general
      – SubjectFull: Learning Experience
        Type: general
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Rating Scales
        Type: general
      – SubjectFull: Course Evaluation
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Educational Technology
        Type: general
      – SubjectFull: Technology Integration
        Type: general
      – SubjectFull: Computer Software
        Type: general
      – SubjectFull: Electronic Learning
        Type: general
      – SubjectFull: Teaching Methods
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
    Titles:
      – TitleFull: Evaluating the Quality of E-Learning at the Degree Level in the Student Experience of Blended Learning
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ginns, Paul
      – PersonEntity:
          Name:
            NameFull: Ellis, Rob A.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Type: published
              Y: 2009
          Identifiers:
            – Type: issn-print
              Value: 0007-1013
          Numbering:
            – Type: volume
              Value: 40
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: British Journal of Educational Technology
              Type: main
ResultId 1