View in EDS HTML Full Text PDF Full Text

Statistical Edutainment: An Experimental Way to Teach for Good Measure

Saved in:
Bibliographic Details
Title: Statistical Edutainment: An Experimental Way to Teach for Good Measure
Language: English
Authors: Pearl, Dennis K. (ORCID 0000-0003-1981-1826), Lesser, Lawrence M. (ORCID 0000-0001-5762-3987)
Source: Teaching Statistics: An International Journal for Teachers. Spr 2023 45(1):45-56.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 12
Publication Date: 2023
Document Type: Journal Articles
Reports - Descriptive
Descriptors: Statistics Education, Experiments, Measurement, Cartoons, Captions
DOI: 10.1111/test.12323
ISSN: 0141-982X
1467-9639
Abstract: Concepts of experimentation and measurement are explored using statistics educational fun items and illustrated by sharing our process in conducting an experiment on cartoon captions.
Abstractor: As Provided
Entry Date: 2023
Accession Number: EJ1362718
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHIYpiH3fq2fLOZ86zw-VnKAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDLz9w9ctxF5kO2-OvgIBEICBmzFGVqdeUGcKiWblJIGUtZltobAlc5JFpXIECEsY0bdhjIuOA-6nkS3Y481iTJ0IQXXJ14Lt3j8nWJyMTPFRtbrHTuPkU0n5owksIuGD1Av5u5mGCGQPVrfGxWCp_GX8SYL5UKKEXZXYZ-t6U_e3WKseB7b0dNC7joAN9tWvTUhzAuXaZqpEHpj1DLQUNiEZg2Q9FoxNXFEB2eZH
Text:
  Availability: 1
  Value: <anid>AN0161395007;d8y01jan.23;2023Jan23.02:37;v2.2.500</anid> <title id="AN0161395007-1">Statistical edutainment: An experimental way to teach for good measure </title> <p>Concepts of experimentation and measurement are explored using statistics educational fun items and illustrated by sharing our process in conducting an experiment on cartoon captions.</p> <p>Keywords: teaching; cartoon captions; experimental design; joke; quote; song; teaching statistics</p> <hd id="AN0161395007-2">INTRODUCTION</hd> <p>Our prior column [[<reflink idref="bib32" id="ref1">32</reflink>]] announced an informal online pilot study (if you participated, thank you!) to explore ideas we have about the qualities of captions of educational cartoons for learning statistics by having respondents rate the educational and entertainment value of captions for six cartoons.</p> <p>The consent statement referred to the study as a survey, but the study design was actually an experiment within a survey because different respondents were randomly selected to receive a different set of six captions for those six cartoons. This article first considers identifying issues of interest and associated research questions about what makes a good caption for an educational cartoon. We then return to the general situation to describe learning objectives on experimentation and measures, and discuss educational fun items which can be used in teaching the key concepts in these objectives. The Appendix shares our process with the decisions and challenges we faced in setting up our experiment and how it connects to the learning objectives in Section 3 below on experimentation and measures so our experiment can be used as a classroom example.</p> <hd id="AN0161395007-3">IDENTIFYING THE RESEARCH ISSUES AND QUESTIONS</hd> <p>The importance of experimentation in science is captured by the Figure 1 collage of quotations by famous men and women. The key initial step is to establish the research questions you hope to study.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/D8Y/01jan23/test12323-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="test12323-fig-0001.jpg" title="1 Quotes on experimentation and measurement from the annotated collections at [[6]] and CAUSEweb.org/fun" /> </p> <p></p> <p>The topic of optimizing the value of cartoon captions is worthy of study because of the many published statistics lessons driven by syndicated comic strips [[<reflink idref="bib21" id="ref2">21</reflink>], [<reflink idref="bib23" id="ref3">23</reflink>], [<reflink idref="bib25" id="ref4">25</reflink>], [<reflink idref="bib27" id="ref5">27</reflink>]]; also, see the Cartoon Corner feature in the January‐February 1996, April 2000, and October 2008 issues of <emph>Mathematics Teaching in the Middle School</emph>] or cartoons [[<reflink idref="bib9" id="ref6">9</reflink>]] and the entirely cartoon‐based statistics textbooks [[<reflink idref="bib7" id="ref7">7</reflink>], [<reflink idref="bib14" id="ref8">14</reflink>]]. Studies on using cartoons in learning statistics [[<reflink idref="bib33" id="ref9">33</reflink>], [<reflink idref="bib38" id="ref10">38</reflink>]], however, have generally not yielded a strong positive effect on student performance, even if students enjoyed them and reported lower anxiety—an important endpoint in its own right. The studies generally chose cartoons based on alignment with content topics but did not also elaborate on specific qualities or standards of quality that the cartoons used had to meet, as if one cartoon is as good as another. This perhaps makes it all the more important to identify characteristics of effective (or at least well‐rated) cartoons to see if there is anything about cartoon captions that could be optimized or at least improved with respect to both educational value as well as entertainment value.</p> <p>The broader literature (ie, beyond statistics education) helps refine this question to get a sense of salient variables and hypotheses. For example, published cartoons and cartoon contests used only for entertainment can likely inform us about the entertainment value of a caption but need more investigation regarding properties affecting educational value. Thus, our experiment asked questions about those two dimensions separately for each of the six cartoons presented. One of the world's most famous cartoon caption contests has been run weekly since 2005 by <emph>The New Yorker</emph>, and a statistical and textual analysis [[<reflink idref="bib37" id="ref11">37</reflink>]] found that the following four properties distinguished contest entries that were shortlisted: more brevity, more novelty (fewer common words), less punctuation, and more abstractness/imaginability. Data science classes could do their own analysis of this contest's publicly available data [[<reflink idref="bib12" id="ref12">12</reflink>]], perhaps informed by more sophisticated analyses such as [[<reflink idref="bib41" id="ref13">41</reflink>]] or [[<reflink idref="bib35" id="ref14">35</reflink>]].</p> <p>The trait of brevity is well supported, as most comedy writers aim for conciseness to give their jokes more "punch." And indeed, the submission page of a statistics cartoon caption contest (https://CAUSEweb.org/caption-contest) advises entrants that "A good caption for the competition will be both well written (words like "humorous," "sharp," "catchy," or "original" might apply) as well as being tied to a specific learning objective that would be taught in an undergraduate statistics course. Preference is given for brevity." This led to our first set of research questions: <emph>Are shorter statistics cartoon captions viewed as more entertaining? Are they also viewed as having more educational value?</emph></p> <p>The trait of abstractness/imaginability suggests a possible effect for the degree to which the caption is independent from the visual part of the cartoon. The possible importance to assess interaction between caption and picture was reinforced by research that used eye tracking measures on these components of cartoons [[<reflink idref="bib3" id="ref15">3</reflink>]]. If the visual back‐and‐forth between caption and picture is a sign of engagement rather than distraction/confusion, then the benefits to educational value would likely be important. This leads to our second set of research questions: <emph>Is a caption that more readily works as a "stand‐alone joke" (with little or no dependence on the picture) viewed as more entertaining? Is such a caption also viewed as having more educational value?</emph></p> <p>Of course, the reception a cartoon receives may also be affected by the characteristics of the viewer, so we also collected data on gender, teaching experience, and age of students with an eye toward including them as covariates in the statistical analysis of the data. Thus, we reached a consensus on two major variables to explore: the caption's length and its dependence on the visual.</p> <p>In conducting this or any other experiment, what are the key statistical issues that arise? We now look in general at the issues of designing an experiment and then move on to issues of measurement that we would want our students to know. Section 4 provides some edutainment examples, which could be useful in teaching these issues. The Appendix provides an extended example of connecting to the learning objectives within the discussion of our processes in setting up our experiment.</p> <hd id="AN0161395007-5">LEARNING OBJECTIVES</hd> <p></p> <hd id="AN0161395007-6">Experimentation</hd> <p>The overall goal is for students to be able to understand the language, purpose, components of good protocols to allow for meaningful interpretation, and the ethics of experimentation. This yields the following learning objectives for experimentation:</p> <p>E1. Distinguish among an experiment, a quasi‐experiment, and an observational study.</p> <p>E2. Identify the explanatory and response variables in studies as well as potential confounding variables.</p> <p>E3. Only a good experiment can provide strong evidence of a cause‐and‐effect relationship, as observational studies are often subject to lurking variables or other potential confounders.</p> <p>E4. In a good experimental design, changes can be ascribed to the treatment rather than the alternate explanation of differences provided by a confounding factor.</p> <p>E5. Randomization of group assignments yields comparison groups that are similar with respect to confounding factors.</p> <p>E6. Understand the components of a well‐designed experiment, including randomization, control groups, blinding, and blocking.</p> <p>E7. Learn how to recognize common problems in experimentation: inability to generalize to relevant populations, lack of applicability to the "real world," bias from unblinded methodology, or a lack of attention to details.</p> <p>E8. Experimental designs that avoid high variability make it easier to separate the signal from the noise. Matching or blocking can help. Also, larger sample sizes improve the power to detect smaller signals.</p> <p>E9. Understand the key concepts of data ethics, including informed consent, confidentiality of personal data, and review of ethical considerations by an Institutional Review Board (IRB).</p> <hd id="AN0161395007-7">Measurement</hd> <p>Students need to understand how well the properties of the measured values are linked to what an experiment is designed to study. Our learning objectives for measurement are then:</p> <p>M1. Measurements will vary when repeated.</p> <p>M2. The reliability of a measurement is seen in the variability of independently repeated measurements.</p> <p>M3. The average of several measurements is less variable than a single measurement.</p> <p>M4. Understand the difference between validity, reliability, and bias in a measurement situation.</p> <p>M5. Repeated measurements cannot help you judge the level of bias, for that, you need an external standard.</p> <p>M6. To reduce bias, the experimenter must improve the measuring instrument.</p> <p>M7. The question of validity must always be addressed; it's often difficult to measure the variable of interest.</p> <p>M8. Distinguish among nominal/ordinal/interval/ratio variable types and how they relate to the appropriate statistical tools and the contextual questions to be examined.</p> <hd id="AN0161395007-8">EDUTAINMENT EXAMPLES</hd> <p></p> <hd id="AN0161395007-9">Experimentation</hd> <p>The effectiveness of using cartoons to explain basic principles of medical research was studied by a Dutch team [[<reflink idref="bib8" id="ref16">8</reflink>]] which developed a 44‐page cartoon book to foster informed consent for children taking part in experiments (a key aspect of objective E9). Figure 2 provides an example from the book that can be used to help students see the difference between research and treatment and between observational and experimental research. Beyond addressing objective E1, we also recommend that teachers tackle the issue of lexical ambiguity head on, as students may be confused by the varied usage of the words <emph>treatment</emph>, <emph>observation</emph>, and <emph>study</emph>. Because many terms associated with experiments overlap with terms used in other contexts, it is important to give students explicit opportunities to make appropriate conceptual distinctions.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/D8Y/01jan23/test12323-fig-0002.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="test12323-fig-0002.jpg" title="2 Cartoon courtesy of science illustrator Irene Cécile (irenececile.com)" /> </p> <p></p> <p>For example, have students discuss in groups the different uses of the word <emph>treatment</emph> in Figure 2 until they recognize that in the left panel it refers to something a doctor does (even if informed by past research studies) to help an individual patient, while in the right panel it refers to interventions whose effects are assessed for groups of study participants. Students should then discuss whether the panel on the right refers to an experiment or an observational study. Since neither term is explicitly used in the cartoon, students have to look for contextual clues, such as how assigning a medicine as a treatment is an intervention and thus part of an experiment. We believe this explicit discussion is needed because, for example, some students initially refer to an experiment as an observational study because they may see a news story that refers to the former as a "study" that involves making "observations."</p> <p>Many terms and concepts about experimental design are in a song parody that won the grand prize in the 2015 CAUSE A‐mu‐sing statistics fun items contest. The lyric, written by former University of Toronto Mississauga biology student and singer/guitarist Laura Krajewski [[<reflink idref="bib15" id="ref17">15</reflink>]], has this chorus:</p> <p>When we plan out our experiment,</p> <p>We must be sure to add</p> <p>A control, replication, randomization,</p> <p>And maybe even blocking ain't bad.</p> <p>An instructor can collect examples students find interesting by asking them to find experiments they see in the media (an out‐of‐class "scavenger hunt"). Guidance for this activity might be to enter "new study" into their search engine, find an interesting one, and then double check that it is experimental and not observational. The teacher then would use a few of these examples to focus discussions on specific learning objectives. Finally, to use Laura Krajewski's song in relating these ideas about experimentation in a different context, an out‐of‐class assignment then asks students to relate the issues discussed in class to specific lyrics in the song.</p> <p>In our cartoon caption study, we had the platform randomize the order of the six cartoons for each person. Also, for each cartoon, the person was randomly assigned one of two possible captions. These randomized components of the experimental design make the associated comparisons "fair" in the sense that regardless of the value of any possible confounder, an individual is just as likely to be in either of the groups being compared (objective E5). Random assignment helps make groups comparable for study but applies only to the population at hand which may not be the relevant population. The subjects in our study were a convenience sample—definitely not a random sample. This means our sample may not be representative of the population of statistics instructors (whose views on educational value are of interest) or of statistics students (whose views on entertainment value are of interest).</p> <p>Of course, random assignment also provides the underpinning for hypothesis testing when the null hypothesis relies only on the chance factors of the randomization. Teachers who cover the calculations underlying randomizing the order of three items might use an educational song that can help teach the permutations involved [[<reflink idref="bib28" id="ref18">28</reflink>]]. Our caption study also had a lesson for us on the important issue of data privacy (objective E9). In particular, our first draft of the announcement of the study promised that responses would be "...voluntary, anonymous, and quick." However, the IRB instructed us to change "anonymous" to "confidential" in our consent description because we could not promise anonymity, but merely that our protocol did not call for collecting data or metadata traceable back to the individual.</p> <p>The cartoons in the bottom half of Figure 3 help to focus students on the importance of a good control in any study (observational or experimental) and the value of blinding in an experiment (objectives E1, E4, and E7). To discuss objective E8, an instructor can use the cartoon in the upper left of Figure 3 that illustrates a whimsical matched pairs design with the roles of humans and rodents reversed and ask students why there could be less variability within a pair than between (eg, sharing the same cage; color of shirt; sleeve length; sex). Objective E8 is also the focus of the cartoon in the upper right of Figure 3 dealing with the value of thinking carefully about sample size when designing an experiment.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/D8Y/01jan23/test12323-fig-0003.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="test12323-fig-0003.jpg" title="3 Cartoons [[16], [18], [20]] to introduce concepts in experimental design" /> </p> <p></p> <p>To quickly introduce the idea that larger sample sizes can be helpful to detect differences between groups in objective E8 (but at the same time warn that it is important to avoid the large sample caution), an instructor might use this humorous story, followed by a discussion of power:</p> <p>A statistician had just calculated the optimal sample size for the new drug trial and was presenting to the pharmaceutical company board of directors, saying: "You should use <emph>n</emph> = 864 subjects in your experiment. A larger sample size is a waste of money." But the company president knew it would be a high‐profit item, so he replied, "I think I'll spend that money anyway." As the statistician walked back to her seat, she turned and said, "Well, more power to you!"</p> <p>Maintaining high ethical standards in all aspects of statistical practice is essential, especially in the design of experiments where human subjects might be involved, and this can be explored with in‐class discussion of major guidelines [[<reflink idref="bib1" id="ref19">1</reflink>], [<reflink idref="bib11" id="ref20">11</reflink>], [<reflink idref="bib42" id="ref21">42</reflink>]]. A fun item discussing some components of learning objective E9 is Greg Crowther's interactive song "Throw that Out?" [[<reflink idref="bib4" id="ref22">4</reflink>]]. After working through Crowther's song, instructors can work with real world experimental situations, discussing proper and improper participant consent. In doing so, we agree with Kardas and Spatz's recommendation [[<reflink idref="bib13" id="ref23">13</reflink>]] to start with positive examples.</p> <hd id="AN0161395007-12">Measurement</hd> <p>Here, we have only considered issues of designing an experiment, assuming we fully understand what and how we are measuring things. Moving from a concept to be examined to the operationalized measurements gathered requires careful thought about the properties of those measurements.</p> <p>A cartoon such as Figure 4 can be used to initiate discussion of the key measurement principles of reliability, bias, and validity [i.e., M1‐M4]. We recommend starting the unit by separating the issues of bias and reliability using the idea of the target practice analogy suggested by Figure 3 of [[<reflink idref="bib36" id="ref24">36</reflink>]]. Since reliability measures the consistency of a measurement process when repeated, it is seen in the closeness to each other of the points where the (archery or rifle) target is struck. On the other hand, bias is illustrated by how far those points are on average from the center of the innermost circle (bullseye). The target analogy can be explored via the interactive song "Target Practice" [[<reflink idref="bib31" id="ref25">31</reflink>]], which can be coupled with an app [[<reflink idref="bib44" id="ref26">44</reflink>]] that challenges students to lay out their own set of strike points on a target to represent high or low bias and high or low reliability.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/D8Y/01jan23/test12323-fig-0004.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="test12323-fig-0004.jpg" title="4 Cartoon [[17]] to introduce key measurement issues" /> </p> <p></p> <p>To help generalize the principle to how far you are from the parameter of interest, the target analogy could be followed by asking students to consider a scenario of a game of darts where strategy called for aiming for a spot other than the bullseye (eg, say you need to get a triple 20 to reach a specific total that wins the game). In this case, bias is measured by the distance from what you intended to hit.</p> <p>Another example would be to ask students how learning objectives M1 to M4 relate to the land distance measurement <emph>vara</emph> which originated in Spain five centuries ago and is still used in real estate deeds in some countries. Students will be surprised to learn (as they explore online) that the length of the <emph>vara</emph> varies among countries (eg, Spain, Colombia, Brazil) and even among states (eg, California, Colorado, Florida, Texas) of the United States!</p> <p>Related to objective M3 is the idea that <emph>independent</emph> observations are needed to judge reliability, and this can be discussed after this quip:</p> <p>"I wanted to know how good of a spouse I'd make so I married 10 times and took an average!"</p> <p>Discussions of bias and validity can be introduced with the 1955 quote by German physicist Werner Heisenberg in Figure 1, which suggests thinking about whether the questions answered by the data collected are aligned with the research questions of interest.</p> <p>Examples of interesting situations where it is difficult to operationalize the concept at hand are quite abundant and make for engaging discussions around objective M7. For example, the role of culture can affect even something as concrete as a linear measurement. For example, the Bascom method of estimating wave height does so from the front (ie, from the view of the beach), while a traditional Hawaiian method estimates height from the back of the wave. This would make a big difference ("desperate measures"?) to a surfer because the former can be up to twice the latter! Big wave surfer Buzzy Trent once said that "waves are not measured in feet and inches, but in increments of fear." Even if we agree on using feet, the question of how to measure the size of a wave is not trivial, especially since most students are far more experienced surfing the web than surfing waves! An instructor might try a classroom breakout into teams to try to define wave height and apply it to the wave in Figure 5. Then critique and compare the teams' different methods and estimates and how they relate to the methods mentioned earlier. This is also likely to help students understand objective M4 with respect to the difference between bias and other assaults on the validity of a measurement (eg, the bias from the inability to determine the height of the top of a wave due to the white water and spray vs. the decision about what aspects of a wave are relevant).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/D8Y/01jan23/test12323-fig-0005.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="test12323-fig-0005.jpg" title="5 Surfing wave needing its height measured" /> </p> <p></p> <p>If you feel your students would not be highly interested in the surfing example, another excellent example is the cross‐cultural measurement of happiness across nations. Teachers can ask what variables students imagine could be readily measured that would be a valid measure of happiness and then compare those to the ones actually reported in the Gallup survey's report [[<reflink idref="bib10" id="ref27">10</reflink>]].</p> <p>A third example for objective M7 might be introduced by the 1965 quote in Figure 1 by American economist and statistician Mollie Orshansky, who developed the definition of the poverty level used by the U.S. government and later adapted it for use in multiple countries. Ask students to brainstorm how they would form an appropriate measure of whether a household is above or below the poverty line.</p> <p>The classification of levels of measurement (objective M8) informs what statistical tools are appropriate [[<reflink idref="bib5" id="ref28">5</reflink>]] and also connects to the context and question of interest [[<reflink idref="bib43" id="ref29">43</reflink>]]. A poem [[<reflink idref="bib29" id="ref30">29</reflink>]] can be playfully read Dr. Seuss‐style to engage students in distinguishing measurement variables from tally counts for values of a categorical variable, inspired by a strategy in [[<reflink idref="bib24" id="ref31">24</reflink>]]. The <emph>Pre‐K‐12 GAISE Report</emph> [[<reflink idref="bib2" id="ref32">2</reflink>]] says (p. 9): "Counts, such as the number of pets a student has, are examples of a discrete quantitative variable. Measurements such as the length of a lizard's tail, are examples of a continuous quantitative variable. It is essential that students become comfortable with analyzing these traditional variable types." The interactive song "Levels of Measurement" [[<reflink idref="bib30" id="ref33">30</reflink>]] (and see Table 3 of [[<reflink idref="bib34" id="ref34">34</reflink>]]) gives students a pre‐song exercise to put the four levels of measurement in hierarchical order and then do a matching game where they pick (and get feedback on) real‐world examples of each type. Students will realize there are very few variables whose type is interval but not ratio.</p> <hd1 id="AN0161395007-15">"Levels of Measurement"</hd1> <p>Lyric © 2015 Lawrence M Lesser</p> <p>Music © 2015 Lawrence M Lesser & Dominic Dousa</p> <p>Nominal, ordinal, interval, ratio</p> <p>Are levels of measurement</p> <p>Let's show that progression</p> <p>With examples that we present</p> <p>With a variable that's nominal, values are just like names.</p> <p>So ordering or averaging religions would really be a shame!</p> <p>A variable that's ordinal sorts values like a chain, but don't assume with Likert scales each jump would mean the same!</p> <p>With a variable that's interval, differences are sound, but Fahrenheit ratios would only just confound.</p> <p>With a variable that's ratio, zero means there's none.</p> <p>And when it comes to incomes, two's twice as much as one!</p> <p>Examples help us learn what measurement levels are!</p> <p>An acronym recalls them: It's the French word NOIR!</p> <p>Is there a related game students could play in class? Yes, and it's in the "syllabus"—the journal <emph>Syllabus</emph>, that is [[<reflink idref="bib40" id="ref35">40</reflink>]]. It's based on Family Feud, a highly popular American television game show since 1976 which has spawned regional adaptations in over 50 international markets. Instructors can write and implement their own Opening Day fun survey in class to yield data for this activity.</p> <hd id="AN0161395007-16">CONCLUSIONS</hd> <p>A variety of quotes, cartoons, and songs provide a potentially effective means of teaching key learning objectives in experimentation and—just for good measure—about measurement issues. We find that students who have fewer technical skills and may shy away from discussions on more mathematical statistical topics are much more willing to participate in conversations about experiment and measurement objectives. This is especially true if the examples at hand are fun and engaging. Our small cartoon caption study, with its built‐in difficulties in defining appropriate measures and controls, can serve as a ready example in this regard. The Appendix shares our process with the decisions and challenges we faced in setting up our experiment and how it connects to the learning objectives in Section 3, focusing on how an instructor might use this example in the classroom. Our next column will be beyond measure as we discuss the findings of our cartoon study ("a panel discussion?"), as a vehicle to illustrate important ideas in the teaching of data visualization.</p> <hd id="AN0161395007-17">A APPENDIX</hd> <hd1 id="AN0161395007-18">Considerations associated with using the relative value of cartoon captions as a classroom example for teaching experimentation and measurement ideas</hd1> <p>Inspired by the literature on using cartoons to enhance learning, we wanted to examine two sets of research questions:</p> <p></p> <ulist> <item> <emph>Are shorter statistics cartoon captions viewed as more entertaining? Are they also viewed as having more educational value?</emph> </item> <p></p> <item> <emph>Is a caption that more readily works as a "stand‐alone joke" (with little or no dependence on the picture) viewed as more entertaining? Is such a caption also viewed as having more educational value?</emph> </item> </ulist> <p>Instructors who want to use this example in class can present the idea for the research to students and ask them to develop an experimental protocol to investigate it and discuss the validity, bias, and reliability of the data they would gather. In this appendix we provide a brief guide to issues that might arise, link them to the learning objectives in the paper, describe how they were handled in the experiment we ran, and give a few caveats that might come up in a classroom discussion. Descriptors for the learning objectives E1 to E9 on experimentation and M1 to M8 on measurement are given in the paper.</p> <p>As part of assigning this activity instructors might let students know that, since 2016, the Consortium for the Advancement of Undergraduate Statistics Education has run a monthly cartoon caption contest (your students are invited to enter at <ulink href="http://causeweb.org/caption-contest">http://causeweb.org/caption-contest</ulink>) that is judged by about a half‐dozen educators. Thus, we have a database of proposed captions and judges' rankings for about eighty cartoons all drawn (for the database) by the same artist. It is important for students to understand that many aspects of experimental design are driven by the specific resources available to the researchers. As examples, a biomedical researcher may have a tissue bank for a particular type of cancer that can be tested against different treatments, or a national transportation department might have a list of roads to resurface that can be tested for the resilience of a new surfacing material made with recycled plastics versus a traditional asphalt mix.</p> <p> <bold>Issue One:</bold> How can the resources available be used to conduct an experiment versus an observational study of our research questions?</p> <p> <bold>Related Objectives:</bold> E1, E3, E5, E7.</p> <p> <bold>Decision:</bold> We could take our database of cartoons and for each winning entry create a measure of how long the winning caption is and how much it relies on the visual features of the cartoon. We could then ask survey respondents to rate the educational and entertainment value for as many of these as possible. An analysis of this data would be an observational study that would have some positive features such as artistic quality being controlled since all are drawn by the same professional artist and all captions being deemed of high quality by multiple judges. But many confounding factors might give us pause in drawing conclusions such as differences in the importance of the learning objectives associated with the different cartoons. Instead, we decided to do an experiment by finding a group of 6 cartoons in our database where there were winners and honorable mentions with different properties of caption length (3 cartoons with the 2 caption options for each being of different lengths) or visual dependence (3 cartoons with the 2 caption options of each having different visual dependence) that we could randomly assign to a large number of subjects for their perceptions of educational value and entertainment value. We also randomized the order that the cartoons were presented in order to avoid bias (eg, people often prefer what they see first). These within‐cartoon random assignments more readily allowed for causal inference about the population from which the subjects were drawn.</p> <p> <bold>Caveats:</bold> To conduct the study we asked for volunteers from the readers of our <emph>Teaching Statistics</emph> column and from the group of instructors who are on the CAUSE eNEWS mailing list. While random‐assignment creates balance with respect to differences in the captions seen—the subjects in our study were a convenience sample ‐ definitely not a random sample. Thus, attempting to generalize our results beyond the group of subjects we used may be problematic.</p> <p>Also note that our database of 80 cartoons had only a small number of cartoons where opposing (with respect to our research questions) quality captions could be found for the same learning objectives. Thus, attempting to generalize all statistics topics may be problematic.</p> <p> <bold>Issue Two:</bold> How can we assure that our experimental data would be adequate to separate the signal from the noise?</p> <p> <bold>Related Objectives:</bold> E6, E8, M3.</p> <p> <bold>Decision:</bold> The caption contest resource available and the within‐cartoon design allowed us to control and reduce the variability due to issues like the quality of the artist and the importance of the learning objective and other characteristics associated with individual cartoons. Our next decision was then to estimate an appropriate sample size for the experiment. Even if students are not yet ready to think about a formal power analysis (eg, using tools like the one at https://<ulink href="http://www.stat.ubc.ca/%7Erollin/stats/ssize/n2),">www.stat.ubc.ca/%7Erollin/stats/ssize/n2),</ulink> they should have a feel for the broad theme involved. In particular, they should realize that averages of measurements are less variable than individual measurements, that the random error in surveys is smaller with increased sample size, and that a smaller "signal" can be detected with bigger sample sizes in experiments. In our experiment, we expected very different means for different cartoons for our 0 to 100 ratings of educational and entertainment value. However, we felt that it was reasonable to believe that the <emph>SD</emph> of ratings might be roughly similar and guessed it would be on the order of 20 or so. Using that information as a rough guide, we found that a sample size of about 200, giving us an expectation of 100 subjects for each caption version of a particular cartoon, would provide the desired understanding of the population we were looking for.</p> <p> <bold>Caveats:</bold> In carrying out the sample size estimation, we did not have a good feel for the size of the correlation between the results that was likely to occur between different cartoons making our estimate of needing <emph>n</emph> = 200 quite rough indeed. We also plan to use a model in our analyses that adjusts for characteristics of the subjects and the degree to which we could reduce the variability of our response by those adjustments is poorly known. Importantly, when researchers have a sample size goal in mind, it is not always easy to have a successful subject recruitment method in place to achieve the goal. This is particularly true of experiments that put a big burden on subjects relative to the possibility for their own benefits. Our experiment involves only 10 min of a subject's time and does not have stringent eligibility requirements—but also offers little in incentives for the subject.</p> <p> <bold>Issue Three:</bold> How can ethical considerations in the experiment be addressed?</p> <p> <bold>Related Objective:</bold> E9.</p> <p> <bold>Decision:</bold> Data privacy and the ability to obtain informed consent, especially when younger students are participating, are commonly the most important issue of experimental ethics in an education study. Suppose, for example, we had gone beyond <emph>instructor perceptions</emph> of the educational value of captions and instead decided to study actual <emph>student performance</emph> on learning assessments under different cartoon interventions. In that case the IRB for our study would require a great detail about how we keep those student academic records confidential to the research team and would focus a good deal of attention on whether the subjects fully understand the risks of accidental exposure of their academic records.</p> <p>Our caption study sought an exempt status from our IRB which requires less scrutiny due to the lower risks. As mentioned in the paper, our first draft of the announcement of the study promised that responses would be "...voluntary, anonymous, and quick." However, the IRB instructed us to change "anonymous" to "confidential" in our consent description because we could not promise anonymity, but merely that our protocol did not call for collecting data or metadata traceable back to the individual. After that change, the study was ruled as exempt from full IRB review and we were cleared to collect data.</p> <p> <bold>Caveats:</bold> The issue of data privacy also has implications for the granularity of the data that can be collected. For example, if we asked for too many details about demographic characteristics of participants, the results may readily point to just one or two possible people—a direct assault on data privacy. Along with data privacy and informed consent, we also needed to consider issues of diversity and inclusion in the items themselves (the questions asked as well as both the cartoons and their captions). For example, we made a point to include non‐binary options for the question about participant gender, but we did not require a response lest it potentially identify the respondent as a gender minority. Also, the fact that participants were partially recruited from readers of an international journal like <emph>Teaching Statistics</emph> might play a role here since, even amongst English speaking countries, innocuous humor in one culture may be viewed as not humorous or even offensive to some in a different culture. While we worked hard to avoid such cases, we will want to review the data to see if there are any hints of that issue in the open‐ended comments participants made in the survey.</p> <p> <bold>Issue Four:</bold> What potential confounders might interfere with interpreting study results?</p> <p> <bold>Related Objectives:</bold> E2, E3, E4.</p> <p> <bold>Decision:</bold> Students are likely to raise other variables they see as potential confounders in this study. Most confounders are dealt with by using random assignment ‐ but if the conversation turns to the issue of making statements about generalizing to the population of students or instructors outside of the group being surveyed, then many possibilities related to characteristics of the cartoon‐viewing respondents would be legitimate concerns. Be sure to focus the discussion on whether there would be a logical reason for the variable to be associated with the effect of caption length or the effect of ties to the visual <emph>and</emph> whether the proposed confounder is related to the difference between the actual population and the desired population. For example, we wondered if younger students might require longer captions that can better connect the meaning of the cartoon to the learning objective at hand, so instructors of younger students could see the educational value of a caption being related to caption length. As mentioned above, we also wondered if the sense of humor displayed in the captions (written by American participants in the CAUSE caption contest) might not elicit the same reactions from instructors in other countries. Importantly, we cannot tell from our data whether a response came from a U.S. instructor or not. Thus, unlike the age level of the students, there is no way to adjust for the possible effect of the country of the participant in our statistical modeling. Instructors might discuss the three types of situations here: some confounding factors are known and included in the data (allowing for model‐based adjustments), some are known and not included, and some confounding factors may simply be unknown to the researcher.</p> <p> <bold>Caveats:</bold> Though we expected that nearly everyone who took our survey would agree to let their data be used for research, it is possible that assumption is wrong. If a sizeable number do not agree, then any variable related to the reason why they did not want their data used would be a potential confounder. Also, note the important tradeoff between gathering more information from respondents vs a survey becoming burdensome on their time and consequently reducing our response rate.</p> <p> <bold>Issue Five:</bold> How were problems with the reliability, bias, or validity of the variables measured dealt with in the design?</p> <p> <bold>Related Objectives:</bold> M2, M5, M6, M7.</p> <p> <bold>Decision:</bold> This question should be discussed for any measures students propose. Here are some discussion points for a few of the measurements in our experiment.</p> <p>Issues of reliability in the variables measured were mainly dealt with by focusing on the more homogeneous within‐cartoon comparisons and by attempting to gather a reasonably large sample size.</p> <p>We assume that instructor perceptions of the educational value of a cartoon might depend heavily on the expectations and learning objectives associated with the standards of the local institutional setting of their classroom. In order to account for this potential source of bias, we asked respondents to provide an age category for their students. But the relationship between age category and these expectations/objectives varies across geographical location, leaving a (now smaller) potential source of bias unaccounted for in some areas.</p> <p>We also recognize that the default setting of 0 out of 100 for our entertainment and educational value questions could play a role in biasing results. Thus, we tried to avoid this problem by considering <emph>comparisons</emph> (where such biases would cancel themselves out) as our primary response variables.</p> <p>A class discussion might ask a question like: "If you want to know about student perceptions of the entertainment value of a cartoon—does the opinions of instructor responses to our survey provide a good measure of that?" Students should readily see that this presents an issue of validity and we must be careful to report our results only in terms of the population from which our respondents were drawn. Recognizing this obvious problem, we have left the experiment website open and in future research, will attempt to recruit a broad range of student participants.</p> <p> <bold>Caveats:</bold> It is important here to be sure that students understand that taking a larger sample size will not help with problems of validity, including bias (objective M5). For example, an instructor might ask: "Would the problem of getting responses <emph>only</emph> from instructors be helped by having 1,000 instead of just 200 respondents?"</p> <p> <bold>Issue Six:</bold> What variable types should be used for the data gathered?</p> <p> <bold>Related Objective:</bold> M8.</p> <p> <bold>Decision:</bold> Students should be able to recognize different variable types (nominal/ordinal/interval/ratio) as an aid to appropriately analyzing and interpreting data and realize there is some grey area in the distinctions, depending on the context. The table below provides such information about the variables in our study.</p> <p></p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Variable</th><th align="left">Scale type</th><th align="left">Comment</th></tr></thead><tbody valign="top"><tr><td>Educational and entertainment value ratings on 100‐point scale</td><td>Interval</td><td>A value of 50 probably does not represent twice the value as a value of 25. Also note that difference between two treatment types is interpretable.</td></tr><tr><td>Typical age of students taught by broad categories</td><td>Ordinal intended to relate to school level (elementary school, middle school, secondary school, intro college)</td><td>Could have chosen a ratio variable and then grouped.</td></tr><tr><td>Number of years taught</td><td>Ratio</td><td>Actual number requested—a poor decision if outlier values result in data privacy issues.</td></tr><tr><td>Type of respondent: Student, instructor, or other</td><td>Nominal</td><td>There were very few students in the population we sampled from</td></tr><tr><td>Gender</td><td>Nominal</td><td>Included non‐binary option and specific mention of right to refuse to answer.</td></tr><tr><td>Treatment variables: Long vs. short, visual independent vs. dependent captions</td><td>Nominal & binary</td><td>Could have created more granular measures but difficult to do so objectively.</td></tr></tbody></table> </ephtml> </p> <p> <bold>Caveats:</bold> We tended to favor simplicity of language and ease of response for our subjects in defining the variables measured in this pilot experiment. Of course, this leaves the possibility that we might have created a loss of information in some cases such as in the grouping of ages. It also may leave us with a lack of precise meaning in other cases. For example, we do not know what respondents really mean by "educational value" or "entertainment value" as operationalized by our 100‐point scale using sliders.</p> <hd1 id="AN0161395007-19">A final point</hd1> <p>In this Appendix, we have discussed a few of the issues arising in the development of the protocol for our cartoon caption pilot experiment. We will publish some results from the experiment in our next paper in this series, so stay "tooned"!</p> <p>Finally, we invite instructors to email with the best ideas arising from their class discussions and how they might improve on the pilot study we conducted.</p> <ref id="AN0161395007-20"> <title> REFERENCES </title> <blist> <bibl id="bib1" idref="ref19" type="bt">1</bibl> <bibtext> American Statistical Association, Ethical guidelines for statistical practice. Available at: https://<ulink href="http://www.amstat.org/your-career/ethical-guidelines-for-statistical-practice">www.amstat.org/your-career/ethical-guidelines-for-statistical-practice</ulink></bibtext> </blist> <blist> <bibl id="bib2" idref="ref32" type="bt">2</bibl> <bibtext> A. Bargagliotti et al., Pre‐K‐12 guidelines for assessment and instruction in statistics education II (GAISE II): A framework for statistics and data science education, American Statistical Association, Alexandria, VA, 2020.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref15" type="bt">3</bibl> <bibtext> P. J. Carroll, J. R. Young, and M. S. Guertin, Visual analysis of cartoons: A view from the far side, Eye movements and visual cognition: Scene perception and reading, K. Rayner (ed.), Springer, Berlin, Germany, 1992, pp. 444 – 461.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref22" type="bt">4</bibl> <bibtext> G. Crowther, Throw that Out?, 2016. Available at: https://CAUSEweb.org/cause/resources/fun/songs/throw-out ; Interactive version at https://<ulink href="http://www.CAUSEweb.org/smiles/songs/throw%5fthat%5fout">www.CAUSEweb.org/smiles/songs/throw%5fthat%5fout</ulink></bibtext> </blist> <blist> <bibl id="bib5" idref="ref28" type="bt">5</bibl> <bibtext> N. Friel and J. D. Mamer, It's a fird! Can you compute a median of categorical data? Math. Teach. Middle School 14 (2009), no. 6, 344 – 351.</bibtext> </blist> <blist> <bibl id="bib6" type="bt">6</bibl> <bibtext> C. C. Gaither and A. E. Cavazos‐Gaither, Statistically speaking: A dictionary of quotations, CRC Press, Boca Raton, FL, 1996.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref7" type="bt">7</bibl> <bibtext> L. Gonick and W. Smith, The Cartoon Guide to Statistics, Harper Collins, New York, 1993.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref16" type="bt">8</bibl> <bibtext> P. Grootens‐Wiegers, M. C. de Vries, M. M. van Beusekom, L. van Dijck, and J. M. van den Broek, Comic strips help children understand medical research: targeting the informed consent procedure to children's needs, Patient Educ. Couns. 98 (2015), no. 4, 518 – 524.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref6" type="bt">9</bibl> <bibtext> D. Guo and Y. Ma, The 'p‐hacking‐is‐terrific' ocean: A cartoon for teaching statistics, Teach. Stat. 44 (2022), no. 2, 68 – 72.</bibtext> </blist> <blist> <bibtext> J. F. Helliwell, R. Layard, J. D. Sachs, J.‐E. de Neve, L. B. Aknin, and S. Wang, Eds., World Happiness Report 2022, Sustainable Development Solutions Network, New York, 2022 Available at: https://happiness-report.s3.amazonaws.com/2022/WHR+22.pdf.</bibtext> </blist> <blist> <bibtext> International Statistical Institute, ISI declaration on professional ethics, Available at: https://<ulink href="http://www.isi-web.org/about/policies/professional-ethics/isi-declaration">www.isi-web.org/about/policies/professional-ethics/isi-declaration</ulink></bibtext> </blist> <blist> <bibtext> L. Jain, K. Jamieson, R. Mankoff, R. Nowak, and S. Sievert, The New Yorker cartoon caption contest dataset, 2020. Available at: https://nextml.github.io/caption-contest-data/</bibtext> </blist> <blist> <bibtext> E. P. Kardas and C. Spatz, Teaching ethics in research methods classes, Best practices for teaching statistics and research methods in the behavioral sciences, D. S. Dunn, R. A. Smith, and B. C. Beins (eds.), Lawrence Erlbaum, Mahwah, NJ, 2007, pp. 159 – 171.</bibtext> </blist> <blist> <bibtext> G. Klein and A. Dabney, The cartoon introduction to statistics, Hill and Wang, New York, NY, 2013.</bibtext> </blist> <blist> <bibtext> L. Krajewski, Our experiment, Available at: https://<ulink href="http://www.CAUSEweb.org/cause/resources/fun/songs/our‐experiment,">www.CAUSEweb.org/cause/resources/fun/songs/our‐experiment,</ulink> 2015.</bibtext> </blist> <blist> <bibtext> J. Landers, Control groups, Available at: https://<ulink href="http://www.CAUSEweb.org/cause/resources/fun/cartoons/control-groups,">www.CAUSEweb.org/cause/resources/fun/cartoons/control-groups,</ulink> 2003.</bibtext> </blist> <blist> <bibtext> J. Landers, The measurement machine, Available at: https://<ulink href="http://www.CAUSEweb.org/cause/resources/fun/cartoons/measurement-machine,">www.CAUSEweb.org/cause/resources/fun/cartoons/measurement-machine,</ulink> 2006.</bibtext> </blist> <blist> <bibtext> J. Landers, The placebo researcher, Available at: https://<ulink href="http://www.CAUSEweb.org/cause/resources/fun/cartoons/placebo‐researcher,">www.CAUSEweb.org/cause/resources/fun/cartoons/placebo‐researcher,</ulink> 2009.</bibtext> </blist> <blist> <bibtext> J. Landers, The rats' lab, Available at: https://<ulink href="http://www.CAUSEweb.org/cause/resources/fun/cartoons/rats-lab,">www.CAUSEweb.org/cause/resources/fun/cartoons/rats-lab,</ulink> 2017.</bibtext> </blist> <blist> <bibtext> J. Landers, Magic act, Available at: https://<ulink href="http://www.CAUSEweb.org/cause/resources/fun/cartoons/magic-act,">www.CAUSEweb.org/cause/resources/fun/cartoons/magic-act,</ulink> 2017.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, Safe Havens graph, Math. Teach. 101 (2008), no. 9, 690 – 692.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, The high school weirdness factor, Math. Teach. 103 (2010), no. 7, 474 – 477.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, Lottery lunacy, Math. Teach. 106 (2012), no. 2, 93 – 94.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, Using context to classify variables, Teach. Stat. 35 (2013), no. 1, 29 – 31. https://doi.org/10.1111/j.1467-9639.2012.00527.x.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, Normally speaking, Math. Teach. 108 (2015), no. 6, 408 – 410.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, The stature of salary: Raised expectations or a tall tale? Math. Teach. 110 (2016), no. 3, 168 – 169.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, Classroom notes: One in ten, Teach. Stat. 40 (2018), no. 1, 33 – 34.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, An [ordered] arrangement with no [lyric] repeats, Available at: https://<ulink href="http://www.CAUSEweb.org/cause/resources/fun/songs/an-ordered-arrangement,">www.CAUSEweb.org/cause/resources/fun/songs/an-ordered-arrangement,</ulink> 2018.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, Very able to know each variable, Available at: https://<ulink href="http://www.CAUSEweb.org/cause/resources/fun/poems/very-able-know-each-variable,">www.CAUSEweb.org/cause/resources/fun/poems/very-able-know-each-variable,</ulink> 2021.</bibtext> </blist> <blist> <bibtext> L. M. Lesser and D. M. Dousa, Levels of measurement, 2015. Available at: https://CAUSEweb.org/cause/resources/fun/songs/levels-measurement ; Interactive version at https://<ulink href="http://www.CAUSEweb.org/smiles/songs/levels%5fof%5fmeasurement">www.CAUSEweb.org/smiles/songs/levels%5fof%5fmeasurement</ulink></bibtext> </blist> <blist> <bibtext> L. M. Lesser and D. M. Dousa, Target practice, 2015, Available at: https://<ulink href="http://www.causeweb.org/cause/resources/fun/songs/target-practice">www.causeweb.org/cause/resources/fun/songs/target-practice</ulink> ; interactive version at https://<ulink href="http://www.CAUSEweb.org/smiles/songs/target%5fpractice">www.CAUSEweb.org/smiles/songs/target%5fpractice</ulink></bibtext> </blist> <blist> <bibtext> L. M. Lesser and D. K. Pearl, Statistical edutainment: you're the subject for our next subject! Teach. Stat. 44 (2022), no. 3, 133. https://doi.org/10.1111/test.12312.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, D. K. Pearl, and J. J. Weber, Assessing fun items' effectiveness in increasing learning of college introductory statistics students: Results of a randomized experiment, J. Stat. Educ. 24 (2016), no. 2, 54 – 62.</bibtext> </blist> <blist> <bibtext> L. M. Lesser, D. K. Pearl, J. J. Weber, D. M. Dousa, R. P. Carey, and S. A. Haddad, Developing interactive educational songs for introductory statistics, J. Stat. Educ. 27 (2019), no. 3, 238 – 252. https://doi.org/10.1080/10691898.2019.1677533.</bibtext> </blist> <blist> <bibtext> R. T. Li, Learning humor through AI: A study on New Yorker's cartoon caption contests using deep learning (2020). Dartmouth College Undergraduate Theses. 159. Available at: https://digitalcommons.dartmouth.edu/senior_theses/159</bibtext> </blist> <blist> <bibtext> M. A. Martin, "It's like... you know": The use of analogies and heuristics in teaching introductory statistical methods, J. Stat. Educ. 11 (2003), no. 2, 1 – 28. https://doi.org/10.1080/10691898.2003.11910705.</bibtext> </blist> <blist> <bibtext> P. McGraw, May 4, 2011 blogpost, Do you want to win The New Yorker's cartoon caption contest? Available at: https://petermcgraw.org/do‐you‐want‐to‐win‐the‐new‐yorkers‐cartoon‐caption‐contest/</bibtext> </blist> <blist> <bibtext> A. A. Özdoğru and R. F. McMorris, Humorous cartoons in college textbooks: Student perceptions and learning, Humor 26 (2013), no. 1, 135 – 154.</bibtext> </blist> <blist> <bibtext> S. P. Schacht and B. J. Stewart, What's so funny about statistics? A technique for reducing student anxiety, Teach. Sociol. 18 (1990), no. 1, 52 – 56.</bibtext> </blist> <blist> <bibtext> J. S. Seiter, T. Curran, and K. M. Scharp, And the survey says...: Using Family Feud to gather, understand, and analyze different levels of data, Syllabus 9 (2020), no. 2, 1 – 9 Available at: <ulink href="http://syllabusjournal.org/syllabus/issue/view/26">http://syllabusjournal.org/syllabus/issue/view/26</ulink>.</bibtext> </blist> <blist> <bibtext> D. Shahaf, E. Horvitz, and R. Mankoff, Inside jokes: identifying humorous cartoon captions, KDD'15: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015, 1065 – 1074. doi: https://doi.org/10.1145/2783258.2783388 Available at: https://erichorvitz.com/phumor.pdf. https://doi.org/10.1145/2783258.2783388</bibtext> </blist> <blist> <bibtext> United Nations Development Group, Data privacy, ethics and protection, Available at: https://unsdg.un.org/sites/default/files/UNDG_BigData_final_web.pdf</bibtext> </blist> <blist> <bibtext> P. F. Velleman and L. Wilkinson, Nominal, ordinal, interval, and ratio typologies are misleading, Am. Stat. 47 (1993), no. 1, 65 – 72. https://doi.org/10.1080/00031305.1993.10475938.</bibtext> </blist> <blist> <bibtext> Y. Zhang, C. Grey, N. J. Hatfield, and D. K. Pearl. (2020). Bias and reliability [R Shiny app]. Available at: https://psu-eberly.shinyapps.io/Bias_and_Reliability.</bibtext> </blist> </ref> <aug> <p>By Dennis K. Pearl and Lawrence M. Lesser</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib32" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib21" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib23" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib25" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib27" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib14" firstref="ref8"></nolink> <nolink nlid="nl7" bibid="bib33" firstref="ref9"></nolink> <nolink nlid="nl8" bibid="bib38" firstref="ref10"></nolink> <nolink nlid="nl9" bibid="bib37" firstref="ref11"></nolink> <nolink nlid="nl10" bibid="bib12" firstref="ref12"></nolink> <nolink nlid="nl11" bibid="bib41" firstref="ref13"></nolink> <nolink nlid="nl12" bibid="bib35" firstref="ref14"></nolink> <nolink nlid="nl13" bibid="bib15" firstref="ref17"></nolink> <nolink nlid="nl14" bibid="bib28" firstref="ref18"></nolink> <nolink nlid="nl15" bibid="bib11" firstref="ref20"></nolink> <nolink nlid="nl16" bibid="bib42" firstref="ref21"></nolink> <nolink nlid="nl17" bibid="bib13" firstref="ref23"></nolink> <nolink nlid="nl18" bibid="bib36" firstref="ref24"></nolink> <nolink nlid="nl19" bibid="bib31" firstref="ref25"></nolink> <nolink nlid="nl20" bibid="bib44" firstref="ref26"></nolink> <nolink nlid="nl21" bibid="bib10" firstref="ref27"></nolink> <nolink nlid="nl22" bibid="bib43" firstref="ref29"></nolink> <nolink nlid="nl23" bibid="bib29" firstref="ref30"></nolink> <nolink nlid="nl24" bibid="bib24" firstref="ref31"></nolink> <nolink nlid="nl25" bibid="bib30" firstref="ref33"></nolink> <nolink nlid="nl26" bibid="bib34" firstref="ref34"></nolink> <nolink nlid="nl27" bibid="bib40" firstref="ref35"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1362718
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Statistical Edutainment: An Experimental Way to Teach for Good Measure
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Pearl%2C+Dennis+K%2E%22">Pearl, Dennis K.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1981-1826">0000-0003-1981-1826</externalLink>)<br /><searchLink fieldCode="AR" term="%22Lesser%2C+Lawrence+M%2E%22">Lesser, Lawrence M.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5762-3987">0000-0001-5762-3987</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Teaching+Statistics%3A+An+International+Journal+for+Teachers%22"><i>Teaching Statistics: An International Journal for Teachers</i></searchLink>. Spr 2023 45(1):45-56.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 12
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Descriptive
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Statistics+Education%22">Statistics Education</searchLink><br /><searchLink fieldCode="DE" term="%22Experiments%22">Experiments</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement%22">Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Cartoons%22">Cartoons</searchLink><br /><searchLink fieldCode="DE" term="%22Captions%22">Captions</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/test.12323
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0141-982X<br />1467-9639
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Concepts of experimentation and measurement are explored using statistics educational fun items and illustrated by sharing our process in conducting an experiment on cartoon captions.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1362718
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1362718
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/test.12323
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 45
    Subjects:
      – SubjectFull: Statistics Education
        Type: general
      – SubjectFull: Experiments
        Type: general
      – SubjectFull: Measurement
        Type: general
      – SubjectFull: Cartoons
        Type: general
      – SubjectFull: Captions
        Type: general
    Titles:
      – TitleFull: Statistical Edutainment: An Experimental Way to Teach for Good Measure
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Pearl, Dennis K.
      – PersonEntity:
          Name:
            NameFull: Lesser, Lawrence M.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 0141-982X
            – Type: issn-electronic
              Value: 1467-9639
          Numbering:
            – Type: volume
              Value: 45
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Teaching Statistics: An International Journal for Teachers
              Type: main
ResultId 1