View in EDS HTML Full Text PDF Full Text

Norwegian Mathematics Teacher Educators' and Research Mathematicians' Views on Different Aspects of Mathematical Definitions: A Comparative Judgement Study

Saved in:
Bibliographic Details
Title: Norwegian Mathematics Teacher Educators' and Research Mathematicians' Views on Different Aspects of Mathematical Definitions: A Comparative Judgement Study
Language: English
Authors: Hermund André Torkildsen, Tore A. Forbregd, David A. Reid, Shaista Kanwal (ORCID 0000-0003-4753-5013)
Source: International Journal of Science and Mathematics Education. 2025 23(7):2157-2180.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 24
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Foreign Countries, Mathematics Education, Mathematics Teachers, Teacher Educators, Teacher Researchers, Teacher Attitudes, Mathematical Concepts, Definitions, Educational Research, Ambiguity (Semantics), Mathematical Logic, Comprehension
Geographic Terms: Norway
DOI: 10.1007/s10763-024-10534-7
ISSN: 1571-0068
1573-1774
Abstract: In this article we investigate the extent to which characteristics used to describe mathematical definitions in mathematics education research literature reflect what is important to research mathematicians and mathematics teacher educators. We report results from a comparative judgement survey of 57 research mathematicians and 62 mathematics teacher educators. Our results indicate that the two groups are mostly in agreement about the rankings, however, there are some differences. Research mathematicians rank most highly characteristics related to non-ambiguity and non-contradiction, while mathematics teacher educators rank characteristics related to comprehension and referential clarity as highly as characteristics related to non-ambiguity and non-contradiction. Neither group ranked minimality highly, even though this is the characteristic most often listed in the mathematics education research literature. Our study contributes towards a more consistent characterization of mathematical definitions in mathematics education research.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1493014
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGeZU_jATVnRgjtBRCKecUsAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDDQGuONbv0dMoiDYUAIBEICBmpjHr6aRu-2rHbfO3hd2INP7kiSb7U4w66F8SoBW0KggjIiB2y5ipYQv6cVswWYb0WMvI-NEuTqAJOxOJM7NrmQH0eJjP4eribiHU6pAWhvXdvyZGr1el7sMFYNYAZw2QZA75pIuQyRjhZ5OHvPMCunFoVCYrysCrk7qdPU1-ZgnLeCJoKGVfThgfOjup74wtZIV7N2I_ikCFKI=
Text:
  Availability: 1
  Value: <anid>AN0189055834;[3d0g]01oct.25;2025Nov05.04:00;v2.2.500</anid> <title id="AN0189055834-1">Norwegian Mathematics Teacher Educators' and Research Mathematicians' Views on Different Aspects of Mathematical Definitions: a Comparative Judgement Study </title> <p>In this article we investigate the extent to which characteristics used to describe mathematical definitions in mathematics education research literature reflect what is important to research mathematicians and mathematics teacher educators. We report results from a comparative judgement survey of 57 research mathematicians and 62 mathematics teacher educators. Our results indicate that the two groups are mostly in agreement about the rankings, however, there are some differences. Research mathematicians rank most highly characteristics related to non-ambiguity and non-contradiction, while mathematics teacher educators rank characteristics related to comprehension and referential clarity as highly as characteristics related to non-ambiguity and non-contradiction. Neither group ranked minimality highly, even though this is the characteristic most often listed in the mathematics education research literature. Our study contributes towards a more consistent characterization of mathematical definitions in mathematics education research.</p> <p>Keywords: Comparative judgement; Mathematical definitions; Mathematics education; Research mathematicians; Mathematical Sciences Pure Mathematics</p> <hd id="AN0189055834-2">Introduction</hd> <p>It has been claimed that mathematics should be taught and learned in a manner that mirrors, or at least respects, the nature of mathematics as a science (Ball & Bass, [<reflink idref="bib3" id="ref1">3</reflink>]; Lampert, [<reflink idref="bib22" id="ref2">22</reflink>]). If this is so then the way definitions are described in schools should be compatible with the way they are perceived by research mathematicians. Understanding definitions is an important part of understanding mathematics, both in university (Edwards & Ward, [<reflink idref="bib13" id="ref3">13</reflink>]) and in schools (Ball, [<reflink idref="bib2" id="ref4">2</reflink>]). Mathematical definitions are essential to mathematical proof (Edwards & Ward, [<reflink idref="bib13" id="ref5">13</reflink>]), but it is also important for students to engage in the process of defining as an activity in itself (Mariotti & Fischbein, [<reflink idref="bib25" id="ref6">25</reflink>]; Ouvrier-Buffet, [<reflink idref="bib29" id="ref7">29</reflink>]; Zandieh & Rasmussen, [<reflink idref="bib39" id="ref8">39</reflink>]).</p> <p>However, as Ouvrier-Buffet ([<reflink idref="bib28" id="ref9">28</reflink>]) notes "The definition of 'definition' cannot be taken for granted" (p. 259). What exactly is necessary for a definition, and what is desirable, is not always clear. Through a systematic literature review, Torkildsen et al. ([<reflink idref="bib35" id="ref10">35</reflink>]) identified four main themes namely Requirements, Preferred features, Role and function, and Nature, divided into sub-categories, used to characterize mathematical definitions (see details below). However, they noted that there is no agreement in the literature on which characteristics are required. For example, Van Dormolen and Zaslavsky ([<reflink idref="bib36" id="ref11">36</reflink>]) list four criteria they consider necessary and three others they consider desirable, but Harel et al. ([<reflink idref="bib15" id="ref12">15</reflink>]) identify nine requirements they consider universally agreed upon as well as three others (pp. 151–152).</p> <p>The twelve characteristics identified by Torkildsen et al. ([<reflink idref="bib35" id="ref13">35</reflink>]) (such as Exemplification, Formal, Axiomatization, and others listed in Fig. 1) are based on those found in research literature in mathematics education. One would hope that mathematical practice would be reflected in the characteristics of mathematical definitions used in the research literature, and that those definitions in turn would be adopted by mathematics teacher educators (MTEs) and passed on to future teachers. Our purpose here is to investigate the extent to which this is so. That is whether these characteristics reflect what is important to research mathematicians and mathematics teacher educators. Through this study, we seek to shed light, first on whether the definition of mathematical definition used in the mathematics education literature is consistent with that used by research mathematicians, and second whether that definition has been adopted by MTEs. We address these research questions:</p> <p></p> <ulist> <item> How are characteristics of definitions identified in the mathematics education research literature ranked in importance by research mathematicians?</item> <p></p> <item> How are characteristics of definitions identified in the mathematics education research literature ranked in importance by mathematics teacher educators?</item> <p></p> <item> How do the rankings of these two groups differ from the existing mathematics education literature?</item> </ulist> <p>Graph: Fig. 1 The four main themes with corresponding categories from Torkildsen et al. ([<reflink idref="bib35" id="ref14">35</reflink>])</p> <hd id="AN0189055834-3">Background</hd> <p>As Alcock and Simpson ([<reflink idref="bib1" id="ref15">1</reflink>]) point out, mathematical definitions are different from dictionary definitions of everyday words like "swan". A mathematical definition has "the property that everything satisfying it belongs to the corresponding category and that everything belonging to the category satisfies the definition" (p. 28). In contrast, given a dictionary definition of "swan" it is "neither possible to say with absolute certainty that everything satisfying the definition is a swan, nor that every swan satisfies the definition" (p. 28). But there is no mathematical definition of "mathematical definition". Instead, we see "mathematical definition" as an example of what Czocher and Weber ([<reflink idref="bib10" id="ref16">10</reflink>]) call a "cluster category" (p. 50 and following). That is, there is "a collection of properties that an object can satisfy to "count toward" category membership, but no single property is necessary or sufficient for category membership" (p. 59). With this in mind, we consider first what properties or criteria have been proposed in the literature as counting towards membership in the category "mathematical definition", and then how the technique of comparative judgment allows us to see differences in the criteria of two academic communities.</p> <hd id="AN0189055834-4">Characteristics of Mathematical Definitions</hd> <p>Van Dormolen and Zaslavsky ([<reflink idref="bib36" id="ref17">36</reflink>]) list four criteria of mathematical definitions they consider necessary: Hierarchy, Existence, Equivalence, Axiomatization (p. 93). They list three others that are "not necessary from a logical standpoint, but ... are part of a general culture" (p. 93). These include the criteria of Minimality, Elegance, and Degenerations.</p> <p>Briefly, Hierarchy requires that "any new concept must be described as a special case of a more general concept. One or more properties must be used to describe this special case" (p. 94). Existence requires that at least one example of the defined concept must exist. Equivalence states that if there are two or more definitions for a concept, they must be equivalent. Axiomatization requires that "a definition fits in and is part of a deductive system" (p. 95). Minimality "demands that no more properties of the concept be mentioned than is required for its existence" (p. 96). Elegance is a subjective criterion related to how briefly a definition can be stated, how easy it is to apply, or the generality of the concepts used in it. The final criterion, Degenerations, is that the definition should exclude cases that do not fit one's intuition of the concept. This is, of course, also subjective.</p> <p>Harel et al.'s ([<reflink idref="bib15" id="ref18">15</reflink>]) nine characteristics for mathematical definitions that are both required and, in their opinion, agreed upon in the literature include Hierarchy, Existence, and Equivalence corresponding to Van Dormolen and Zaslavsky's ([<reflink idref="bib36" id="ref19">36</reflink>]) criteria. They also add six new characteristics: Non-contradictory/internally consistent, Unambiguity, Invariance under change of representation, Suitability to their purpose, being Well-defined, and Usability. They list Minimality and Elegance as two of the characteristics they do not consider agreed upon, as well as being easily comprehended by students (pp. 151–152).</p> <p>Figure 1 shows the four main themes identified by Torkildsen et al. ([<reflink idref="bib35" id="ref20">35</reflink>]) through a systematic literature review of 124 research articles in mathematics education. The themes of Requirements and Preferred features refer directly to the criteria for definitions. Formal requirements include Harel et al.'s ([<reflink idref="bib15" id="ref21">15</reflink>]) characteristics of Consistency, Unambiguity and being Well-defined. Exemplification includes the characteristic of existence identified by both Harel et al. ([<reflink idref="bib15" id="ref22">15</reflink>]) and Van Dormolen and Zaslavsky ([<reflink idref="bib36" id="ref23">36</reflink>]), as well as Van Dormolen and Zaslavsky's ([<reflink idref="bib36" id="ref24">36</reflink>]) preferred criterion of Degenerations and Harel et al.'s ([<reflink idref="bib15" id="ref25">15</reflink>]) characteristics of Invariance under change of representation, as well as the property of being able to discriminate between instances and non-instances. Axiomatization encompasses Van Dormolen and Zaslavsky's ([<reflink idref="bib36" id="ref26">36</reflink>]) criteria of Hierarchy, Equivalence, and Axiomatization, and also using only previously defined concepts and avoiding circularity.</p> <p>Regarding the Preferred features, Torkildsen et al. ([<reflink idref="bib35" id="ref27">35</reflink>]) list Minimality, Aesthetic (which includes elegance) and Comprehension (which includes Harel et al.'s ([<reflink idref="bib15" id="ref28">15</reflink>]) characteristics of being easily comprehended by students, being suitability to purpose, and usability). Comprehension also includes being closely related to natural language and there being easily identifiable examples of the concept defined.</p> <p>Torkildsen et al. ([<reflink idref="bib35" id="ref29">35</reflink>]) further classified mathematical definitions into twelve criteria: Hierarchy, Equivalence, Existence, Non-contradiction, Minimality, Elegance, Axiomatization, Non-ambiguity, Degenerations, Referential clarity, Discrimination and Mathematical essence (see Table 1).</p> <p>Table 1 Categories and criteria from Torkildsen et al. ([<reflink idref="bib35" id="ref30">35</reflink>])</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left"><p>Theme</p></th><th align="left"><p>Category</p></th><th align="left"><p>Criterion</p></th></tr></thead><tbody><tr><td align="left" rowspan="8"><p>Requirements</p></td><td align="left" rowspan="2"><p>Formal requirements</p></td><td align="left"><p>Non-contradiction</p></td></tr><tr><td align="left"><p>Non-ambiguity</p></td></tr><tr><td align="left" rowspan="3"><p>Exemplification</p></td><td align="left"><p>Existence</p></td></tr><tr><td align="left"><p>Discrimination</p></td></tr><tr><td align="left"><p>Degenerations</p></td></tr><tr><td align="left" rowspan="3"><p>Axiomatization</p></td><td align="left"><p>Axiomatization</p></td></tr><tr><td align="left"><p>Hierarchy</p></td></tr><tr><td align="left"><p>Equivalence</p></td></tr><tr><td align="left" rowspan="4"><p>Preferred features</p></td><td align="left"><p>Minimality</p></td><td align="left"><p>Minimality</p></td></tr><tr><td align="left"><p>Aesthetic</p></td><td align="left"><p>Elegance</p></td></tr><tr><td align="left" rowspan="2"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td></tr><tr><td align="left"><p>Mathematical essence</p></td></tr></tbody></table> </ephtml> </p> <p>Rupnow and Randazzo ([<reflink idref="bib33" id="ref31">33</reflink>]) conducted interviews to explore mathematical values upheld through definitions by algebraists and category theorists. Preliminary to this they reviewed the mathematics education literature and identified criteria that are considered important for mathematical definitions. The criteria they identified are divided into criteria related to logical necessity, practical criteria and preference-based criteria. Within these three groups they identified a set of criteria that largely overlaps with that of Torkildsen et al. ([<reflink idref="bib35" id="ref32">35</reflink>]): existence of the concept, Hierarchy, placement of the definition in a general axiomatic system, specification of necessary and sufficient conditions, being well-defined, being internally consistent, equivalence of different definitions, lack of dependence on the representatives used, addressing the purpose for which definitions were made, being stated in a usable way for their purpose, minimality of the set of conditions, degenerations, elegance, and easy comprehension by students (p. 299). The fact that Rupnow and Randazzo ([<reflink idref="bib33" id="ref33">33</reflink>]) arrived at similar criteria to Torkildsen et al. ([<reflink idref="bib35" id="ref34">35</reflink>]) suggests that this set of criteria is an adequate representation of criteria used in the field.</p> <p>As there is no universally accepted definition for mathematical definitions, they can be thought of as a cluster category combining many different characteristics. To compare how mathematics teacher educators and mathematicians define such a cluster category, the technique of comparative judgement is useful.</p> <hd id="AN0189055834-5">Comparative Judgement</hd> <p>Comparative judgement is useful in situations where it is difficult to establish agreement on criteria, which is the case with the cluster category "mathematical definition." It does this by asking respondents to make comparisons between pairs of prompts. The comparison is simply to decide which prompt is better. The comparative judgement thus simplifies the evaluation process by reducing it to direct pairwise comparisons rather than requiring judges to spend time assigning detailed scores to each category. Moreover, the judges compare the prompts directly, which reduces the bias associated with scoring. Due to these reasons, CJ methodology is useful for our purposes.</p> <p>The judged prompts can then be ranked using the Bradley-Terry model (Bradley & Terry, [<reflink idref="bib8" id="ref35">8</reflink>]) for paired comparisons (For details, see Analysis below). Davies et al. ([<reflink idref="bib12" id="ref36">12</reflink>]) used comparative judgement in a study on the meaning of "proof". They had research mathematicians compare written descriptions of what is meant by "proof". Some prompts were written by other research mathematicians and some were written by undergraduate mathematics students. They showed that comparative judgement is a useful technique for quantifying beliefs about the meanings of mathematical concepts. They specifically showed that it is useful for a concept like "proof" that is considered a cluster category by Czocher and Weber ([<reflink idref="bib10" id="ref37">10</reflink>], p. 50). Our work differs, however, from the work of Davies et al. ([<reflink idref="bib12" id="ref38">12</reflink>]) as we begin from characteristics of mathematical definition drawn from the research literature rather than from research mathematicians and undergraduate students.</p> <p>Comparative judgement has also been used in other areas of mathematics education, for example, to assess problem solving (Jones & Inglis, [<reflink idref="bib20" id="ref39">20</reflink>]), conceptual understanding (Bisson et al., [<reflink idref="bib5" id="ref40">5</reflink>], [<reflink idref="bib6" id="ref41">6</reflink>]; Jones et al., [<reflink idref="bib19" id="ref42">19</reflink>]) and general reasoning in primary school (Hunter & Jones, [<reflink idref="bib17" id="ref43">17</reflink>]).</p> <hd id="AN0189055834-6">Methodology</hd> <p></p> <hd id="AN0189055834-7">Materials</hd> <p>We build directly on the systematic literature review of Torkildsen et al. ([<reflink idref="bib35" id="ref44">35</reflink>]). The categories identified in that study were used to find and refine the statements from the literature on criteria for mathematical definitions that were used in this comparative judgement study. The six categories under the themes Requirements and Preferred features were used. Statements found in the literature were sometimes phrased in ways that suggested whether they were requirements, by beginning "A definition must ..." or "A definition should ...". Such statements were rephrased, removing the words "must" and "should", so that they did not suggest a requirement. This was done so that a participant who agreed that a feature was important, but did not see it as a requirement, would still accept the statement. For example, a statement about minimality of mathematical definitions "A definition should not contain properties which can be mathematically inferred from other parts of the definition" was rephrased as "That they do not contain properties which can be mathematically inferred from other parts of the definition, i.e. they are minimal". Within each category, some statements had the same meaning and, in such cases, only one of the statements, or a composite of them, was used. In some cases, a more precise and a less precise version of the statement was prepared.</p> <p>We consulted three experienced mathematics teacher educators, with degrees in both mathematics and mathematics education, to determine which of the equivalent statements were more understandable and if the statements were interpreted as expected. These experts did not participate in the study. Redundant statements were removed. Statements that were not understandable to our colleagues were refined and clarified. The final set of 32 statements is listed in Table 2, along with a Key used in coding and the statement's category and criterion from Torkildsen et al. ([<reflink idref="bib35" id="ref45">35</reflink>]). The PRECISE statement "That it is precise" is difficult to place in Torkildsen et al.'s ([<reflink idref="bib35" id="ref46">35</reflink>]) scheme as being precise is mentioned as a characteristic of definition under both the Aesthetic and Comprehension categories.</p> <p>Table 2 Overview of statements and corresponding keys from Torkildsen et al. ([<reflink idref="bib35" id="ref47">35</reflink>])</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left"><p>Key</p></th><th align="left"><p>Category</p></th><th align="left"><p>Criterion</p></th><th align="left"><p>Statement</p></th></tr></thead><tbody><tr><td align="left"><p>COEXIST</p></td><td align="left"><p>Formal</p></td><td align="left"><p>Non-contradiction</p></td><td align="left"><p>That all the properties stated in the definition can coexist</p></td></tr><tr><td align="left"><p>CONSIST1</p></td><td align="left"><p>Formal</p></td><td align="left"><p>Non-contradiction</p></td><td align="left"><p>That it is consistent and non-contradicting</p></td></tr><tr><td align="left"><p>CONSIST2</p></td><td align="left"><p>Formal</p></td><td align="left"><p>Non-contradiction</p></td><td align="left"><p>That it is consistent with the mathematical theory formed thus far</p></td></tr><tr><td align="left"><p>WELLDEF1</p></td><td align="left"><p>Formal</p></td><td align="left"><p>Non-ambiguity</p></td><td align="left"><p>That it has a unique interpretation, in other words it is well-defined and unambiguous</p></td></tr><tr><td align="left"><p>WELLDEF2</p></td><td align="left"><p>Formal</p></td><td align="left"><p>Non-ambiguity</p></td><td align="left"><p>That it is well-defined, that is to say, the meaning is unambiguous</p></td></tr><tr><td align="left"><p>DISCRIM1</p></td><td align="left"><p>Exemplification</p></td><td align="left"><p>Discrimination</p></td><td align="left"><p>That it allows to discriminate between instances and non-instances</p></td></tr><tr><td align="left"><p>DISCRIM2</p></td><td align="left"><p>Exemplification</p></td><td align="left"><p>Discrimination</p></td><td align="left"><p>That it allows instances and non-instances of the concept to be discriminated with certainty, consistency, and efficiency</p></td></tr><tr><td align="left"><p>EXIST1</p></td><td align="left"><p>Exemplification</p></td><td align="left"><p>Existence</p></td><td align="left"><p>That it can be proven that at least one instance of the defined concept exists</p></td></tr><tr><td align="left"><p>EXIST2</p></td><td align="left"><p>Exemplification</p></td><td align="left"><p>Existence</p></td><td align="left"><p>That at least one example of the defined concept exists</p></td></tr><tr><td align="left"><p>DEDUCTIV</p></td><td align="left"><p>Axiomatisation</p></td><td align="left"><p>Axiomatisation</p></td><td align="left"><p>That it fits into and is part of a deductive system</p></td></tr><tr><td align="left"><p>EQUIV1</p></td><td align="left"><p>Axiomatisation</p></td><td align="left"><p>Equivalence</p></td><td align="left"><p>That equivalence can be proven if more than one definition is given for the same concept</p></td></tr><tr><td align="left"><p>EQUIV2</p></td><td align="left"><p>Axiomatisation</p></td><td align="left"><p>Equivalence</p></td><td align="left"><p>That if there are multiple definitions for a given concept, they must be mathematically equivalent</p></td></tr><tr><td align="left"><p>GENERAL</p></td><td align="left"><p>Axiomatisation</p></td><td align="left"><p>Hierarchy</p></td><td align="left"><p>That it describes any new concept as a special case of a more general concept</p></td></tr><tr><td align="left"><p>HIERARC1</p></td><td align="left"><p>Axiomatisation</p></td><td align="left"><p>Hierarchy</p></td><td align="left"><p>That it only employs previously defined concepts</p></td></tr><tr><td align="left"><p>HIERARC2</p></td><td align="left"><p>Axiomatisation</p></td><td align="left"><p>Hierarchy</p></td><td align="left"><p>That it is part of a hierarchical system in the sense that it employs only terms priorly defined</p></td></tr><tr><td align="left"><p>HIERARC3</p></td><td align="left"><p>Axiomatisation</p></td><td align="left"><p>Hierarchy</p></td><td align="left"><p>That it is hierarchical, in the sense that the terms used in the definition is known to the target group</p></td></tr><tr><td align="left"><p>MINI1</p></td><td align="left"><p>Minimality</p></td><td align="left"><p>Minimality</p></td><td align="left"><p>That it is minimal. Minimality means that no condition in a definition can be inferred from the other conditions; that is, there is no redundancy</p></td></tr><tr><td align="left"><p>MINI2</p></td><td align="left"><p>Minimality</p></td><td align="left"><p>Minimality</p></td><td align="left"><p>That it only mentions necessary terms and properties so that it is possible to distinguish an instance from a non-instance</p></td></tr><tr><td align="left"><p>MINI3</p></td><td align="left"><p>Minimality</p></td><td align="left"><p>Minimality</p></td><td align="left"><p>That it does not contain properties which can be mathematically inferred from other parts of the definition</p></td></tr><tr><td align="left"><p>ELEGANT1</p></td><td align="left"><p>Aesthetic</p></td><td align="left"><p>Elegance</p></td><td align="left"><p>That it is elegant</p></td></tr><tr><td align="left"><p>ELEGANT2</p></td><td align="left"><p>Aesthetic</p></td><td align="left"><p>Elegance</p></td><td align="left"><p>That it does not contain superfluous words or symbols, and that it 'looks nice'</p></td></tr><tr><td align="left"><p>CAPTURES</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Mathematical essence</p></td><td align="left"><p>That it captures and synthesizes the mathematical essence of the concept</p></td></tr><tr><td align="left"><p>CLEAR</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td><td align="left"><p>That it is clear</p></td></tr><tr><td align="left"><p>EXAMPLES</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td><td align="left"><p>That it has easily identifiable examples</p></td></tr><tr><td align="left"><p>INTUITIV</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td><td align="left"><p>That it is intuitive</p></td></tr><tr><td align="left"><p>MATCH</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td><td align="left"><p>That it matches the target group's knowledge and needs</p></td></tr><tr><td align="left"><p>NATURAL</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td><td align="left"><p>That the name of the concept must be closely related to its natural-language usage</p></td></tr><tr><td align="left"><p>PRECISE</p></td><td align="left"><p>Comprehension*</p></td><td align="left"><p>Referential clarity*</p></td><td align="left"><p>That it is precise</p></td></tr><tr><td align="left"><p>PREVCONC</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td><td align="left"><p>That it only employs previously defined concepts known to the target group</p></td></tr><tr><td align="left"><p>SUITABLE</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td><td align="left"><p>That it is didactically suitable to the target group</p></td></tr><tr><td align="left"><p>UNDTAR</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td><td align="left"><p>That it is understandable to the target group</p></td></tr><tr><td align="left"><p>USEFUL</p></td><td align="left"><p>Comprehension</p></td><td align="left"><p>Referential clarity</p></td><td align="left"><p>That it is useful, for example, it is useful for proving theorems</p></td></tr></tbody></table> </ephtml> </p> <p>* The Key PRECISE could also be classified in the criterion Elegance under the Aesthetic category</p> <hd id="AN0189055834-8">Participants</hd> <p>The participants were selected according to the following criteria:</p> <p>That they</p> <p></p> <ulist> <item> were affiliated with a university or city college in Norway.</item> <p></p> <item> were affiliated with an appropriate department, that is to say, department for mathematical science or similar for research mathematicians and department of teacher education or similar for mathematics teachers educators.</item> </ulist> <p>The list of candidates matching the selection criteria was compiled by searching through the appropriate departmental webpages and candidates were invited to participate in the study by personal email. As the main criterion for selection was affiliation with an appropriate department, no distinction was made, for example between pure and applied mathematicians. A total of 191 RMs and 116 MTEs were asked to participate in the study. 57 research mathematicians (RM) and 62 mathematics teacher educators in teacher education (MTE) agreed to participate. Norwegian MTEs have a range of backgrounds. Some become teacher educators directly after completing their own teacher education, with or without completing a PhD. Others come to the university after teaching in schools, and their university level studies of mathematics may be limited or partly forgotten. Still others are research mathematicians who become interested in working with future teachers. This means that the variability in mathematics background of MTEs is wider than for RMs, but also that there may be some overlap between the two groups, as a RM may have become a MTE.</p> <hd id="AN0189055834-9">Procedure</hd> <p>Each participant responded to a comparative judgement survey in No More Marking (No More Marking, [<reflink idref="bib27" id="ref48">27</reflink>]). The only differences between the RM and MTE conditions were the instruction and the number of comparisons (due to smaller sample size, the RMs made on average fewer comparisons than the MTEs did). Each RM was asked to judge 40 pairs of statements, one pair at a time, with the following question: "As a researcher in mathematics, where your target group is other mathematicians, what is more important about mathematical definitions?" Each MTE was asked to judge 41 pairs of statements with the following question: "For a mathematical definition in the context of teaching and learning, what is more important?" For each judging session the pairs of statements to be compared were randomized by No More Marking. The randomization algorithm was designed to equalize the frequency of each statement in the total number of comparisons and also to ensure judges see as many different definitions as possible. Each participant was given an ID (WM followed by a number for RMs, and ME followed by a number for MTEs) and no personal data were stored.</p> <p>There was no possibility for ranking statements as equally important, meaning that each judge had to rank one statement as more important than the other, even in cases where they might have no preference. Jones et al. ([<reflink idref="bib21" id="ref49">21</reflink>]) found that even when individual judges stated they were guessing without having any preferences, the data showed that they were making consistent judgement even when they were not aware of it. Not all participants completed all comparisons (41 for the MTEs and 40 for the RMs). Partial sessions were included in the data. Among the RMs, 40 of the 57 completed all the comparisons, 10 completed 5 or less comparisons. Among the MTEs, 51 of the 62 completed more than 35 comparisons, and 10 of them completed less than 7 comparisons. The comparative judgement survey resulted in <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>N</mi><mo>=</mo><mn>1780</mn></mrow></math> </ephtml> comparisons for the RMs and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>N</mi><mo>=</mo><mn>1827</mn></mrow></math> </ephtml> comparisons for the MTEs. The data are available in Forbregd et al. ([<reflink idref="bib14" id="ref50">14</reflink>]).</p> <hd id="AN0189055834-10">Analysis</hd> <p>We used the Bradley-Terry model (Bradley & Terry, [<reflink idref="bib8" id="ref51">8</reflink>]) for paired comparisons in this study. The purpose of the Bradley-Terry model is to create rank orders from the pairwise comparison of data. In previous studies of comparative judgement (Bisson et al., [<reflink idref="bib5" id="ref52">5</reflink>]; Davies et al., [<reflink idref="bib12" id="ref53">12</reflink>]), the researchers have used the Bradley-Terry model to create rank order parameter estimates. In the Bradley-Terry model, the outcome is interpreted as a binary (win/lose). The model can be written as a Rasch measurement model and expresses the log odds that a statement i with measure <emph>θ</emph><subs>i</subs> beats a statement j with measure <emph>θ</emph><subs>j</subs>:</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>l</mi><mi>o</mi><mi>g</mi><mfenced close=")" open="("><mfrac><mfenced close=")" open="("><mi>P</mi><mi>i</mi><mo>></mo><mi>j</mi></mfenced><mfenced close=")" open="("><mi>P</mi><mi>j</mi><mo>></mo><mi>i</mi></mfenced></mfrac></mfenced><mo>=</mo><msub><mi>θ</mi><mi>i</mi></msub><mo>-</mo><msub><mi>θ</mi><mi>j</mi></msub></mrow></math> </ephtml> </p> <p>Graph</p> <p>In essence, when <emph>θ</emph><subs>i</subs> increases relative to <emph>θ</emph><subs>j</subs>, the likelihood that i beats j increases; when <emph>θ</emph><subs>i</subs> decreases relative to <emph>θ</emph><subs>j</subs>, the likelihood that i beats j decreases; and when <emph>θ</emph><subs>i</subs> equals <emph>θ</emph><subs>j</subs>, the likelihood that i beats j is 50%. The Bradley-Terry does allow for ties; however, there may be reasons to disallow ties in many cases. Accordingly, a common assumption is that when individuals compare statements they consider equally important, they select purely at random. Based the data, the parameters (i.e., the measures of how the respondents valued the relative importance of the features of definitions) were estimated using a Maximum-likelihood estimation algorithm (Hunter, [<reflink idref="bib16" id="ref54">16</reflink>]), which is implemented in the Sirt-package (Robitzsch, [<reflink idref="bib32" id="ref55">32</reflink>]) in R (R Core Team, [<reflink idref="bib31" id="ref56">31</reflink>]).</p> <p>Reliability of the parameter estimates was checked in three steps. First, the internal consistency was measured employing Scale Separation Reliability (SSR), as was done by Davies et al. ([<reflink idref="bib12" id="ref57">12</reflink>]). SSR is derived from Rasch modelling (Jones & Alcock, [<reflink idref="bib18" id="ref58">18</reflink>]) and is somewhat analogous to Cronbach's alpha (Pollitt, [<reflink idref="bib30" id="ref59">30</reflink>]). However, there are some issues concerning the validity of this measure; it tends to overestimate and may be inflated (Bramley & Vitello, [<reflink idref="bib9" id="ref60">9</reflink>]; Davies, [<reflink idref="bib11" id="ref61">11</reflink>]; Pollitt, [<reflink idref="bib30" id="ref62">30</reflink>]). Second, the inter-rater reliability was measured by employing a split-halves technique in which the set of judges was randomly split into two groups and parameter estimating on each group. The Pearson's correlation coefficient was calculated on the two sets of parameter estimates. This procedure was run 1000 times, and the median of correlation coefficients was used to measure judges' stability (Bisson et al., [<reflink idref="bib5" id="ref63">5</reflink>]; Verhavert et al., [<reflink idref="bib37" id="ref64">37</reflink>]). Third, the infit mean square, or "misfit" figures, of judges and statements were scrutinized. The misfit figures are, respectively, a measure of the consistency of each judge's performance compared to all the other judges and how consistently each statement is valued by all judges (Bond & Fox, [<reflink idref="bib7" id="ref65">7</reflink>]). In the comparative judgement literature, the convention is to treat a judge as misfitting and a statement as inconsistently ranked if their infit mean square value is greater than two standard deviations above the mean (Davies et al., [<reflink idref="bib12" id="ref66">12</reflink>]; Jones & Alcock, [<reflink idref="bib18" id="ref67">18</reflink>]; Pollitt, [<reflink idref="bib30" id="ref68">30</reflink>]). The infit mean square values were calculated to flag problematic statements, flag problematic judges, and assess whether there were significant disagreements between the judges within each context (See Table 4 in Appendix for the values of infit mean square).</p> <p>Five judges from RMs and eight judges from MTEs had infit mean square values above the threshold the simulation yielded, A new model was fitted with data where the misfitting judges were removed, and Pearson's product-moment correlation was calculated between the estimates of the new and the original model. The correlation coefficient was 1; thus, the misfit had no practical consequence on the estimates or the ranking of the statements. From the group of RMs, the statements EQUIV2 (That if there are multiple definitions for a given concept, they must be mathematically equivalent), MINI1 (That it is minimal. Minimality means that no condition in a definition can be inferred from the other conditions; that is, there is no redundancy), SUITABLE (That it is didactically suitable to the target group) and USEFUL had infit figures that exceeded the simulated threshold of 1.1345 (see Table 4 in Appendix). From MTEs, the infit mean square of the statements CONSISTS2 (That it is consistent with the mathematical theory formed thus far) and EXISTS2 (That at least one example of the defined concept exists) exceeded the threshold simulated. That the misfit figure exceeded the misfit-threshold is most likely due to factors other than random noise. The most likely explanation is that the judges did not sufficiently agree on the relative importance of the statement or that the judges disagreed on the interpretation of the statement. For instance, for SUITABLE, it may have been difficult to understand how to interpret it in the context of doing mathematics. The phrasing of the statement may have been dubious or confusing for some of the judges.</p> <hd id="AN0189055834-11">Results</hd> <p>The parameter estimates for statements by the two groups can be seen in Table 3.</p> <p>Table 3 Ratings for statements for RMs and MTEs, sorted by RMs' ratings</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left"><p>Key</p></th><th align="left"><p>Statement</p></th><th align="left"><p>RMs</p></th><th align="left"><p>MTEs</p></th></tr></thead><tbody><tr><td align="left"><p>CONSIST1</p></td><td align="left"><p>That it is consistent and non-contradicting</p></td><td char="." align="char"><p>2.051</p></td><td char="." align="char"><p>1.170</p></td></tr><tr><td align="left"><p>WELLDEF1</p></td><td align="left"><p>That it has a unique interpretation, in other words it is well-defined and unambiguous</p></td><td char="." align="char"><p>1.962</p></td><td char="." align="char"><p>0.846</p></td></tr><tr><td align="left"><p>WELLDEF2</p></td><td align="left"><p>That it is well-defined, that is to say, the meaning is unambiguous</p></td><td char="." align="char"><p>1.870</p></td><td char="." align="char"><p>0.819</p></td></tr><tr><td align="left"><p>PRECISE</p></td><td align="left"><p>That it is precise</p></td><td char="." align="char"><p>1.225</p></td><td char="." align="char"><p> − 0.268</p></td></tr><tr><td align="left"><p>CAPTURES</p></td><td align="left"><p>That it captures and synthesizes the mathematical essence of the concept</p></td><td char="." align="char"><p>0.962</p></td><td char="." align="char"><p>0.774</p></td></tr><tr><td align="left"><p>EQUIV2</p></td><td align="left"><p>That if there are multiple definitions for a given concept, they must be mathematically equivalent</p></td><td char="." align="char"><p>0.925</p></td><td char="." align="char"><p>0.617</p></td></tr><tr><td align="left"><p>CONSIST2</p></td><td align="left"><p>That it is consistent with the mathematical theory formed thus far</p></td><td char="." align="char"><p>0.891</p></td><td char="." align="char"><p>0.707</p></td></tr><tr><td align="left"><p>DISCRIM1</p></td><td align="left"><p>That it allows to discriminate between instances and non-instances</p></td><td char="." align="char"><p>0.510</p></td><td char="." align="char"><p>0.803</p></td></tr><tr><td align="left"><p>EXIST2</p></td><td align="left"><p>That at least one example of the defined concept exists</p></td><td char="." align="char"><p>0.489</p></td><td char="." align="char"><p> − 0.202</p></td></tr><tr><td align="left"><p>COEXIST</p></td><td align="left"><p>That all the properties stated in the definition can coexist</p></td><td char="." align="char"><p>0.432</p></td><td char="." align="char"><p>0.221</p></td></tr><tr><td align="left"><p>USEFUL</p></td><td align="left"><p>That it is useful, for example, it is useful for proving theorems</p></td><td char="." align="char"><p>0.431</p></td><td char="." align="char"><p>0.080</p></td></tr><tr><td align="left"><p>HIERARC2</p></td><td align="left"><p>That it is part of a hierarchical system in the sense that it employs only terms priorly defined</p></td><td char="." align="char"><p>0.417</p></td><td char="." align="char"><p> − 0.063</p></td></tr><tr><td align="left"><p>MINI2</p></td><td align="left"><p>That it only mentions necessary terms and properties so that it is possible to distinguish an instance from a non-instance</p></td><td char="." align="char"><p>0.223</p></td><td char="." align="char"><p>0.293</p></td></tr><tr><td align="left"><p>DISCRIM2</p></td><td align="left"><p>That it allows instances and non-instances of the concept to be discriminated with certainty, consistency, and efficiency</p></td><td char="." align="char"><p>0.206</p></td><td char="." align="char"><p>0.794</p></td></tr><tr><td align="left"><p>EQUIV1</p></td><td align="left"><p>That equivalence can be proven if more than one definition is given for the same concept</p></td><td char="." align="char"><p>0.195</p></td><td char="." align="char"><p> − 0.156</p></td></tr><tr><td align="left"><p>HIERARC1</p></td><td align="left"><p>That it only employs previously defined concepts</p></td><td char="." align="char"><p>0.189</p></td><td char="." align="char"><p>0.399</p></td></tr><tr><td align="left"><p>UNDTAR</p></td><td align="left"><p>That it is understandable to the target group</p></td><td char="." align="char"><p>0.084</p></td><td char="." align="char"><p>0.907</p></td></tr><tr><td align="left"><p>CLEAR</p></td><td align="left"><p>That it is clear</p></td><td char="." align="char"><p> − 0.013</p></td><td char="." align="char"><p>0.018</p></td></tr><tr><td align="left"><p>HIERARC3</p></td><td align="left"><p>That it is hierarchical, in the sense that the terms used in the definition is known to the target group</p></td><td char="." align="char"><p> − 0.169</p></td><td char="." align="char"><p>0.114</p></td></tr><tr><td align="left"><p>EXIST1</p></td><td align="left"><p>That it can be proven that at least one instance of the defined concept exists</p></td><td char="." align="char"><p> − 0.266</p></td><td char="." align="char"><p> − 0.606</p></td></tr><tr><td align="left"><p>DEDUCTIV</p></td><td align="left"><p>That it fits into and is part of a deductive system</p></td><td char="." align="char"><p> − 0.333</p></td><td char="." align="char"><p> − 0.464</p></td></tr><tr><td align="left"><p>PREVCONC</p></td><td align="left"><p>That it only employs previously defined concepts known to the target group</p></td><td char="." align="char"><p> − 0.397</p></td><td char="." align="char"><p>0.505</p></td></tr><tr><td align="left"><p>MATCH</p></td><td align="left"><p>That it matches the target group's knowledge and needs</p></td><td char="." align="char"><p> − 0.409</p></td><td char="." align="char"><p>0.966</p></td></tr><tr><td align="left"><p>MINI1</p></td><td align="left"><p>That it is minimal, Minimality means that no condition in a definition can be inferred from the other conditions; that is, there is no redundancy</p></td><td char="." align="char"><p> − 0.681</p></td><td char="." align="char"><p> − 1.491</p></td></tr><tr><td align="left"><p>MINI3</p></td><td align="left"><p>That it does not contain properties which can be mathematically inferred from other parts of the definition</p></td><td char="." align="char"><p> − 0.684</p></td><td char="." align="char"><p> − 0.997</p></td></tr><tr><td align="left"><p>ELEGANT2</p></td><td align="left"><p>That it does not contain superfluous words or symbols, and that it 'looks nice'</p></td><td char="." align="char"><p> − 1.160</p></td><td char="." align="char"><p> − 1.447</p></td></tr><tr><td align="left"><p>INTUITIV</p></td><td align="left"><p>That it is intuitive</p></td><td char="." align="char"><p> − 1.169</p></td><td char="." align="char"><p> − 0.861</p></td></tr><tr><td align="left"><p>EXAMPLES</p></td><td align="left"><p>That it has easily identifiable examples</p></td><td char="." align="char"><p> − 1.174</p></td><td char="." align="char"><p>0.273</p></td></tr><tr><td align="left"><p>ELEGANT1</p></td><td align="left"><p>That it is elegant</p></td><td char="." align="char"><p> − 1.201</p></td><td char="." align="char"><p> − 2.082</p></td></tr><tr><td align="left"><p>SUITABLE</p></td><td align="left"><p>That it is didactically suitable to the target group</p></td><td char="." align="char"><p> − 1.340</p></td><td char="." align="char"><p>1.009</p></td></tr><tr><td align="left"><p>NATURAL</p></td><td align="left"><p>That the name of the concept must be closely related to its natural-language usage</p></td><td char="." align="char"><p> − 1.873</p></td><td char="." align="char"><p> − 1.465</p></td></tr><tr><td align="left"><p>GENERAL</p></td><td align="left"><p>That it describes any new concept as a special case of a more general concept</p></td><td char="." align="char"><p> − 2.193</p></td><td char="." align="char"><p> − 1.214</p></td></tr></tbody></table> </ephtml> </p> <p>The group of RMs conducted a total of 1780 comparisons. The comparison data was fitted to a Bradley-Terry-model, and the results are shown in Fig. 2 (left side). The Scale Separation Reliability was calculated to be <emph>SSR</emph><subs><emph>RM</emph></subs> = 0.95, which is high. The inter-rater consistency was calculated by means of Pearson product-moment correlation coefficient (<emph>Median</emph> = 0.9), which is acceptable and within the bounds of the simulated inter-rater reliability (see Fig. 5 in appendix). This shows high stability in rating under variation of judges.</p> <p>Graph: Fig. 2 Plot of estimated parameters for statements for Research Mathematicians (left) and Mathematics teacher educators (right)</p> <p>The group of MTEs conducted 1827 comparisons in total. Figure 2 (right side) shows the fitted Bradely-Terry-model for the comparison data for the mathematics educators. The Scale Separation Reliability for the mathematics educators was calculated to be <emph>SSR</emph><subs><emph>MTE</emph></subs> = 0.95. The inter-rater reliability was calculated (<emph>Median</emph> = 0.89), which was slightly lower than for the research mathematicians (see Fig. 5 in appendix).</p> <hd id="AN0189055834-12">Discussion</hd> <p>We now consider our three research questions:</p> <p></p> <ulist> <item> How are characteristics of definitions identified in the mathematics education research literature ranked in importance by research mathematicians?</item> <p></p> <item> How are characteristics of definitions identified in the mathematics education research literature ranked in importance by mathematics teacher educators?</item> <p></p> <item> How do the rankings of these two groups differ from the existing mathematics education literature?</item> </ulist> <hd id="AN0189055834-13">Ranking of Characteristics by Research Mathematicians</hd> <p>The statements ranked most highly by research mathematicians were those for the keys CONSIST1, WELLDEF1, and WELLDEF2. These three statements fall under the criteria of Non-ambiguity and Non-contradiction in Torkildsen et al.'s ([<reflink idref="bib35" id="ref69">35</reflink>]) Formal requirements category (see Fig. 3). The other statements in that category, CONSIST2 and COEXIST, are also rated highly by research mathematicians. This suggests that this category includes the criteria that are most important to mathematicians for mathematical definitions, which is sensible, if they are, as Torkildsen et al. ([<reflink idref="bib35" id="ref70">35</reflink>]) claim, required. Recall that Harel et al. ([<reflink idref="bib15" id="ref71">15</reflink>]) included the characteristics of Consistency, unambiguity and being well-defined as both required and agreed upon in the literature, but these characteristics were not listed by Van Dormolen and Zaslavsky ([<reflink idref="bib36" id="ref72">36</reflink>]). Furthermore, Bergman et al. ([<reflink idref="bib4" id="ref73">4</reflink>]) point out that in practice, "sometimes, different definitions for the same term exist, but do not define the same class of objects, introducing ambiguity into mathematical tasks" (p. 94), so these characteristics may be required in theory but not in practice.</p> <p>Graph: Fig. 3 Ranking of required characteristics by research mathematicians</p> <p>The statements in the other two categories considered to be Requirements by Torkildsen et al. ([<reflink idref="bib35" id="ref74">35</reflink>]) were not ranked so highly. In the category Exemplification the statements DISCRIM1 and EXIST2 were ranked eighth and ninth (about the same as CONSIST2 and COEXIST), but DISCRIM2 was ranked 14th out of the 32 statements and EXIST1 was ranked 20th. It is interesting to compare the two EXIST statements: EXIST1: "That it can be proven that at least one instance of the defined concept exists"; and EXIST2: "That at least one example of the defined concept exists". It may be that the RMs interpreted EXIST1 to say that an existence proof was required, beyond simply providing an example to show one exists. In other words, the existence of an example is required, but it can be either provided or proved to exist. Another possibility is that RMs do not see a problem with making a definition where the example space is empty. This might in fact be useful, for example, in a proof by contradiction where a set is defined and later shown to be empty. Under either interpretation we can say that the category Exemplification was fairly highly ranked, and so considering it a requirement for a mathematical definition, according to research mathematicians, seems valid. It is also listed as a characteristic by both Harel et al. ([<reflink idref="bib15" id="ref75">15</reflink>]) and Van Dormolen and Zaslavsky ([<reflink idref="bib36" id="ref76">36</reflink>]).</p> <p>The statements in Torkildsen et al.'s ([<reflink idref="bib35" id="ref77">35</reflink>]) third Requirement category, Axiomatization, were ranked in widely different positions by the research mathematicians. EQUIV2 was ranked sixth overall, while GENERAL was ranked in the lowest position. The others were ranked close to the middle. EQUIV2 is "That if there are multiple definitions for a given concept, they must be mathematically equivalent". EQUIV1 is: "That equivalence can be proven if more than one definition is given for the same concept" and was ranked 15th, close to the middle. As with EXIST1 and EXIST2, the difference here seems to be that proving equivalence was considered less important than the fact of the equivalence. Alternatively, EQUIV2 had an infit figure that exceeded the simulated threshold, so it may be that its high ranking should not be trusted, and the middle range rankings of the other statements in the category Axiomatization considered instead.</p> <p>GENERAL states "That it describes any new concept as a special case of a more general concept". This statement is essentially Van Dormolen and Zaslavsky's ([<reflink idref="bib36" id="ref78">36</reflink>]) criterion of Hierarchy. That it was ranked so low when the other statements in the Hierarchy criterion were ranked in the middle suggests the relevance of this criterion should be investigated in more detail.</p> <p>In summary, the categories Formal requirement and Exemplification are important in characterizing mathematical definitions for research mathematicians, and it seems reasonable to consider these as Requirements, as Torkildsen et al. ([<reflink idref="bib35" id="ref79">35</reflink>]) did. Characteristics of definitions in the category Axiomatization, however, may not be considered required by all mathematicians, although they were included by both Harel et al. ([<reflink idref="bib15" id="ref80">15</reflink>]) and Van Dormolen and Zaslavsky ([<reflink idref="bib36" id="ref81">36</reflink>]).</p> <p>Four of the statements ranked least highly by research mathematicians are those for the keys NATURAL, SUITABLE, EXAMPLES, and INTUITIV. These all come from the Comprehension category and the Referential clarity criterion in Torkildsen et al.'s ([<reflink idref="bib35" id="ref82">35</reflink>]) scheme. Of the other keys in that category, one, PRECISE, was among the most highly rated, while the rest fall in the middle range. This suggests that while characteristics like being intuitive, and closely related to natural language may be desirable for mathematicians, they are not as important as other characteristics. The same can be said of Elegance and other Aesthetic criteria, as the keys ELEGANT1 and ELEGANT2 were also among the lower ranked. Both Harel et al. ([<reflink idref="bib15" id="ref83">15</reflink>]) and Van Dormolen and Zaslavsky ([<reflink idref="bib36" id="ref84">36</reflink>]) noted that these characteristics may be desirable, but they are not required.</p> <p>The statement PRECISE is simply: "That it is precise". The ranking of this statement would have depended strongly on how the word "precise" was interpreted. The RMs ranked this statement near WELLDEF1, which suggests that they interpreted it to mean "has a unique interpretation". A ranking closer to statements like UNDTAR, CLEAR and ELEGANT2 would have suggested it was interpreted to mean "understandable and clear" or "does not contain superfluous words or symbols".</p> <p>Minimality is the criterion that is most often mentioned in the mathematics education research articles Torkildsen et al. ([<reflink idref="bib35" id="ref85">35</reflink>]) include in their overview. However, the three statements related to Minimality were ranked near the middle by the RMs. There is disagreement in the mathematics education literature about whether minimality should be a Requirement or a Preferred feature (see e.g., Zaslavsky & Shir, [<reflink idref="bib40" id="ref86">40</reflink>]), but it seems that for RMs it is not required.</p> <hd id="AN0189055834-14">Ranking of Characteristics by Mathematics Teacher Educators</hd> <p>As with the RMs, statements under the criteria of Non-ambiguity and Non-contradiction in Torkildsen et al.'s ([<reflink idref="bib35" id="ref87">35</reflink>]) Formal requirements category were ranked highly by MTEs. CONSIST1, WELLDEF1, and WELLDEF2 all fall in the top six positions, CONSIST2 is ranked 10th and COEXIST falls in the middle in position 16. This suggests that this category includes the criteria for mathematical definitions that are most important to mathematics teacher educators, although it is not always included in published lists of characteristics, such as Van Dormolen and Zaslavsky's ([<reflink idref="bib36" id="ref88">36</reflink>]).</p> <p>Three other highly ranked statements come from the Comprehension category and the Referential clarity criterion in Torkildsen et al.'s ([<reflink idref="bib35" id="ref89">35</reflink>]) scheme. They are SUITABLE, MATCH, and UNDTAR. Other statements in the same category are ranked much lower (INTUITIV is 26th and NATURAL 30th). SUITABLE, MATCH, and UNDTAR are the only three statements in this category that refer to a "target group", so this may reflect consideration by the MTEs of school contexts where the way of defining a concept might differ according to the age of the pupils. This is an interesting result as these characteristics are considered as Preferred features but not Requirements by Torkildsen et al. ([<reflink idref="bib35" id="ref90">35</reflink>]) and Harel et al. ([<reflink idref="bib15" id="ref91">15</reflink>]) and are not even included among Van Dormolen and Zaslavsky's ([<reflink idref="bib36" id="ref92">36</reflink>]) characteristics. This suggests that in this area the MTEs have different values from those reflected in the mathematics education literature.</p> <p>In the Exemplification category, the statements DISCRIM1 and DISCRIM2 were ranked highly (7th and 8th), but the other statements, under the Existence criterion, were ranked low (22nd and 25th). This suggests that only the ability to distinguish between instances and non-instances is important to MTEs and showing the existence of an instance is not.</p> <p>The lowest ranked statements include those from the Elegance criterion, as well as Minimality. As with the RMs, MINI2 seems to be an exception, but this result is consistent with Harel et al.'s ([<reflink idref="bib15" id="ref93">15</reflink>]) description of these characteristics as not agreed upon in the mathematics education literature.</p> <p>Statements from the Axiomatization category are ranked in the middle or below, with GENERAL being the lowest ranked of them, in position 28. This is a contrast with Torkildsen et al.'s ([<reflink idref="bib35" id="ref94">35</reflink>]) description of Axiomatization as a Requirement, and this characteristic being listed by both Harel et al. ([<reflink idref="bib15" id="ref95">15</reflink>]) and Van Dormolen and Zaslavsky ([<reflink idref="bib36" id="ref96">36</reflink>]).</p> <hd id="AN0189055834-15">Comparison of Rankings</hd> <p>The rankings made by the RMs and MTEs were similar and the correlation coefficient is found to be 0.65. Figure 4 plots the parameter estimates (θ) for each group. The dashed line is the trend-line through the mean of both sets (also known as the line of commonality). The black curves indicate 95% confidence bands (Wright & Stone, [<reflink idref="bib38" id="ref97">38</reflink>]). Points that lie near the line of commonality are statements about which the two groups agreed. Statements above and left of the line were ranked higher by the RMs than by the MTEs. Statements below and right of the line were ranked higher by the MTEs than by the RMs.</p> <p>Graph: Fig. 4 Scatter plot of statements as estimated in the two contexts. The dashed line is the line of commonality. The solid lines approximate a 95% confidence band</p> <p>Although CONSIST1, WELLDEF1, and WELLDEF2 are a little above the line of commonality, they were also ranked highly by the MTEs. SUITABLE, MATCH, and UNDTAR, however, were ranked highly by the MTEs but not by the RMs. As noted above (Section "Ranking of Characteristics by Mathematics Teacher Educators") these statements refer to the "target group" and the MTEs high ranking of them may reflect consideration of school contexts where the way of defining a concept might differ according to the age of the pupils. They are also in the category Comprehension, which overall seems to have been more highly ranked by the MTEs than by the RMs. The statements EXAMPLES and PREVCONC were ranked in the middle by the MTEs but lower by the RMs. PREVCONC also refers to a "target group" and EXAMPLES is in the category Comprehension.</p> <p>PRECISE is the only statement that is ranked much higher by the RMs than by the MTEs, which is surprising given that it is in the Comprehension category. However, as we have discussed above, the RMs may have interpreted it as referring to Non-ambiguity rather than to Referential clarity.</p> <p>In summary, the results show that the two groups largely agree on the ranking of statements about mathematical definitions. Specifically, we can say that these RMs and MTEs agreed on the importance of statements concerning the Formal requirements of Non-contradiction and Non-ambiguity. They also agreed that the criterion of Minimality is not as important as many other criteria. They disagreed on the importance of statements that referred to a "target group" and statements about the Referential clarity of mathematical definitions, which MTEs ranked more highly than RMs. This resonates with the findings of Rupnow and Randazzo ([<reflink idref="bib33" id="ref98">33</reflink>]), who found that the mathematicians they interviewed valued clarity very highly, but, at least in the case of precision, had higher standards for students than they had for themselves. This suggests that some aspects of clarity may be more important in teaching than in research, which is consistent with MTEs ranking statements about Referential clarity more highly than RMs.</p> <p>Torkildsen et al.'s ([<reflink idref="bib35" id="ref99">35</reflink>]) categories and criteria were useful for identifying patterns in the statements, however, there were differences. These suggest that the criteria identified by mathematics education researchers in the literature (based on personal judgments, not empirical evidence) are not entirely aligned with either the criteria used by research mathematics nor by mathematics teacher educators.</p> <hd id="AN0189055834-16">Conclusion</hd> <p>Understanding definitions is an important part of understanding mathematics, but it is not always clear what is required and what is desirable for a mathematical definition. Our study shows that some of the categories and criteria identified by Torkildsen et al. ([<reflink idref="bib35" id="ref100">35</reflink>]) through their literature review are considered important, both by research mathematicians and by mathematics teacher educators. The Formal requirements of Non-contradiction and Non-ambiguity were highly ranked by both groups. Other categories and criteria considered to be Requirements by Torkildsen et al. ([<reflink idref="bib35" id="ref101">35</reflink>]) were not so highly ranked, and the criterion of Minimality, which was the characteristic most often mentioned in the articles surveyed, is not seen as very important by either group.</p> <p>There may be reasons why minimality is mentioned so often in the educational literature; for example, working with minimality requires more advanced mathematical thinking (Miller, [<reflink idref="bib26" id="ref102">26</reflink>]). However, in the context of teaching, minimality may become an obstacle for learning, and therefore it should not be emphasized over other characteristics, as has been noted by some researchers (e.g., Linchesvky et al., [<reflink idref="bib24" id="ref103">24</reflink>]; Van Dormolen & Zaslavsky, [<reflink idref="bib36" id="ref104">36</reflink>]; Zaslavsky & Shir, [<reflink idref="bib40" id="ref105">40</reflink>]).</p> <p>School mathematics is different from university mathematics, and mathematics teacher educators must consider didactical factors as well as mathematical factors when characterizing definitions. Therefore, it is not surprising that there are differences between the two groups. Specifically, mathematics teacher educators value the Referential clarity of mathematical definitions more highly than research mathematicians.</p> <p>These results are based on data gathered in only one context, Norway, and it may be that different results would be found elsewhere. We hypothesize that because of the international nature of research mathematics, the ranking derived from the RMs' data is likely to be generalizable, but further research is needed to confirm this. Education is much more variable across national contexts, and an international comparison of MTEs ranking of statements characterizing definitions would be a valuable addition to the field. In Norway, mathematics teacher educators have various backgrounds. Some have been RMs in the past, while others are experienced school teachers whose university level studies of mathematics may be partly forgotten. We have not investigated the influence of these different backgrounds on the MTEs judgments.</p> <p>We are encouraged that, for the most part, the rankings derived from the MTEs' data and from the the RMs's data agree. As Schoenfeld ([<reflink idref="bib34" id="ref106">34</reflink>]) notes, the doing of mathematics and teaching of mathematics need not determine each other. However, the nature of fundamental concepts like definitions should be compatible. We found this to be the case, except for cases where MTEs ranked didactically important characteristics more highly than the RMs did. As Leikin and Zazkis ([<reflink idref="bib23" id="ref107">23</reflink>]) comment:Teachers' personal knowledge of mathematical definitions affects (a) their curricular decisions regarding the way mathematical concepts are taught, and (b) their pedagogical conception of the ways in which students may or may not learn these concepts. (p. 454)</p> <p>Therefore, it is important that the mathematics teacher educators who prepare those teachers use characterizations of mathematical definitions that are both compatible with those of research mathematicians, and that take into consideration the needs of learners.</p> <p>The basic agreement between the RMs and the MTEs is all the more interesting in that in some cases it did not agree with the characteristics considered important in the research literature. We hope our study will provide the basis for a more consistent characterization of mathematical definitions in mathematics education research. It is worth pointing out that employing comparative judgement on the set of definitions assumes they are uni-dimensional. However, this assumption can be tested empirically, a potential direction for future research.</p> <hd id="AN0189055834-17">Author Contributions</hd> <p>Hermund André Torkildsen: Conceptualization, Investigation, Methodology, Formal analysis, Data Curation, Writing—original draft.</p> <p>Tore A. Forbregd: Conceptualization, Methodology, Formal analysis, Data Curation.</p> <p>David A Reid: Validation, Writing—review & editing, Visualizations.</p> <p>Shaista Kanwal: Validation, Methodology, Writing—review & editing.</p> <hd id="AN0189055834-18">Funding</hd> <p>Open access funding provided by University of Agder.</p> <hd id="AN0189055834-19">Data Availability</hd> <p>The data from comparative judgment results is accessible. The link to the data is added in references.</p> <hd id="AN0189055834-20">Declarations</hd> <p></p> <hd id="AN0189055834-21">Ethical Approval</hd> <p>No personal data was collected or stored about the participants, and participation was completely voluntary. Therefore, no formal ethical approval was required, according to the defintions in the 2021 Guidelines for Research Ethics in the Social Sciences and the Humanities of the Norwegian National Committee for Research Ethics in the Social Sciences and the Humanities.</p> <p>To contact participants, email addresses were collected from official and public websites for the universities and university colleges, i.e. their public work profiles. Emails were sent to all, then deleted. After emails were sent, it was not possible for us to track incoming answers to the survey or link them to anyone that got an email. We could not track persons or their workplaces. All we knew was that incoming answers to one survey was research mathematicians, and to the other survey mathematics educators.</p> <hd id="AN0189055834-22">Competing Interests</hd> <p>There are no competing interests to declare for any of the authors.</p> <hd id="AN0189055834-23">Appendix</hd> <p>Table 4 shows the infit mean square values for both groups. To find critical values for infit mean squares and internal reliability, we conducted 2000 simulations (1000 to simulate RM and 1000 to simulate MTE) in R (R Core Team, [<reflink idref="bib31" id="ref108">31</reflink>]). simulation was conducted as follows: (<reflink idref="bib1" id="ref109">1</reflink>) each statement was given a measure that equaled the measure we obtained in the empirical study (e.g., the measure of the statement "CLEAR" was constrained to be <emph>θ</emph><subs>RM</subs> = -0.013 in RM and <emph>θ</emph><subs>MTE</subs> = 0.018 in MTE, see Table 3); (<reflink idref="bib2" id="ref110">2</reflink>) we assumed no bias in the judges (i.e., we assumed perfect inter-rater reliability); (<reflink idref="bib3" id="ref111">3</reflink>) we used the same set of comparisons as in the empirical study; (<reflink idref="bib4" id="ref112">4</reflink>) for each comparison, we calculated P(i > j), that is, the probability that the first statement would beat the second; (<reflink idref="bib5" id="ref113">5</reflink>) for each comparison, we simulated a random number between 0 and 1. If this number was below P(i > j), we concluded that the first statement beat the second. Conversely, if the simulated number was above P(i > j), we concluded that the second statement beat the first; and (<reflink idref="bib6" id="ref114">6</reflink>) from analysis of this data set (using the Sirt-package), we stored three values: the largest item infit mean square, the largest judge infit mean square, and the inter-rater reliability, which were further used to flag problematic statements, flag problematic judges, and assess whether there were significant disagreements between the judges within each context.</p> <p>Table 4 Measures and misfit figures for statements for RM and MTE (infit figures exceeding the threshold in bold)</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left" colspan="3"><p>Research mathematicians</p></th><th align="left" colspan="3"><p>Mathematics teacher educators</p></th></tr><tr><th align="left"><p>Statement</p></th><th align="left"><p>θ</p></th><th align="left"><p>se. θ</p></th><th align="left"><p>infit</p></th><th align="left"><p>θ</p></th><th align="left"><p>se.θ</p></th><th align="left"><p>infit</p></th></tr></thead><tbody><tr><td align="left"><p>CAPTURES</p></td><td char="." align="char"><p>0.962</p></td><td char="." align="char"><p>0.221</p></td><td char="." align="char"><p>0.909</p></td><td char="." align="char"><p>0.774</p></td><td char="." align="char"><p>0.193</p></td><td char="." align="char"><p>0.936</p></td></tr><tr><td align="left"><p>CLEAR</p></td><td char="." align="char"><p> − 0.013</p></td><td char="." align="char"><p>0.210</p></td><td char="." align="char"><p>1.004</p></td><td char="." align="char"><p>0.018</p></td><td char="." align="char"><p>0.190</p></td><td char="." align="char"><p>0.878</p></td></tr><tr><td align="left"><p>COEXIST</p></td><td char="." align="char"><p>0.432</p></td><td char="." align="char"><p>0.214</p></td><td char="." align="char"><p>0.951</p></td><td char="." align="char"><p>0.221</p></td><td char="." align="char"><p>0.190</p></td><td char="." align="char"><p>1.001</p></td></tr><tr><td align="left"><p>CONSIST1</p></td><td char="." align="char"><p>2.051</p></td><td char="." align="char"><p>0.284</p></td><td char="." align="char"><p>1.039</p></td><td char="." align="char"><p>1.170</p></td><td char="." align="char"><p>0.209</p></td><td char="." align="char"><p>0.987</p></td></tr><tr><td align="left"><p>CONSIST2</p></td><td char="." align="char"><p>0.891</p></td><td char="." align="char"><p>0.220</p></td><td char="." align="char"><p>0.896</p></td><td char="." align="char"><p>0.707</p></td><td char="." align="char"><p>0.195</p></td><td char="." align="char"><p><bold>1.112</bold></p></td></tr><tr><td align="left"><p>DEDUCTIV</p></td><td char="." align="char"><p> − 0.333</p></td><td char="." align="char"><p>0.209</p></td><td char="." align="char"><p>0.997</p></td><td char="." align="char"><p> − 0.464</p></td><td char="." align="char"><p>0.196</p></td><td char="." align="char"><p>1.037</p></td></tr><tr><td align="left"><p>DISCRIM1</p></td><td char="." align="char"><p>0.510</p></td><td char="." align="char"><p>0.218</p></td><td char="." align="char"><p>0.990</p></td><td char="." align="char"><p>0.803</p></td><td char="." align="char"><p>0.197</p></td><td char="." align="char"><p>1.026</p></td></tr><tr><td align="left"><p>DISCRIM2</p></td><td char="." align="char"><p>0.206</p></td><td char="." align="char"><p>0.214</p></td><td char="." align="char"><p>0.980</p></td><td char="." align="char"><p>0.794</p></td><td char="." align="char"><p>0.203</p></td><td char="." align="char"><p>0.975</p></td></tr><tr><td align="left"><p>ELEGANT1</p></td><td char="." align="char"><p> − 1.201</p></td><td char="." align="char"><p>0.237</p></td><td char="." align="char"><p>1.063</p></td><td char="." align="char"><p> − 2.082</p></td><td char="." align="char"><p>0.277</p></td><td char="." align="char"><p>0.955</p></td></tr><tr><td align="left"><p>ELEGANT2</p></td><td char="." align="char"><p> − 1.160</p></td><td char="." align="char"><p>0.238</p></td><td char="." align="char"><p>0.881</p></td><td char="." align="char"><p> − 1.447</p></td><td char="." align="char"><p>0.222</p></td><td char="." align="char"><p>0.996</p></td></tr><tr><td align="left"><p>EQUIV1</p></td><td char="." align="char"><p>0.195</p></td><td char="." align="char"><p>0.213</p></td><td char="." align="char"><p>0.908</p></td><td char="." align="char"><p> − 0.156</p></td><td char="." align="char"><p>0.191</p></td><td char="." align="char"><p>0.955</p></td></tr><tr><td align="left"><p>EQUIV2</p></td><td char="." align="char"><p>0.925</p></td><td char="." align="char"><p>0.227</p></td><td char="." align="char"><p><bold>1.184</bold></p></td><td char="." align="char"><p>0.617</p></td><td char="." align="char"><p>0.188</p></td><td char="." align="char"><p>0.960</p></td></tr><tr><td align="left"><p>EXAMPLES</p></td><td char="." align="char"><p> − 1.174</p></td><td char="." align="char"><p>0.237</p></td><td char="." align="char"><p>0.981</p></td><td char="." align="char"><p>0.273</p></td><td char="." align="char"><p>0.190</p></td><td char="." align="char"><p>0.976</p></td></tr><tr><td align="left"><p>EXIST1</p></td><td char="." align="char"><p> − 0.266</p></td><td char="." align="char"><p>0.210</p></td><td char="." align="char"><p>1.097</p></td><td char="." align="char"><p> − 0.606</p></td><td char="." align="char"><p>0.198</p></td><td char="." align="char"><p>0.958</p></td></tr><tr><td align="left"><p>EXIST2</p></td><td char="." align="char"><p>0.489</p></td><td char="." align="char"><p>0.216</p></td><td char="." align="char"><p>1.022</p></td><td char="." align="char"><p> − 0.202</p></td><td char="." align="char"><p>0.167</p></td><td char="." align="char"><p><bold>1.123</bold></p></td></tr><tr><td align="left"><p>GENERAL</p></td><td char="." align="char"><p> − 2.193</p></td><td char="." align="char"><p>0.302</p></td><td char="." align="char"><p>1.023</p></td><td char="." align="char"><p> − 1.214</p></td><td char="." align="char"><p>0.219</p></td><td char="." align="char"><p>1.031</p></td></tr><tr><td align="left"><p>HIERARC1</p></td><td char="." align="char"><p>0.189</p></td><td char="." align="char"><p>0.213</p></td><td char="." align="char"><p>0.924</p></td><td char="." align="char"><p>0.399</p></td><td char="." align="char"><p>0.229</p></td><td char="." align="char"><p>0.996</p></td></tr><tr><td align="left"><p>HIERARC2</p></td><td char="." align="char"><p>0.417</p></td><td char="." align="char"><p>0.217</p></td><td char="." align="char"><p>1.008</p></td><td char="." align="char"><p> − 0.063</p></td><td char="." align="char"><p>0.230</p></td><td char="." align="char"><p>0.914</p></td></tr><tr><td align="left"><p>HIERARC3</p></td><td char="." align="char"><p> − 0.169</p></td><td char="." align="char"><p>0.213</p></td><td char="." align="char"><p>0.902</p></td><td char="." align="char"><p>0.114</p></td><td char="." align="char"><p>0.189</p></td><td char="." align="char"><p>1.104</p></td></tr><tr><td align="left"><p>INTUITIV</p></td><td char="." align="char"><p> − 1.169</p></td><td char="." align="char"><p>0.225</p></td><td char="." align="char"><p>0.853</p></td><td char="." align="char"><p> − 0.861</p></td><td char="." align="char"><p>0.204</p></td><td char="." align="char"><p>1.049</p></td></tr><tr><td align="left"><p>MATCH</p></td><td char="." align="char"><p> − 0.409</p></td><td char="." align="char"><p>0.211</p></td><td char="." align="char"><p>1.029</p></td><td char="." align="char"><p>0.966</p></td><td char="." align="char"><p>0.202</p></td><td char="." align="char"><p>1.017</p></td></tr><tr><td align="left"><p>MINI1</p></td><td char="." align="char"><p> − 0.681</p></td><td char="." align="char"><p>0.217</p></td><td char="." align="char"><p><bold>1.134</bold></p></td><td char="." align="char"><p> − 1.491</p></td><td char="." align="char"><p>0.237</p></td><td char="." align="char"><p>0.948</p></td></tr><tr><td align="left"><p>MINI2</p></td><td char="." align="char"><p>0.223</p></td><td char="." align="char"><p>0.212</p></td><td char="." align="char"><p>0.965</p></td><td char="." align="char"><p>0.293</p></td><td char="." align="char"><p>0.191</p></td><td char="." align="char"><p>0.902</p></td></tr><tr><td align="left"><p>MINI3</p></td><td char="." align="char"><p> − 0.684</p></td><td char="." align="char"><p>0.217</p></td><td char="." align="char"><p>1.018</p></td><td char="." align="char"><p> − 0.997</p></td><td char="." align="char"><p>0.208</p></td><td char="." align="char"><p>1.017</p></td></tr><tr><td align="left"><p>NATURAL</p></td><td char="." align="char"><p> − 1.873</p></td><td char="." align="char"><p>0.265</p></td><td char="." align="char"><p>0.944</p></td><td char="." align="char"><p> − 1.465</p></td><td char="." align="char"><p>0.232</p></td><td char="." align="char"><p>1.059</p></td></tr><tr><td align="left"><p>PRECISE</p></td><td char="." align="char"><p>1.225</p></td><td char="." align="char"><p>0.235</p></td><td char="." align="char"><p>0.893</p></td><td char="." align="char"><p> − 0.268</p></td><td char="." align="char"><p>0.192</p></td><td char="." align="char"><p>0.958</p></td></tr><tr><td align="left"><p>PREVCONC</p></td><td char="." align="char"><p> − 0.397</p></td><td char="." align="char"><p>0.212</p></td><td char="." align="char"><p>0.958</p></td><td char="." align="char"><p>0.505</p></td><td char="." align="char"><p>0.190</p></td><td char="." align="char"><p>1.060</p></td></tr><tr><td align="left"><p>SUITABLE</p></td><td char="." align="char"><p> − 1.340</p></td><td char="." align="char"><p>0.243</p></td><td char="." align="char"><p><bold>1.178</bold></p></td><td char="." align="char"><p>1.009</p></td><td char="." align="char"><p>0.200</p></td><td char="." align="char"><p>1.040</p></td></tr><tr><td align="left"><p>UNDTAR</p></td><td char="." align="char"><p>0.084</p></td><td char="." align="char"><p>0.210</p></td><td char="." align="char"><p>0.971</p></td><td char="." align="char"><p>0.907</p></td><td char="." align="char"><p>0.204</p></td><td char="." align="char"><p>0.996</p></td></tr><tr><td align="left"><p>USEFUL</p></td><td char="." align="char"><p>0.431</p></td><td char="." align="char"><p>0.215</p></td><td char="." align="char"><p><bold>1.148</bold></p></td><td char="." align="char"><p>0.080</p></td><td char="." align="char"><p>0.189</p></td><td char="." align="char"><p>0.943</p></td></tr><tr><td align="left"><p>WELLDEF1</p></td><td char="." align="char"><p>1.962</p></td><td char="." align="char"><p>0.268</p></td><td char="." align="char"><p>0.965</p></td><td char="." align="char"><p>0.846</p></td><td char="." align="char"><p>0.202</p></td><td char="." align="char"><p>0.987</p></td></tr><tr><td align="left"><p>WELLDEF2</p></td><td char="." align="char"><p>1.870</p></td><td char="." align="char"><p>0.279</p></td><td char="." align="char"><p>1.092</p></td><td char="." align="char"><p>0.819</p></td><td char="." align="char"><p>0.199</p></td><td char="." align="char"><p>0.992</p></td></tr></tbody></table> </ephtml> </p> <p>The results from the simulations showed that:</p> <p></p> <ulist> <item> The largest statement infit mean square from 95% of the statements of lowest infit mean square was 1.1345 (in RM) and 1.1092 (in MTE).</item> <p></p> <item> The largest judge infit mean square from 95% of the judges of lowest infit mean square was 1.406 (in RM) and 1.348 (in MTE).</item> <p></p> <item> The middle 95-percentile of the simulated inter-reliability was in the interval [0.8633, 0.9539] (in RM) and [0.8462 0.9466] (in MTE).</item> </ulist> <p>See Fig. 5</p> <p>Graph: Fig. 5 Scatter plot of estimates after 1000 iterations of split-halves of RM judges (left) and MTE judges (right). Pearsons' correlation coefficient, R, is for the entire data set of 1000 iterations</p> <ref id="AN0189055834-24"> <title> References </title> <blist> <bibl id="bib1" idref="ref15" type="bt">1</bibl> <bibtext> Alcock L, Simpson A. Definitions: Dealing with categories mathematically. For the Learning of Mathematics. 2002; 22; 2: 28-34</bibtext> </blist> <blist> <bibl id="bib2" idref="ref4" type="bt">2</bibl> <bibtext> Ball DL. With an eye on the mathematical horizon: Dilemmas of teaching elementary school mathematics. The Elementary School Journal. 1993; 93; 4: 373-397. 10.1086/461730</bibtext> </blist> <blist> <bibl id="bib3" idref="ref1" type="bt">3</bibl> <bibtext> Ball DL, Bass HKilpatrick J, Martin WG, Schifter D. Making mathematics reasonable in school. A research companion to principles and standards for school mathematics. 2003; NCTM: 27-44</bibtext> </blist> <blist> <bibl id="bib4" idref="ref73" type="bt">4</bibl> <bibtext> Bergman AM, Kercher A, Gallagher K, Zazkis R. Definitional ambiguity in mathematics: Three cases. Educational Studies in Mathematics. 2024; 115; 1: 93-110. 10.1007/s10649-023-10241-0</bibtext> </blist> <blist> <bibl id="bib5" idref="ref40" type="bt">5</bibl> <bibtext> Bisson M-J, Gilmore C, Inglis M, Jones I. Measuring Conceptual Understanding Using Comparative Judgement. International Journal of Research in Undergraduate Mathematics Education. 2016; 2; 2: 141-164. 10.1007/s40753-016-0024-3</bibtext> </blist> <blist> <bibl id="bib6" idref="ref41" type="bt">6</bibl> <bibtext> Bisson, M. J, Gilmore, C, Inglis, M, & Jones, I. (2020). Teaching using contextualised and decontextualised representations: Examining the case of differential calculus through acomparative judgement technique. Research in Mathematics Education, 28(3), 84–303. https://doi.org/10.1080/14794802.2019.1692060</bibtext> </blist> <blist> <bibl id="bib7" idref="ref65" type="bt">7</bibl> <bibtext> Bond, T. G, & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Lawrence Erlbaum Associates Publishers.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref35" type="bt">8</bibl> <bibtext> Bradley RA, Terry ME. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika. 1952; 39; 3/4: 324-345. 10.2307/2334029</bibtext> </blist> <blist> <bibl id="bib9" idref="ref60" type="bt">9</bibl> <bibtext> Bramley T, Vitello S. The effect of adaptivity on the reliability coefficient in adaptive comparative judgement. Assessment in Education: Principles, Policy & Practice. 2019; 26; 1: 43-58. 10.1080/0969594X.2017.1418734</bibtext> </blist> <blist> <bibtext> Czocher JA, Weber K. Proof as a cluster category. Journal for Research in Mathematics Education. 2020; 51; 1: 50-74. 10.5951/jresematheduc.2019.0007</bibtext> </blist> <blist> <bibtext> Davies, B. (2020). Comparative judgment and proof (Doctoral dissertation). University of Southhampton. https://doi.org/10.13140/RG.2.2.33043.63527</bibtext> </blist> <blist> <bibtext> Davies B, Alcock L, Jones I. What do mathematicians mean by proof? A comparative-judgement study of students' and mathematicians' views. The Journal of Mathematical Behavior. 2021; 61: 100824. 10.1016/j.jmathb.2020.100824</bibtext> </blist> <blist> <bibtext> Edwards B, Ward MB. Surprises from mathematics education research: Student (mis)use of mathematical definitions. The American Mathematical Monthly. 2004; 111; 5: 411-424. 10.1080/00029890.2004.11920092</bibtext> </blist> <blist> <bibtext> Forbregd, T, Torkildsen, H, Kaspersen, E, & Solstad, T. (2021). Comparative judgement of statements about mathematical definitions [Data set]. DataverseNO. https://doi.org/10.18710/EOZKTR</bibtext> </blist> <blist> <bibtext> Harel G, Selden A, Selden JGutiérrez A, Boero P. Advanced mathematical thinking: Some PME perspectives. Handbook of research on the psychology of mathematics education. 2006; Brill: 147-172. 10.1163/9789087901127_007</bibtext> </blist> <blist> <bibtext> Hunter, D. R. (2004). MM algorithms for generalized Bradley-Terry models. The Annals of Statistics, 32(1), 384–406. https://doi.org/10.1214/aos/1079120141</bibtext> </blist> <blist> <bibtext> Hunter J, Jones IHunter J, Perger P, Darragh L. Free-response tasks in primary mathematics: A window on students' thinking. Proceedings of the 41st Annual Conference of the Mathematics Education Research Group of Australasia. 2018; MERGA: 400-407</bibtext> </blist> <blist> <bibtext> Jones I, Alcock L. Peer assessment without assessment criteria. Studies in Higher Education. 2014; 39; 10: 1774-1787. 10.1080/03075079.2013.821974</bibtext> </blist> <blist> <bibtext> Jones I, Bisson M, Gilmore C, Inglis M. Measuring conceptual understanding in randomised controlled trials: Can comparative judgement help?. British Educational Research Journal. 2019; 45; 3: 662-680. 10.1002/berj.3519</bibtext> </blist> <blist> <bibtext> Jones I, Inglis M. The problem of assessing problem solving: Can comparative judgement help?. Educational Studies in Mathematics. 2015; 89; 3: 337-355. 10.1007/s10649-015-9607-1</bibtext> </blist> <blist> <bibtext> Jones I, Swan M, Pollitt A. Assessing mathematical problem solving using comparative judgement. International Journal of Science and Mathematics Education. 2014; 13; 1: 151-177. 10.1007/s10763-013-9497-6</bibtext> </blist> <blist> <bibtext> Lampert M. When the problem is not the question and the solution is not the answer: Mathematical knowing and teaching. American Educational Research Journal. 1990; 27; 1: 29-63. 10.3102/0002831202700102</bibtext> </blist> <blist> <bibtext> Leikin R, Zazkis R. On the content-dependence of prospective teachers' knowledge: A case of exemplifying definitions. International Journal of Mathematical Education in Science and Technology. 2010; 41; 4: 451-466. 10.1080/00207391003605189</bibtext> </blist> <blist> <bibtext> Linchesvky L, Vinner S, Karsenty R. To be or not to be minimal? Student teachers' views about definitions in geometry. Proceedings of the Sixteenth International Conference for the Psychology of Mathematics Education, Durham, USA, 1992. 1992; 2: 48-55</bibtext> </blist> <blist> <bibtext> Mariotti MA, Fischbein E. Defining in classroom activities. Educational Studies in Mathematics. 1997; 34: 219-248. 10.1023/A:1002985109323</bibtext> </blist> <blist> <bibtext> Miller, S. M. (2018). An analysis of the form and content of quadrilateral definitions composed by novice pre-service teachers. The Journal of Mathematical Behavior, 50, 142–154. https://doi.org/10.1016/j.jmathb.2018.02.006</bibtext> </blist> <blist> <bibtext> No More Marking. (n.d.). Retrieved Ferbuary 17, 2024, from https://<ulink href="http://www.nomoremarking.com/">www.nomoremarking.com/</ulink></bibtext> </blist> <blist> <bibtext> Ouvrier-Buffet C. Exploring mathematical definition construction processes. Educational Studies in Mathematics. 2006; 63; 3: 259-282. 10.1007/s10649-005-9011-3</bibtext> </blist> <blist> <bibtext> Ouvrier-Buffet C. A mathematical experience involving defining processes: In-action definitions and zero-definitions. Educational Studies in Mathematics. 2011; 76; 2: 165-182. 10.1007/s10649-010-9272-3Scopus</bibtext> </blist> <blist> <bibtext> Pollitt A. The method of adaptive comparative judgement. Assessment in Education: Principles, Policy & Practice. 2012; 19; 3: 281-300. 10.1080/0969594X.2012.665354</bibtext> </blist> <blist> <bibtext> R Core Team. (2021). R: A language and environment for statistical computing. Foundation for Statistical Computing. Retrieved August 1, 2023, from https://<ulink href="http://www.r-project.org/">www.r-project.org/</ulink></bibtext> </blist> <blist> <bibtext> Robitzsch, A. (2020). sirt: Supplementary item response theory models (3.9–4) [Computer software]. Retrieved August 1, 2023, from https://CRAN.R-project.org/package=sirt</bibtext> </blist> <blist> <bibtext> Rupnow R, Randazzo B. Norms of mathematical definitions: Imposing constraints, permitting choice, or both?. Educational Studies in Mathematics. 2023; 114; 2: 297-314. 10.1007/s10649-023-10227-y</bibtext> </blist> <blist> <bibtext> Schoenfeld, A. H. (1994). Reflections on doing and teaching mathematics. In A. H. Schoenfeld & A. H. Sloane (Eds.), Mathematical thinking and problem solving.The study of mathematical thinking and learning (pp. 53–75). Lawrence Erlbaum.</bibtext> </blist> <blist> <bibtext> Torkildsen, H. A, Forbregd, T, Kaspersen, E, & Solstad, T. (2023). Toward a unified account of definitions in mathematics education research: A systematic literature review. International Journal of Mathematical Education in Science and Technology.https://doi.org/10.1080/0020739X.2023.2180678</bibtext> </blist> <blist> <bibtext> Van Dormolen J, Zaslavsky O. The many facets of a definition: The case of periodicity. The Journal of Mathematical Behavior. 2003; 22; 1: 91-106. 10.1016/S0732-3123(03)00006-3</bibtext> </blist> <blist> <bibtext> Verhavert S, De Maeyer S, Donche V, Coertjens L. Scale separation reliability: What does it mean in the context of comparative judgment?. Applied Psychological Measurement. 2018; 42; 6: 428-445. 10.1177/0146621617748321</bibtext> </blist> <blist> <bibtext> Wright, B. D, & Stone, M. H. (1979). Best test design. MESA Press.</bibtext> </blist> <blist> <bibtext> Zandieh M, Rasmussen C. Defining as a mathematical activity: A framework for characterizing progress from informal to more formal ways of reasoning. The Journal of Mathematical Behavior. 2010; 29; 2: 57-75. 10.1016/j.jmathb.2010.01.001</bibtext> </blist> <blist> <bibtext> Zaslavsky O, Shir K. Students' conceptions of a mathematical definition. Journal for Research in Mathematics Education. 2005; 36; 4: 317-346. 10.2307/30035043</bibtext> </blist> </ref> <aug> <p>By Hermund André Torkildsen; Tore A. Forbregd; David A. Reid and Shaista Kanwal</p> <p>Reported by Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib22" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib13" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib25" firstref="ref6"></nolink> <nolink nlid="nl4" bibid="bib29" firstref="ref7"></nolink> <nolink nlid="nl5" bibid="bib39" firstref="ref8"></nolink> <nolink nlid="nl6" bibid="bib28" firstref="ref9"></nolink> <nolink nlid="nl7" bibid="bib35" firstref="ref10"></nolink> <nolink nlid="nl8" bibid="bib36" firstref="ref11"></nolink> <nolink nlid="nl9" bibid="bib15" firstref="ref12"></nolink> <nolink nlid="nl10" bibid="bib10" firstref="ref16"></nolink> <nolink nlid="nl11" bibid="bib33" firstref="ref31"></nolink> <nolink nlid="nl12" bibid="bib12" firstref="ref36"></nolink> <nolink nlid="nl13" bibid="bib20" firstref="ref39"></nolink> <nolink nlid="nl14" bibid="bib19" firstref="ref42"></nolink> <nolink nlid="nl15" bibid="bib17" firstref="ref43"></nolink> <nolink nlid="nl16" bibid="bib27" firstref="ref48"></nolink> <nolink nlid="nl17" bibid="bib21" firstref="ref49"></nolink> <nolink nlid="nl18" bibid="bib14" firstref="ref50"></nolink> <nolink nlid="nl19" bibid="bib16" firstref="ref54"></nolink> <nolink nlid="nl20" bibid="bib32" firstref="ref55"></nolink> <nolink nlid="nl21" bibid="bib31" firstref="ref56"></nolink> <nolink nlid="nl22" bibid="bib18" firstref="ref58"></nolink> <nolink nlid="nl23" bibid="bib30" firstref="ref59"></nolink> <nolink nlid="nl24" bibid="bib11" firstref="ref61"></nolink> <nolink nlid="nl25" bibid="bib37" firstref="ref64"></nolink> <nolink nlid="nl26" bibid="bib40" firstref="ref86"></nolink> <nolink nlid="nl27" bibid="bib38" firstref="ref97"></nolink> <nolink nlid="nl28" bibid="bib26" firstref="ref102"></nolink> <nolink nlid="nl29" bibid="bib24" firstref="ref103"></nolink> <nolink nlid="nl30" bibid="bib34" firstref="ref106"></nolink> <nolink nlid="nl31" bibid="bib23" firstref="ref107"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1493014
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Norwegian Mathematics Teacher Educators' and Research Mathematicians' Views on Different Aspects of Mathematical Definitions: A Comparative Judgement Study
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Hermund+André+Torkildsen%22">Hermund André Torkildsen</searchLink><br /><searchLink fieldCode="AR" term="%22Tore+A%2E+Forbregd%22">Tore A. Forbregd</searchLink><br /><searchLink fieldCode="AR" term="%22David+A%2E+Reid%22">David A. Reid</searchLink><br /><searchLink fieldCode="AR" term="%22Shaista+Kanwal%22">Shaista Kanwal</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-4753-5013">0000-0003-4753-5013</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Science+and+Mathematics+Education%22"><i>International Journal of Science and Mathematics Education</i></searchLink>. 2025 23(7):2157-2180.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 24
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Education%22">Mathematics Education</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Teachers%22">Mathematics Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Educators%22">Teacher Educators</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Researchers%22">Teacher Researchers</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Attitudes%22">Teacher Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+Concepts%22">Mathematical Concepts</searchLink><br /><searchLink fieldCode="DE" term="%22Definitions%22">Definitions</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Research%22">Educational Research</searchLink><br /><searchLink fieldCode="DE" term="%22Ambiguity+%28Semantics%29%22">Ambiguity (Semantics)</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+Logic%22">Mathematical Logic</searchLink><br /><searchLink fieldCode="DE" term="%22Comprehension%22">Comprehension</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Norway%22">Norway</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10763-024-10534-7
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1571-0068<br />1573-1774
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In this article we investigate the extent to which characteristics used to describe mathematical definitions in mathematics education research literature reflect what is important to research mathematicians and mathematics teacher educators. We report results from a comparative judgement survey of 57 research mathematicians and 62 mathematics teacher educators. Our results indicate that the two groups are mostly in agreement about the rankings, however, there are some differences. Research mathematicians rank most highly characteristics related to non-ambiguity and non-contradiction, while mathematics teacher educators rank characteristics related to comprehension and referential clarity as highly as characteristics related to non-ambiguity and non-contradiction. Neither group ranked minimality highly, even though this is the characteristic most often listed in the mathematics education research literature. Our study contributes towards a more consistent characterization of mathematical definitions in mathematics education research.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1493014
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1493014
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10763-024-10534-7
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 24
        StartPage: 2157
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Mathematics Education
        Type: general
      – SubjectFull: Mathematics Teachers
        Type: general
      – SubjectFull: Teacher Educators
        Type: general
      – SubjectFull: Teacher Researchers
        Type: general
      – SubjectFull: Teacher Attitudes
        Type: general
      – SubjectFull: Mathematical Concepts
        Type: general
      – SubjectFull: Definitions
        Type: general
      – SubjectFull: Educational Research
        Type: general
      – SubjectFull: Ambiguity (Semantics)
        Type: general
      – SubjectFull: Mathematical Logic
        Type: general
      – SubjectFull: Comprehension
        Type: general
      – SubjectFull: Norway
        Type: general
    Titles:
      – TitleFull: Norwegian Mathematics Teacher Educators' and Research Mathematicians' Views on Different Aspects of Mathematical Definitions: A Comparative Judgement Study
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Hermund André Torkildsen
      – PersonEntity:
          Name:
            NameFull: Tore A. Forbregd
      – PersonEntity:
          Name:
            NameFull: David A. Reid
      – PersonEntity:
          Name:
            NameFull: Shaista Kanwal
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1571-0068
            – Type: issn-electronic
              Value: 1573-1774
          Numbering:
            – Type: volume
              Value: 23
            – Type: issue
              Value: 7
          Titles:
            – TitleFull: International Journal of Science and Mathematics Education
              Type: main
ResultId 1