Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7 69
Developing a digital tool for vignette-based professional development
Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7 70
use of video technology in mathematics teacher education and professional development. Video vignettes are expected to contain a high amount of context information of a classroom situation and are, therefore, considered as particularly close to the classroom situation they represent and to its professional requirements (e.g.
Richter et al., 2019; Petko et al., 2003).
Still, there is a need of empirical evidence about the specific strengths of video vignettes in comparison with other vignette formats, such as text vignettes, cartoons, or animations. For the design of vignette-based digital learning or assessment environments, empirical evidence is, however, needed in the search for effective ways of technology use. Consequently, this paper addresses this research need and collects available findings with relevance for identifying potentials of different vignette formats. Implications will be drawn for the development of a digital tool for representing classroom situations in the European project coReflect@maths.
The paper will first outline a theoretical background related to vignette-based professional learning and assessment of pre- and in-service teachers. Based on the research interest derived from this background, we will then present and examine available findings related to different vignette formats and to their respective potential.
Against these findings, conclusions will be discussed for the design of a digital tool for representing classroom situations in the project coReflect@maths, and an outlook on further related project goals and activities will be given.
THEORETICAL BACKGROUND
Vignette-based professional learning in mathematics education aims at fostering a broad range of aspects of professional expertise. In recent approaches of describing aspects of mathematics teachers’ expertise, several interrelated and overlapping terms are used, such as “awareness” (Mason, 2002), “noticing” in the sense of “selective attention” (e.g., Seidel, Blomberg, & Renkl, 2013) or “knowledge-based reasoning”
(Sherin, Jacobs, & Philipp, 2011), “professional vision” (Sherin & van Es, 2009), or
“usable knowledge” (Kersting et al., 2012). All these terms have in common that they relate aspects of teachers’ expertise to profession-related situation contexts, such as classroom situations (Kuntze & Friesen, 2018), so that in these approaches, profession-related situation contexts are particularly relevant for assessing teachers’ expertise and for teachers’ professional learning.
However, classroom situations are very complex (Petko et al., 2003): in classroom situations, many processes take place simultaneously and some of them are not directly visible or hearable. Moreover, an individual situation in the classroom happens only once in the same way and cannot be reproduced exactly as it had originally taken place.
Even if a classroom situation is recorded on video, the video can capture only specific perspectives and only visual and auditive information can be stored, so that multitude of perspectives and thoughts of all the actors of the situation, for instance, cannot be mapped. Therefore, even if videos may cover many aspects of a classroom situation in an information-rich way, it is not possible to cover all aspects of it. Classroom videos
Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7 71
thus should not be confounded with the situation they show, they are only representations of them (Buchbinder & Kuntze, 2018).
In general, classroom vignettes can be understood as representations of practice.
According to Buchbinder and Kuntze (2018), a representation of practice (e.g. a classroom situation) is something that stands for this classroom situation (e.g. a cartoon showing a conversation between two students, or a transcript of a dialog) and represents some, but not all aspects of the classroom situation it refers to.
This reduction of information entails some key benefits of vignettes: Vignettes can be viewed (or read, etc.) repeatedly and independently from a specific classroom situation;
they can be stored, collected and provided to other persons. Vignettes, therefore, afford an easy access to analysing elements of classroom situations by teachers (Kuntze, 2018), which makes vignettes a useful instrument in mathematics teacher education in many ways (e.g. Skilling & Stylianides, 2019). In particular, vignettes can help to bridge the gap between theoretical contents and goals in mathematics education on the one hand, and requirements of specific classroom situations on the other hand.
Vignettes can use different formats, which can be expected to have different specific advantages, but also challenges. While video vignettes, for instance, may cover a relatively high amount of information from the classroom situation they represent, creating such a vignette requires effort, and data protection issues requires care and may restrict their use. Compared to cartoons, for example, video vignettes can hardly be constructed quickly or modified easily so as to contain specific elements intended for reflection. Cartoon or text vignettes contain less information, but are more flexible in use and can be constructed relatively easily. Especially without the help of a digital tool, the design of cartoon vignettes requires creating and dealing with graphical elements, which may represent a difficulty compared with text vignettes. A reduced complexity also can be a benefit as it may support the access for learners and reduce cognitive load (e.g. Syring et al., 2015).
RESEARCH AIM
When it comes to developing vignette-based learning opportunities or vignette-based assessment instruments, it has to be decided which vignette format to choose and how to frame the vignette-based work. For these design decisions, empirical evidence about best-practice ways of using vignettes is needed, as different vignette formats might impact differently on teachers’ analysis outcomes, for instance. A video vignette might, for example, better support teachers in engaging with the represented classroom situation compared to a text vignette, but the large amount of potentially irrelevant context information of the video vignette might also be an obstacle for successful noticing or criteria-based analysis. Empirical research related to the potential of different vignette formats is particularly essential for the design of digital tools and environments which aim at facilitating vignette-based work by supporting their design and their implementation as learning opportunities. In particular, the following questions are in the center: Are there differences between vignette formats, as far as
Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7 72
characteristics of their effectiveness for professional learning and assessment (such as noticing, analysis, perceptions of authenticity, etc.) are concerned? Which vignette formats can be considered as particularly effective according to available empirical findings? What implications can be drawn for the design of a digital tool facilitating the creation of vignette-based learning opportunities?
Responding to this research need, we will in the following collect empirical findings concerning comparisons of different vignette formats. Based on this, we will discuss implications for the development of a digital tool, which aims at supporting the design of vignette-based learning opportunities.
EMPIRICAL FINDINGS ON DIFFERENT VIGNETTE FORMATS
In a study with N = 298 pre-service and in-service teachers (Friesen, 2017; Friesen &
Kuntze, 2018), three different vignette formats (text, video, cartoon) were compared in terms of the extent of engagement perceived by the participants when analysing the vignettes; engagement can be seen as important prerequisite for sufficiently representing classroom situations by means of vignettes (ibid.). For evaluating to what extent teachers were engaged with the vignettes, they were asked by means of Likert scales about the extent of their (1) motivation to analyse the vignettes, (2) regarding perceived authenticity of the vignette, and to what extent they perceived (3) immersion and (4) resonance when analysing the vignettes. Immersion means that participants felt
“put in” the situation; resonance refers to whether one thinks about the own professional practice when working with the vignettes (cf. Seidel et al., 2011).
Moreover, it was investigated whether the participants’ analyses varied among the different vignette formats, which would indicate that the vignette format can have an impact on teachers’ analysis of the represented classroom situations.
For the study, sets of three vignettes in different formats (text, cartoon, and video) were designed representing the same classroom situation. In order to validate the accordance of the vignettes to the respective classroom situations, the sets of vignettes were subjected to an expert rating by teacher educators. Based on that rating, six sets of vignettes rated as highly authentic and representative were chosen. The vignettes together with Likert scales for measuring how teachers assess the authenticity of each vignette and their perceived engagement were administered in a multiple matrix design to the participants (Friesen, 2017; Friesen & Kuntze, 2018).
In the analysis of the teachers’ responses it was found that the perceived motivation, immersion, and resonance was on a similar and relatively high level for all three vignette formats. The authenticity of the video-based vignettes was rated by most of the participants on average as less authentic than the text and cartoon vignettes; only the sub-sample of n = 22 in-service teachers rated the authenticity of video vignettes similarly as the authenticity of text and cartoon vignettes. Based on teachers’ analyses of the vignettes, a Rasch analysis was conducted. No evidence was found implying multiple dimensions, e.g. according to different vignette formats. There were also no implications for interrelatedness of vignette formats and teachers’ analysis of the
Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7 73
represented classroom situations, which would have become apparent in divergent item difficulties. In conclusion, the results did not reveal relevant differences according to different vignette formats in terms of engagement, and there were no hints that the vignette format influenced teachers’ analysis.
A study with N = 61 teacher candidates by Herbst, Aaron, and Erickson (2013) provided similar results. In this study, it was investigated whether there are differences of using animations (animated cartoons with audio track) compared with videos of classroom situations in the categories genuineness, projectiveness, mathematics, noticing, reflectiveness, and alternativity, which are described in detail in Herbst et al.
(2013). The results show that participants only rated authenticity of the video vignettes higher; no other significant differences were found. Similar to Friesen’s (2017) study, the study did not reveal any hints that the participants’ analyses of the represented classroom situations were influenced by different vignette formats. The authors, therefore, conclude that videos and animations “can be comparably effective” (ibid, p.
11). In a further qualitative study (Herbst & Kosko, 2014), there were also no relevant differences between video vignettes and vignettes with animations. The authors, therefore, concluded, that “animations are just as useful as videos” (p. 515) for investigating teachers’ professional knowledge.
In conclusion, the reported empirical findings indicate that the different vignette formats appear to be similarly effective to make classroom situations accessible for teachers’ analysis.
DEVELOPING A DIGITAL TOOL FOR VIGNETTE-BASED LEARNING A central aim of the European project coReflect@maths is to develop a digital tool (DIVER – Designing and Investigating Vignettes for Education and Research Tool), which supports pre- and in-service teachers as well as teacher students in creating, sharing, and collaboratively reflecting on vignettes representing classroom situations.
Existing online tools are mostly limited to one vignette format (e.g. cartoons or videos), can only be used by speakers of one language (e.g. English), and do often not sufficiently solve issues with data protection, which is a major concern in educational contexts. The DIVER tool will be programmed as a plugin for the learning platform Moodle. Moodle is used by many European universities and provides a secure, data-protected learning environment in different languages. Since it is open-source, external applications such as DIVER can be integrated to add specific functionalities. Another advantage of Moodle is its potential for remote teaching since it allows the implementation of online courses as well as blended learning scenarios.
As the empirical findings outlined above do not imply general advantages of video vignettes in comparison with the other formats, the development of the DIVER tool mainly concentrates on cartoon vignettes, as they can combine advantages of video-based and text-video-based vignettes without sharing most of their disadvantages. For instance, cartoon vignettes allow to sketch various classroom situations and to vary them systematically, which hardly can be done with video vignettes. In addition, as
Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7 74
cartoons afford representing aspects of classroom situations visually, long descriptions, which could be needed when choosing text vignettes, are not necessary. A central aim of the tool, therefore, is to support the easy creation of cartoon-based vignettes. At the same time, the tool will also facilitate the integration of text and video vignettes within the same environment, as multiple vignette formats might support specific goals of professional learning.
For developing vignettes, the tool is intended to have an easy access graphical interface containing several graphical elements (different student and teacher characters, classroom environments, classroom material, etc.) that can easily be arranged in order to create classroom scenarios. There will be possibilities to add speech bubbles, to edit students’ notebooks, and to add writings on the board, so that there are possibilities of creating cartoon vignettes on the base of video material which cannot be published as a consequence of data protection limitations. The creation of cartoon-based vignettes in DIVER is supposed to support creating classroom scenarios in a systematic way, highlighting, e.g., certain quality aspects of teaching and learning in the mathematics classroom. It allows at the same time to create classroom situations based on personal teaching experience of pre-service and in-service teachers. By capturing that teaching experience, e.g. in the form of a cartoon-based vignette, it can be shared with colleagues and made accessible to collaborative analysis and reflection as well as to further improvement. For facilitating the sharing of vignettes and collaborative reflection on presented classroom practice, it is planned that the DIVER tool should also offer the opportunity to follow up on already designed cartoon-based or existing video-based vignettes that can then be analysed and commented on within the tool.
This function of the tool can also be used for evaluating vignette-based university courses or PD programmes and holds, therefore, potential for corresponding evaluation research.
OUTLOOK
In coReflect@maths, we plan to implement the support of the languages English, Spanish, German, and Czech. The relatively quick possibility of translating speech bubbles in cartoon vignettes will enable a multinational use of some of the vignettes, highlighting a further advantage of cartoon vignettes. This possibility will also enable to detect, compare, and discuss cultural specificities of the mathematics classroom of the different member countries. Corresponding multi-lingual vignette-based material will be published on the project homepage (www.coreflect.eu) for public use by the end of the project.
ACKNOWLEDGEMENTS
The project is co-funded by funds distributed by the Senate of Ludwigsburg University of Education and in the framework of the Erasmus+ Programme of the European Union (project coReflect@maths, 2019-1-DE01-KA203-004947). The European Commission’s support for the production of this publication does not constitute an
Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7 75
endorsement of the contents, which reflect the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
REFERENCES
Blume, F., Göllner, R., Möller, K., Dresler, T., Ehlis, A.-C., & Gawrilow, C. (2018).
Do students learn better when seated close to the teacher? A virtual classroom study considering individual levels of inattention and hyperactivity-impulsivity. Learning and Instruction, 61, 138 –147.
Buchbinder, O., & Kuntze, S. (2018). Representations of Practice in Teacher Education and Research – Spotlights on Different Approaches. In O. Buchbinder & S. Kuntze (Eds.), Mathematics Teachers Engaging with Representations of Practice. A Dynamically Envolving Field (pp. 1–8). Cham, Switzerland: Springer
Friesen, M. (2017). Teachers’ competence of analysing the use of multiple representations in mathematics classroom situations and its assessment in a vignette-based test (Doctoral thesis). Retrieved from https://phbl-opus.phlb.de/frontdoor/deliver/index/docId/545/file/Dissertationsschrift_Friesen_O nlineversion.pdf.
Friesen, M. & Kuntze, S. (2018). Competence Assessment with Representations of Practice in Text, Comic and Video Format. In O. Buchbinder & S. Kuntze (Eds.), Mathematics Teachers Engaging with Representations of Practice. A Dynamically Evolving Field (pp. 113–130). Cham, Switzerland: Springer.
Herbst, P., Aaron, W., & Erickson, A. (2013). How Preservice Teachers Respond to Representations of Practice: A Comparison of Animations and Video. Paper presented at the 2013 Annual Meeting of the American Educational Research Association, San Francisco. Retrieved from http://hdl.handle.net/ 2027.42/97424.
Herbst, P. & Kosko, K.W. (2014). Using representations of practice to elicit mathematics teachers’ tacit knowledge of practice: a comparison of responses to animations and videos. Journal of Mathematics Teacher Educ., 17(6), 515-537.
Kersting, N., Givvin, K., Thompson, B., Santagata, R, & Stigler, J. (2012). Measuring usable knowledge: Teachers’ analyses of mathematics classroom videos predict teaching quality and student learning. Am. Educ. Res. J., 49(3), 568–589.
Kuntze, S. (2018). Representations of Practice in a Video-Based In-Service Teacher Professional Development Project and in Its Evaluation. In O. Buchbinder & S.
Kuntze (Eds.), Mathematics Teachers Engaging with Representations of Practice. A Dynamically Envolving Field (pp. 47–70). Cham, Switzerland: Springer
Kuntze, S., & Friesen, M. (2018). The role of mathematics teachers’ views for their competence of analysing classroom situations. In B. Rott, G. Törner, J. Peters-Dasdemir, A. Möller, Safrudiannur (Eds.), Views and Beliefs in Mathematics
Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7 76
Education. The Role of Beliefs in the Classroom (pp. 183–194). Cham, Switzerland:
Springer.
Mason, J. (2002). Researching your own practice. The discipline of noticing. London:
Routledge Falmer.
Petko, D., Waldis, M., Pauli, C., & Reusser, K. (2003). Methodologische Überlegungen zur videogestützten Forschung in der Mathematikdidaktik.
ZDM, 35(6), 265–280.
Richter, D., Wiepke, A., Richter, E., Huang, Y., & Zender, R. (2019). “Nutzung von Virtual Reality-Technologie zur Erfassung von Klassenführungskompetenzen in der Lehrerausbildung”. Poster presented at the 7th annual conference of the Gesellschaft für Empirische Bildungsforschung (GEBF), Cologne.
Seidel, T., Blomberg, G., & Renkl, A. (2013). Instructional strategies for using video in teacher education. Teaching and Teacher Education, 34, 56–65.
Seidel, T., Stürmer, K., Blomberg, G., Kobarg, M., & Schwindt, K. (2011). Teacher learning from analysis of videotaped classroom situations: Does it make a difference whether teachers observe their own teaching or that of others? Teaching and Teacher Education, 27(2), 259–267.
Sherin, M. G., & van Es, E. A. (2009). Effects of video club participation on teachers’
professional vision. Journal of Teacher Education, 60(1), 20–37.
Sherin, M., Jacobs, V., Philipp, R. (2011). Mathematics teacher noticing. Seeing through teachers’ eyes. New York: Routledge.
Skilling, K. & Stylianides, G. (2019). Using vignettes in educational research: a framework for vignette construction. Intern. Journal of Research and Method in Education.
Syring, M., Kleinknecht, M., Bohl, T., Kuntze, S., Rehm, M., & Schneider, J. (2015).
How problem-based or direct instructional case-based learning environments influence secondary school pre-service teachers’ cognitive load, motivation and emotions: A quasi-experimental intervention study in teacher education. Journal of Education and Human Development, 4(4), 115–129.
Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7 77