How may the collected data be used to answer the research questions on a case-specific level? To answer this, Schoenfeld’s dimension of trustworthiness must be discussed.
However, not all elements can be discussed at this point in the dissertation, since some require a completed data analysis. This particularly goes for the elements of descriptive and explanatory power, but to some extent also falsification, rigor, and specificity.
Thus, in this section I shall discuss the method for answering research questions 1 and 2 (students’ abilities to engage in meta-issue discussions and the anchoring of these, respectively) on the basis oftriangulation,prediction, andreplicability for each of the
4.4 Answering the Research Questions on a Case-Specific Level 105
two teaching modules, and afterwards the answering of research question 3 (possible changes and developments in beliefs due to the modules) on a similar basis.
Before I begin, it is worth mentioning that in the data analysis not all of the eighteen data sources as depicted above may be of equal significance in answering the different research questions. For instance, some data may be more relevant for answering question 1 than question 2, and vice versa. And some will mostly be of relevance to question 3, e.g.
the first round of questionnaires and followup interviews. Furthermore, the significance of a given data source may depend on the possible answers to the research questions.
For instance, some sources may become more relevant if the answer is negative since they may then suggest possible explanations for the negative answers. I shall return to this scenario below.
Concerning the First Teaching Module
I begin by addressing the possible and meaningful triangulations on data sources in order to provide evidence for the answering of research questions 1 and 2. To answer these questions, the idea is to begin by analyzing the first module hand-in essay assignments (introductory and final ones). Such an analysis may provide a ‘broad’ answer to the research questions, based on the essays from all student groups. For question 1, it is possible to compare data from the essay assignments, the second questionnaire, and the second round of interviews to perform a triangulation in order to support conclusions.
For question 2, an indication of an anchoring of the meta-issue discussions in the related in-issues may be supported by looking into the mathematical exercises/problems of the students in a particular group, i.e. the same triangulation as before, only involving yet another data source. A deepening of the answers to question 1 and 2 may be performed for the focus group in particular, by involving the video material in the triangulations also. In the second round of followup interviews, the students were asked questions about their answering of the essay assignments. Thus, a meaningful triangulation to perform for the focus group could be that of hand-in essays, videos, and followup interviews. Again, if required, for question 2 it could become relevant to include the hand-in mathematical exercises in the triangulation as well.
Thus, the triangulations to be performed serve as methodological triangulations in the form of addressing and uncovering the phenomena at hand, but also as a way to validate interpretations and conclusions. The idea of first providing a broad answer and then later to narrow it down in terms of studying individual students is also suggested by Brown (1992, p. 156), who claims that such an approach “enables us to see the magnitude of the effect in terms of outcome measures and to get a feel for the phenomenon itself by looking at a particular child or group in depth.”
Now, as for the video films of the classroom teaching, these may be relevant for different scenarios. One example could be if students appear to do better at engaging in some of the essay assignments compared to others. In this case, the success could be due to the topics of these assignments having been dealt with very thoroughly as part of the teaching. Something the video films of the teaching could give an idea about. In a somewhat similar manner, the videos of the classroom teaching could be used as an extra dimension in triangulations involving the mathematical exercises to see if students’
problems in understanding certain concepts could be traced back to a teaching situation.
Another scenario in which the classroom videos might become relevant are in the case of
106 Methodology
negative answers to the research questions. Say, for instance, that the answer to research question 1 is negative, i.e. that the students when doing their essay assignments are not capable of engaging in certain meta-issue discussions and reflections. A watching of the classroom teaching videos might reveal situations where a topic had not been dealt with properly or explained correctly, either due to a true misconception on the teacher’s behalf or due to poor coaching on my behalf.
Schoenfeld’s element of prediction may be interpreted in two different ways:
1. prediction may refer to the actual capability of a study to predict outcomes of certain experiments, or
2. it may refer to the potential of a given study, its design and methods, to perform predictions.
In the first sense, not much prediction is possible in the first teaching module. This mainly has to do with the fact that no previous empirical, experimental studies have focussed clearly on the use of history as a goal. And that no pre-made methodologies exist for that reason, as well as it not being possible to pose any literature-based hypotheses about the outcome of the teaching module. In the second sense, however, the first teaching module bears a higher degree of prediction. For instance, some prediction may take place as the implementation proceeds, e.g. of the ways in which the students cope with the designed classroom activities. Such predictions can not be stated before formulating the research questions and designing the experiment, as would be expected of the first interpretation. But the design and method used have the potential to make predictions from lesson to lesson, and, more importantly, from implementation to implementation, should the investigation be carried out again.
A similar situation is seen for the element of replicability, since this may be viewed in the two same ways as the element of prediction. In the first sense, it is not possible to replicate anything, because nothing has been done previously. In the second sense, some kind of replicability of results from, for instance, essay assignment to essay assignment could be the case during the implementation. On the overall, the first teaching module bears the potential of replicability of results, from lesson to lesson and from implementation to implementation.
Concerning the Second Teaching Module
The triangulations to be performed on the data material from the implementation of the second teaching module are not very different from those of the first, except that the data are different. Again the idea is first to use the (second module) essay assignments in combination with the third questionnaire and round of followup interviews to provide a broad answering of the research questions. And then deepen the answers by looking at the videos of the focus group, and possibly in the answering of research question 2 also including some of the collected mathematical exercises (including the historical ones).
Moreover the video of the classroom teaching may provide answers in situations like the ones discussed above for the first teaching module.
Another way to obtain insight and to deepen the answers of the research questions could be to perform triangulations on data sources from the first module and sources from the second as a way to reveal new information and as a way to further validate conclusions and interpretations. When dealing with the focus group in particular, this might prove to be beneficial. And also if case-studies of single (focus group) students
4.4 Answering the Research Questions on a Case-Specific Level 107
are to be conducted, such a ‘cutting across the data’ approach might turn out to be what is needed to obtain new knowledge about students’ meta-issue reflections etc.
Since the second teaching module is similar to the first, the second of course also bears the potential of being able to both predict and replicate results. However, concerning the actual capability of doing these things, the second teaching module is much better off than the first. Based on the analysis of the first module, predictions can be made regarding the nature of the students’ meta-issue discussions and reflections, the anchoring of these in the related in-issues, the way they work with the essay assignments in the groups (particularly the focus group), how the design of these as well as the teaching material in general functions, how the teacher and the teaching influence the work done on the essay assignments and the reflections done in connection with these, etc. Of course, there are substantial differences in the two teaching materials and the topics of the essay assignments (the variation of ‘parameters’), but since some common key-factors may be identified in both, e.g. the kind of topics and issues addressed in the essay assignments, the basis for prediction should be present. Concerning replicability, the entire implementation of the second teaching module may be seen as an attempt to replicate the findings of the implementation of the first module. Again there are differences in the material, some redesign will have occurred and other aspects of the implementations may vary, but the students and the teacher are the same.
Concerning Students’ Beliefs
The most relevant data sources for answering research question 3 are the questionnaires, especially the first and fourth, and the followup interviews for these. As explained earlier, the first questionnaire served the purpose of providing a before-picture of the students’ general ideas of what mathematics is, how it has evolved, where it is being used, and so forth. The fourth questionnaire, which was more or less identical to the first, served the purpose of providing an after-picture, and thus, making it possible to see if any changes in students’ beliefs had come about or if new beliefs had arisen. Of course, there is the possibility that some students in the fourth round may recall having been asked the same questions before. However, this is not necessarily a bad thing since it might indicate that the students have become aware of their beliefs (cf. section 3.8), which may be seen not as a change in their actual beliefs, but as a change in the way the hold these. In many cases, whether the students recall the questions or not, it is likely that they do not recall what they answered a year earlier.
Though the population of the class was rather small, as mentioned 23 students, some quantitative measurements of the students’ beliefs and potential changes in these over the one-year period, may be provided by means of the questionnaires and interviews.
Again, as is the idea with the student groups’ essay assignments, the questionnaires may be used to provide a broad answering of the research question, an answer which may be deepened and to some degree verified by (a qualitative) looking into changes for individual students (interviewees). Some of the questionnaire questions were repeated in one form or another throughout all four rounds of questionnaires and interviews.
For these questions in particular, it may be possible to follow changes in beliefs and developments of new beliefs, for the class in general as well as for individual students.
Students’ beliefs in relation to questions which were not present in all four questionnaires may on some level be investigated through the hand-in essay assignments (from both
108 Methodology
modules), and for the focus group students through the videos (from both modules).
Again, the answering of this research question may possibly be deepened by involving the videos of the classroom teaching, like the students’ understanding of the in-issues through the mathematical exercises of the modules.
Concerning the possibility of attributing any identified changes in beliefs to the teaching modules, the only way to go about this seems to be to look for elements in the students’ questionnaire answers, interview statements, and possible utterances on video which may be directly related to the two teaching modules.
This completes the treatment of the dimension of trustworthiness for now. Next are those of importance and generality.