Grounded theory and experimental methods formed the basic methodological foundation in this PhD project. However, design-based research also offered interesting and relevant approaches to the constructed research design and the implied dilemmas. The first reason is that the project addressed didactic issues. In the music course, students were encouraged to create music. The students were being instructed, introduced, interviewed, and so on. Consequently, it is neither possible nor
productive to avoid didactical issues. In other words, it is not very productive to create a course that,
40
in general, inhibits creative work. Furthermore, the project also addressed the pragmatic and learning-theoretical question ‘how to facilitate creativity’. In this regard, this thesis is inspired by pragmatic philosophers and researchers like Dewey and Pierce, who suggest that theories must be judged not only by their claim of truth, but also by their ability to do work in the world (Dewey, 1938 and Barab & Squire, 2004).
In line with the above, there exists an obvious affinity between the chosen methodology and some of the essential elements in DBR, which makes it constructive to include this approach as a supplement to the described methods. Firstly, the music course took place in an institutional context, which is the main field of interest in DBR studies. Secondly, the project balanced between different scientific traditions, which is equally a typical approach in DBR. For instance DBR is often balancing between effect research rooted in experimental methodologies, and contextually-oriented research rooted in qualitative traditions (Collins, Joseph & Bielaczyc, 2004). The same applies to the present research design, as described in the previous sections.
2.4.1 What is design-based research?
According to design-based researchers such as Cobb, Confrey, DiSessa, Lehrer & Schauble (2003), and Barab & Squire (2004), DBR consists of a variety of different types of approaches. However, some common features form the basic core in the majority of DBR studies.
Prototypically, design experiments entail both ‘engineering’ particular forms of learning and systematically studying those forms of learning within the context defined by the means of supporting them. This designed context is subject to test and revision, and the successive iterations that result play a role similar to that of systematic variation in experiment. (Cobb et al., 2003, p. 9)
Thus, DBR is in many ways connected to experimental methodology, in the sense that subjects may be tested, the independent variable is controlled by the researcher-team, and the independent
variable is systematically manipulated. On the other hand, DBR differs from experimental studies in the sense that it is conducted in naturalistic settings as opposed to laboratory settings. Hence, DBR is an attempt to include the context in the design as well as in the analyses. The basic notion is that such a methodological strategy leads to findings with a higher degree of validity, a stronger
41
relevance for praxis matters, and a pragmatic ability to change procedures in educational systems (Brown, 1992; Collins, 1992).
2.4.2 Problems with DBR
In design-based research, the researcher takes on different positions and plays different roles. Often, the researcher is simultaneously functioning as observer, teacher, designer and interventionist. The same applies to the present design. From a classic positivistic point of view this represents quite a problem. Most importantly, the question is how does one detach the empirical claims from the researcher’s interventions and how does one secure an objective examination of the produced empirical data? In the article ‘Putting a stake in the ground’ (2004) Barab & Squire address these questions.
How do we account for the role of the researcher in the design experiments and the associated threats to validity that they bring with them? If a researcher is intimately involved in the conceptualization, design, development,
implementation, and researching of a pedagogical approach, then ensuring that researchers can make credible and trustworthy assertions is a challenge.
Researchers working in schools often face difficult ethical choices. Do they stand idly by and watch a teacher struggle to use their curricula, or do they intervene, providing additional support? Do researchers share stories of struggling students with teachers and allow them to change instruction accordingly, or do they play a ‘hands-off’ role, minimizing their impact on classroom practices? (Barab & Squire, 2004, p. 10)
Apparently Barab and Squire are trying to find the balance between DBR as a local methodological tool and DBR as a way to generate theories of learning and didacticism on a general level. On the one hand Barab and Squire stress that DBR must deliver theory about learning that can be
generalized to a broader field: ‘[W]e believe that while demonstrating local consequence and utility is necessary it is not sufficient—design scientists must draw connections to theoretical assertions and claims that transcend the local context’ (Barab & Squire, 2004, p. 8). On the other hand, they question the very idea of generalization and emphasize that ‘any classroom context, even without the manipulations of a design researcher, is impacted by the systemic constraints in which it is nested, thereby making the generalizability of any naturalistic findings highly suspect’ (Barab &
42
Squire, 2004, pp. 10-11). Additionally, they stress the fact that the generalization may not be possible because of the researcher’s idiosyncratic involvement in the pedagogical setting (Barab &
Squire, 2004, p. 10).
The ‘solution’ to the diagnosed dilemma is, according to Barab and Squire, triple. Firstly, the researcher must adopt strategies derived from qualitative methodology, including GT. ‘[I]t is the responsibility of the researcher to draw on methodological practices consistent with other
qualitative methods to convince others of the trustworthiness and credibility of claims being advanced’ (Barab & Squire, 2004, p. 10). Secondly, the researcher must be aware of the fact that generalization might not be possible (Barab & Squire, 2004, p. 10). Thirdly, theory generated through the use of DBR methodology must be flexible in the sense that it may be applied to many different types of contexts:
We believe that contexts are never without agency; there are always teachers, administrators, students, and community members creating context and, therefore, local adaptability must be allowed for in the theory. The goal is not to ‘sterilize’ naturalistic contexts from all confounding variables so the generated theory is more valid and reliable. Instead, the challenge is to
develop flexibly adaptive theories that remain useful even when applied to new local contexts. This potential of flexibly adaptive theory does not result because the theory was somehow generated in a context that was free of confounding situational variables, but rather, because the theory is supple enough to maintain its robustness even in the context of changing situational variables.
Theory generated from design-based research, from this perspective, must strike a balance between refinement and adaptability. (Barab & Squire 2004, p.
11)
It is obvious that Barab and Squire wish to include the best elements from different methodological traditions (and exclude the bad ones). On the one hand they honor the qualitative, naturalistic, and local dependent approach. On the other hand they wish to verify specific hypotheses and construct theories of general relevance. In the following table the attempt to include the best elements from respectively qualitative and quantitative derived traditions is depictured.
43
Table 2.4 DBR and the attempt to combine the best of two worlds
Pros …and pros
The purpose is to generate theory in a naturalistic and local context without the artificial biases caused by a laboratory design
…and still be able to produce theory that can be applied to different contexts on a general level
The purpose is to involve all the participants in the design and analysis
…and still be able to produce a relatively consistent conclusion
The purpose is to do research in constantly changing iterative circles
…and still be able to test, explore and verify specific hypotheses
The purpose is to include multiple variables in the design and analysis
…and still be able to isolate specific causalities between specific variables
The purpose is to produce pragmatic knowledge, that might lead to improvements in a local context
…and still produce fundamental theory about how people think and learn
The question is whether or not this attempt to combine different traditions and interests is effective.
From a critical point of view, the strategy might lead to a combination of the worst of two worlds instead of the better of two worlds. For instance, attempts to generalize findings that are developed in a multiple variable setting may lead to highly questionable results. Furthermore, the constantly changing design applied in the diverse iterative cycles may problematize the attempt to verify specific hypotheses. And finally, the attempt to include all kinds of variables in the analysis and still identify specific causalities between independent and dependent variables may produce conclusions with low validity.
Given the fact that the present study is drawing on qualitative as well as experimental methodology, I will argue, naturally, that such a combination of different traditions is possible, and, in many respects, also preferable. However, I find the amount of complexity in DBR studies threatening in terms of the quantity of variables involved (see also, Nielsen, 2009). Accordingly, the attempt in the present study is to reduce the number of variables and retain a certain level of consistency
throughout the different cycles of the experiment. Naturally, this may lead to occasional inflexibility. Furthermore, my own heterogeneous position as observer, teacher, designer, and interventionist must be taken into account and addressed, for example by the use of triangulation (see Chapter Three).
44
2.4.3 Closing remarks
It the above, DBR is discussed as a possible mix between qualitative and experimental
methodologies. However, as emphasized, such a mix might result in some challenges. Nonetheless, it is important to stress that a basic contrast between partiality and generalizability is not suggested.
First, reality is always part of a situation. Thus, it is not the point to indicate that a general and underlying truth exists independently of context and situation. Next, findings produced through local and qualitative investigations might easily represent general implications. Yet, the existence of such general implications cannot just be claimed. Accordingly, the question of generalizability is addressed by the use of relevant aspects of GTM, experimental methodology, and DBR (see chapters three and eleven).