Design Based Research (DBR)

Ann Brown coined the concept ‗design experiments‘ (Barab & Squire, 2004). She found that laboratory settings were not sufficient to explain classroom learning. Owing to the limitations of laboratory experimentation, she envisioned the need to develop an approach for better understanding learning in a classroom setting (Brown, 1992). Instead of controlled experiments, Brown envisioned the classroom as a natural lab. Thus the idea of conducting a classroom experiment by intervention design arose. During the same year, Collin (Collin, 1992) defined the term ‗design science‘. He defined design science as the science of

‗designing the artefacts and studying the behaviour of these artefacts under the different conditions‘. In his view, design science could be implemented in educational research. Collin suggested that there were similarities between the two fields. He suggested that, instead of engineering artefacts, the ‗learning environment can be designed and tested to investigate its

31 effect on teaching and learning‘. This insight is in very close alignment with the research objectives mentioned earlier.

Since, the pioneering work of Brown (1992) and Collins (1992), design experiments as a methodology have steadily increased in educational research. Examples of experiments can be found in the Handbook of Research on Math and Science Education (Kelly & Lesh, 2000).Although ‗design experiments‘ and ‗design science‘ have historically had similar meanings, the method has been commonly referred to as design based research (DBR) in the recent literature (Sandoval & Bell, 2004).

3.3.1 Definition of DBR

DBR is considered to be an emerging field in the landscape of education research (DBRC, 2003). Its ability to ground the research in practice makes DBR a promising methodology for educational interventions. DBR is used as a method for understanding learning in the complex environment or for designing a new learning environment to improve the learning of the participants involved.

Shavelson et al. (2003) described DBR as follows:

―DBR is a research which strongly relies on a prior research, and is seldom carried out in an educational setting. It seeks to trace the evolution of learning in complex and messy settings like classrooms and schools. DBR tests and build theories of teaching-learning. It produces instructional tools that survive the challenge of everyday practice‖ (p-25)

This definition provides deeper insight and interconnection between three important goals of DBR as a research, development of theory and improvement in pedagogical practice. One of the most cited definition of DBR (Cobb et al 2003) in the literature is as follows

―Design experiments entail both ―engineering‖ particular forms of learning and systematically studying those forms of learning within the context defined by the means of supporting them. This designed context is subject to test and revision, and the successive iterations that result play a role similar to that of systematic variation in experiment‖ ( p- 2 ).

The above definition addresses three aspects of DBR: design, test and revision of design.

The definition indicates that DBR provides the required characteristics for designing educational interventions and test them in a real life context. Furthermore, the newly designed interventions could be revised in successive iterations. Wang &Hannafin (2005) similarly defined DBR as,

―A systematic but flexible methodology aimed to improve educational practice through iterative analysis, design, development and implementation, based on collaboration among researchers, and practitioners in a real world setting, and leading to contextually sensitive design principles and theories‖ (p. 5)

This definition outlines the suitability of the DBR method in addressing issues of context by designing an intervention to improve practice. Thus, intervention design requires an

32 understanding of the context in which it is to be implemented and collaboration with practitioners. Salomon (1993) asserted that the learning environment could be designed to initiate cognition and to improve learning. In this sense, DBR complements the cognitive process. Barab & Squire, (2004) stated that DBR is based on the fact that individuals and learning environments are inseparable. In other words, there exists a close relation between the learner and learning environment. The learning environment could be designed to enable an effective learning process.

3.3.2 Characteristics of DBR

The previous discussion identified DBR as a useful methodology for the design of educational interventions and for studying its effect on teaching and learning. DBR illustrates how elements (or issues) of the learning environment can be suitably modified to improve students‘ learning (Sandoval & Bell, 2004). Hence, the intention of DBR is to bring improvements in educational practice. In this sense, DBR can be suitable for addressing the issues in the Indian education system discussed in the first chapter.

The purpose of the design experimentation is to explain both the process of learning and how designed artefact supports this process of learning (Cobb et al., 2003). Such explanation would elaborate successive patterns of student‘s learning. The DBR researcher must explain why and how these patterns are generated and must collect the data for proper explanation of these patterns. This kind of reflective process has potential to develop the underlying theory behind the patterns. The purpose of the design experiments is to carry out formative research to test and refine educational designs (Collins, Joseph and Bielaczyc, 2004). Hence DBR is useful methodology for conducting the formative research and development of improved practice that is in line with my second and third requirement for suitable methodology.

Many authors (Wang & Hannafin, 2005, Barab & Squire, 2004, DBRC, 2003) have described the characteristics of DBR. The characteristics have been outlined as follows:

1. Design experiments are carried out in a natural context.

2. Design experiments are pragmatic and involve multiple methods of data collection.

3. Design Experiments promotes innovations in education.

4. Since the design experiments are conducted in a complex setting, it is not possible for a single design to address all issues. To achieve a more robust design, experiments must be progressively refined. Hence, design experiments are iterative in nature.

5. DBR is characterised by multiple variables. The focus of DBR is to identify the variables so as to understand how they affect the learning process and characterise the learning context. This is unlike lab experiments where variables are controlled.

6. DBR often leads to development of a theory.

Collins (1999) outlined many differences between psychology experiments and DBR including location of the research, number of variables, procedures, and amount of social interactions, nature of research and role of participants in the research. These differences are used to reinforce the choice of DBR for this research. Psychological experimentation is conducted in laboratory settings, whereas DBR is conducted in real life settings. In this research, a classroom, and fieldwork provide real life settings. Psychological experimentation frequently involves one or two dependent variables whereas DBR is characterised by multiple variables. In the current research, content learning, learning patterns, and experiences are a few of the identified outcome variables. Also, system variables such as institutional academic and administrative culture influence the design. Psychological experimentation focuses on

33 identifying a few variables and holding them constant. However, the focus of the current research is on characterising the PBL model and it‘s utility to address the issues of the Indian institute.

In psychology experiments, fixed procedures are typically followed. DBR is characterised by flexible design revisions in which a tentative initial set is revised depending on its success in practice. In this research, three designs were prepared. The first design was modified to prepare the second design, and the third design was modified based on the outcomes of the first and second models. These systematic revisions of the design are carried out based on feedback from the students, the research data and self-reflection. In psychology experiments, the learner is isolated from social environment to control the interaction. DBR, however, frequently involves complex social interactions in which participants share ideas, distract each other etc. In this study, 375 students participated in the group work. The group work comprised a social system in which many students of varied intellectual, cultural and academic backgrounds studied and worked together on the project. In this research, these social interactions are not controlled. On the contrary, the goal is to explore them.

DBR involves looking at multiple aspects of a design and developing the profile that characterise the design in practice. Undoubtedly, the focus of this research is to develop the design by considering multiple aspects of current academic practice. However, psychology experiments focus on testing a hypothesis. A psychology experiment tends to treat participants as subjects. In DBR, participants play an active role in helping to improve the design through feedback. In this research, a researcher, three teachers and 375 students participated. They provided useful insight and feedback to improve designs.

3.3.3 Summing up

The previous section discussed a few definitions of DBR and the characteristics and differences between DBR and psychology experiments. Considering the research objectives outlined in this chapter, DBR appears to be a suitable methodology to conduct this research.

In the following paragraph, the basis for selecting DBR is discussed. Firstly, I selected DBR because of my design engineering background. During my preliminary reading on DBR, I noticed the similarities between design engineering and DBR. This background helped me to understand DBR and to generate the self-confidence to follow this methodology. However, this was one of the less significant factors in choosing DBR as a methodological framework.

The focus of my research is to design a PBL intervention, and to research its effectiveness in giving students an authentic learning experience. DBR fits these requirements, as discussed at the start of this chapter. Also, the characteristics of DBR are in close alignment with my research objectives. Although DBR appears to be the appropriate method for this research, it brings many challenges that must be considered in the design and development stage of this study. In the following section, these challenges are noted.

3.3.4 Challenges in DBR

3.3.4.1 Interventionist process

In Design experiments, a researcher (or team of researchers) must deal with multiple variables that can affect the learning process. These multiple variables are difficult to control in complex situations like a classroom and can affect design enactment. To make the design work in a complex situation, the researcher must adapt as per the situation. To adapt changes in variables, the researcher needs to make change in the planned design experiment which is

34 counter to traditional methods of the scientific planned experiment. This raises a methodological issue (Sandoval & Bell, 2004).

3.3.4.2 Design and comparing across designs

Design is considered to be a creative but demanding process. Every design must be considered from multiple perspectives. Designing even a single experiment is a challenging task. Design experiments are often criticised for being context dependent. One design may not work as well when transferred to another context. For example Aalborg PBL model may not work if it transferred as it is to Indian context. Appropriate change in the design must be done to be effective in Indian context. Designs are context dependent and comparing them is natural in DBR. Thus, the researcher has to be sure to make the research feasible and applicable in similar contexts, which provides a challenge. Also, the same design may not work in the given context; because variables may change. Hence, the design has to go through successive iterations to improve the learning experience.

In the current research, efforts have been made to design an Indian version of the PBL model. As a result, the designs are different from Aalborg‘s PBL model in terms of organisation and implementation point of view. These designs have been adjusted to address context dependent issues. The designs used in this research are unique in its characteristics and modified in each successive semester. The designs were modified according to the learning requirements of the students. This kind of flexibility in research is characteristic of DBR.

3.3.4.3 Large amount of the data

Since DBR is carried out in a practical context, researchers usually prefer to use a mixture of qualitative and quantitative data as evidence for and against the design. Such data is useful for judging the effectiveness of the design and for improving the design. This process tends to produce a large amount and variety of data to be analysed, which is a challenging task. In the context of the current research, I have dealt with a large amount of data during the collection and analysis stages.

3.3.4.4 Role of the researcher and team

Due to its characteristics, DBR is usually carried out by a research team comprising of a designer, researcher and practitioner. These team members have different roles to play depending on the stage of the research. Designing the single experiment requires design, research, and analysis skills. It is always beneficial to have a range of expertise on the team in order to build workable designs and to develop alternative interpretations of the data and results. Finding a suitable team for conducting DBR is a challenge. In the context of the current research, all roles (designer, researcher and practitioner) were held by me, which put me under considerable stress. Detailed discussion of these roles and their management is discussed in the coming chapters.

3.3.4.5 Role of technology for collecting data

In DBR, the researcher has to collect the data in support of the design. This often requires a variety of instruments, which leads to challenge in integrating technology within the construction of the design. The available technology (e.g. video cameras, audio-recording systems, and mass electronic storage devices), technological support (software and technical experts) and possible integration challenges (space and use of devices in a class or system)

35 must be thought of in the design process itself. It is also expected that the DBR team be a reasonably competent in operating these instruments. The generation of multiple forms of data also creates challenges in managing and analysing the large quantities of data.

In this research, the handling of video cameras and audio-video devices to collect data was done predominantly by me. For the transcription and storage of data web services, an available technology is used. Microsoft office Word and Excel programmes were used for data analysis. Most of the time, I was involved in the collection and processing ofdata. More detailed discussion on data collection and analysis is done in the later part of this chapter.