Exploring the field – the pilot study

Based on the initial focus of the project – improving practice – I had created a project plan based on my understanding of the action research cycle (Baskerville 1999;

Davison et al. 2004; Van de Ven 2007) as depicted in Figure 1. My first task was to familiarize myself with how BI was used in the organization and to gather data about the needs of decision makers. Therefore, I designed a pilot study in which I interviewed managers and analysts who used BI in their work.

This pilot study included 12 interviews (see Table 5) over a period of 3 months with key personnel in the organization, and two with external subject experts, to assist in triangulation (Denzin, 1978). The entry point to the organization was through Group IT and from there other relevant interviewees and units were identified. My company supervisor, together with the development manager of the business intelligence

competency center (BICC) where I was located helped me in identifying participants who were relevant to my study.

The form of the interviews was semi-structured, based on an interview guide (see Appendix A). The interviews started with demographic and open-ended questions followed by questions focusing on the interviewees’ daily work and use of BI in decision-making. I also asked for specific examples in order to gain a better grasp of what was being said. Towards the end of the interviews, I asked for further reflections on the examples given. Each interview was conducted in English in the interviewee’s office and lasted an average of 60 minutes. Participants were informed that the interview was about the use of BI in their daily activities, but were not shown the questions ahead of time. All the interviews were recorded and transcribed (approximately 230 pages) with the consent of the interviewees. Table 5 shows the interviewees who participated in this pilot study.

Organizational role Organizational Unit

IT Business Analyst Group IT

Head of IT Credit Processes Group Credits Performance Management Specialist Group Finance Branch Performance Analyst Danske Bank DK IT Finance Business Analyst Group Finance

Regional Manager Danske Bank DK

Business Analyst Danske Bank DK

Head of Forecasting Models Group Credits

IT Finance Manager Group IT

Follow up Branch Performance Analyst Danske Bank DK External expert on BI A.P. Moller External expert on BI and portfolio

management

A.P. Moller

Table 5: Interview participants in the pilot study

Background information and organizational documents (including organization charts, reports, spreadsheets, forms, PowerPoint presentations, memos and press releases)

were also collected and served as complementary material to the interviews. The participants used BI systems, tools or the BI output in their everyday work and represented different managerial levels.

During the pilot study, I realized that when referring to BI decision makers would refer to the technologies, tools or applications or the process of accessing, retrieving and analysing data. However, when referring to the use of BI in their decision making processes it was clear that BI was used in organizational decision making neither as a process nor as a technology. Rather, it was the output of the BI process and of BI technologies that was used in decision-making. As such, BI in decision making was perceived as a product. The decision makers at the bank would refer to the BI output as the outcome of an analysis as illustrated in the following quote:

I'm not using BI tools for making decisions on what direction we should invest. In that case, we actually do get some analyses which we use in our decision-making or in what direction we should go or where we should at least focus. (IT Finance Business Analyst)

Further, I observed that it was easy for decision makers to talk how the BI output was created and the tools that were used but when asked to elaborate on the use of the BI output in decision processes they found it difficult to provide details. Further, the interviewees were more straightforward when talking about the impact of BI analysis on operational decisions than on strategic decisions. The Head of IT Credit Processes explains:

[For strategic decisions] when you see the results or data you should use common sense. So when you make decisions you should sort of respect the story. You should investigate does it make sense? (Head of IT Credit Processes)

Contrary to the view of BI as a process or a set of technologies, there were no standard templates, procedures or manuals that defined the use of BI as an output in decision

processes. Many of the interviewees would point to the selection and prioritization of IT projects as an example of strategic decisions.

At the same time, I was exploring the BI literature to get a hold of previous knowledge on BI and its relation to decision making. This initial exploration of the BI literature and its relation to decision-making revealed that there were almost no studies on how the output of BI is used in decision-making processes. Most previous work had been done on methods and technologies to collect, store and analyze data (Arnot and Pervan 2008). There was little research on how the output of BI was used in decision-making practices. In fact, there was no universally accepted definition of what BI as an output was.

As a result, there was no knowledge base, practical or theoretical, upon which I could build the design of the intervention in the organization. Reflecting on the difficulty the decision-makers had in elaborating on the use of BI in decision processes and the lack of understanding of the role of BI in such processes in the BI literature, a number of questions started to emerge. Should I make an intervention in a situation that nobody, and certainly not me, understood? How is BI as an output used by decision-makers in decision making processes? How is BI used in highly complex and ambiguous decisions such as strategic decisions?

In summary, the observations during the pilot study led me to realize that the action research approach under which the PhD project was framed was not adequate under these circumstances. After many discussions with my supervisors, colleagues at the university, two subject experts and colleagues at the company I realized that I was making too many assumptions and was taking too many things for granted.

Reflecting on the emerging questions, I realized that I should first strive to gain an understanding through an in-depth investigation of the use of BI in decision making processes. Therefore, the research goal and design changed: I was no longer interested in designing and evaluating BI applications per se, but rather in analyzing how the

output of BI is mobilized and used by decision makers within organizational decision-making processes. Consequently, my focus changed from improving practice to understanding practice.

As such, a new research design was needed for understanding practice. The new design consisted of a literature study and an interpretive study. The purpose of the literature study was to investigate the concept of BI and its use in decision-making through a systematic review in order to capture and summarize all relevant research on BI. The purpose of the interpretive study was to investigate the use of the BI output in a complex and ambiguous decision-making process, specifically the IT project prioritization process in the organization. In the next section, I present how I conducted the literature study and the interpretive study respectively.