4 Research Approach
4.2 Research design
4.2.4 Designing and conducting the interpretive study
4.2.4.2 Data collection
Data collection was performed simultaneously with the organization’s IT project prioritization process for 2011 over a period of 14 months from May 2010 until July 2011. This process involved the evaluation of ongoing projects, the initiation and monitoring of new projects, as well as the prioritization of IT project for the next year.
IT project prioritization is a core activity of project portfolio management.
In order to understand the prioritization process of IT projects and the use of BI in such a complex process, the researcher must be familiar with the context in which the study takes place. It also necessitates observation of the process as well as a range of interviews with the involved managers to capture changes over time. In this phase, I used a combination of semi-structured interviews, observation studies, recorded meetings, validation workshops and written material collected from different business units of the Danske Bank Group (described in chapter 5), as depicted in Figure 9, that were involved in the IT project prioritization process.
Figure 9: Data collection source units
Observation studies took place in two departments of Group IT in which I observed managers in their natural setting. The observation studies served primarily as a way to gain an understanding of the activities of the two departments and the context in which
project prioritization took place. I was formally introduced as a researcher who would be present and who might ask questions about projects and their prioritization. The field notes collected from these observations also served as background information for the interviews and meetings.
The richest sources of empirical data for exploring the prioritization process originated from the 22 semi-structured interviews with the managers (approximately 580 pages of transcription) and the transcripts of the eight recorded meetings (approximately 170 pages). I adopted an open approach as I was studying a largely unexplored phenomenon. The study design was not strictly defined in advance. Instead, I followed an evolutionary, iterative approach in which the activities of data collection supported each other throughout the process. For example, the meeting observations were used as input into the follow-up interviews that were conducted at a later stage.
Semi-structured interviews with stakeholders from different levels and units of the organization were conducted in order to understand the IT project prioritization process (see Table 6). These interviews provided personal views of the current prioritization process, its context and the history of the process.
Organizational Role Organizational Unit Head of IT Governance Group IT
Development Director 1 Group IT Development Director 2 Group IT Development Director 3 Group IT Development Director 4 Group IT
CIO Group IT
Senior Business Representative 1 Group Business Development &
Marketing Business Representative 1 Danske Bank DK Development Director 5 Group IT
Business Representative 2 Realkredit Denmark
Senior Business Representative 2 Group Business Development &
Marketing Head of Digital Banking Program Group IT Development Manager 1 Group IT
Head of PPMO Group IT
Portfolio Manager Group IT
Business Representative 3 Group Operations Idea Qualification Manager Group IT
Business Analyst Group IT
Followup Business Analyst Group IT
Head of IT Strategy Group IT
Business Analyst 2 Group IT
Head of Strategic Business Unit Group Business Development &
Marketing
Table 6: Participants in the first round of interviews
Two main stakeholders of the process were identified. First, Group IT, which governs the project prioritization process and implements the prioritized projects. Sixteen interviews were performed with stakeholders from Group IT at different levels of seniority. Second, the business units responsible for product-related functions and actively involved in the project prioritization process. Six representatives from different levels of the business units were interviewed.
All the interviewees were active participants in the prioritization process. The form of the interviews was semi-structured and open ended questions were asked about the prioritization process on a one-to-one basis. Each interview lasted an average of 60 minutes. An interview guide was prepared in advance (see Appendix B). Not every question was asked, however, but rather the guide served as a checklist that all the topics were touched upon, always depending on the available time. All interviewees were asked to reflect on their participation in the IT project prioritization process, their perceptions of the process, the information that was provided to them, and specifically
the BI output of the projects. They were further asked how they used the BI output in this process and what their own criteria were for prioritizing a project. They were continually probed to provide examples or to describe specific events from their own experiences.
To gain insights into how prioritization decisions were made in real settings, I also observed different meetings in which the participants were required to produce a prioritized list of projects by participating in the system steering groups of the two departments that I was following. The system steering groups (SSG) are in charge of monitoring, approving and prioritizing projects (more details in chapter 5). These groups met together four times per year (See Table 11 for a timeline of these meetings). I participated in the two last meetings (see Table 7). To distinguish between the different meetings of the two SSGs I refer to the meetings of the first SSG as “SSG meeting 1.1” and “SSG meeting 1.2” and to meetings of the second SSG I as “SSG meeting 2.1” and “SSG meeting 2.2”. For the previous two meetings of the SSGs I collected related materials including meeting minutes, presentations given at the meetings and any reading material that was sent to the participants before the meeting.
Type of meeting Nr. of participants Duration
SSG 1 SSG Meeting 1.1 13 2 hours
SSG Meeting 1.2 11 1.5 hours
SWG Meeting 1.1 8 1 hour
SWG Meeting 1.2 11 1 hour
SSG 2 SSG Meeting 2.1 11 1 hour
SSG Meeting 2.2 10 1.5 hours
SWG Meeting 2.1 13 1 hour
SWG Meeting 2.2 6 1 hour
Table 7: Meetings observations
I also observed some preparation meetings that were held by lower management staff – the system working groups (SWG) – before the meeting of the prioritization groups. In
all the meetings, I was only an observer and did not participate in any way. I introduced myself in the beginning of the meeting and stated my presence. In every meeting, I also asked for permission to record it. The prioritization groups and the prioritization meetings are thoroughly described in chapter 5 along with a thorough description of the IT project prioritization process.
The meeting observations led to a follow-up round of 13 interviews (see Table 8) and a collection of additional background material. From these 13 interviews, 4 interviews were follow-up interviews on the 4 SSG meetings that were observed. These interviews were conducted with the development manager who steers the meeting. No interview guide was prepared for the follow-up interviews. Rather, the PowerPoint presentation that was presented at the meeting was used as a guide (not included in the thesis due to the confidential nature of the presentations). In the interview with the development manager, we went through the presentation and discussed specific instances that I had noted as important during the meeting or based on the meeting transcript. I asked for explanations of technical terms that I was not familiar with, while at the same time I asked about specific projects and their BI output. For example, why one project was prioritized over another, or why one member mentioned the BI output for a specific project, etc.
The rest of the follow-up interviews concerned the IT committee meeting. The IT committee is the final approval body of the overall IT portfolio of projects for the Group IT. The committee meets four times per year, but only in their last meeting do they prioritize projects and finally approve the overall portfolio. This process is thoroughly described in chapter 5. The follow-up interviews focus on this final prioritization meeting and include the managers who were participating in the meeting, and both decision-makers and information providers such as the portfolio manager, the Head of PPMO and the SSG representatives. Table 8 shows the interviews conducted in this round.
Organizational Role Organizational Unit
Follow-up on SSG Meeting 1.1 Group IT
Follow-up on SSG Meeting 2.1 Group IT
Follow-up on SSG Meeting 2.2 Group IT
Follow-up on SSG Meeting 1.2 Group IT
Follow-up on IT committee meeting with Head of PPMO
Group IT Follow up on IT committee final prioritization
meeting with Portfolio Manager Group IT
Follow up on IT committee final prioritization
meeting with Head of PPMO Group IT
Follow up on IT committee final prioritization
meeting with CFO Group Finance
Follow up on IT committee final prioritization meeting with Senior Business Representative 1
Group Business Development &
Marketing Follow up on IT committee final prioritization
meeting with Development Director 4 Group IT
Follow up on IT committee final prioritization
meeting with Development Director 1 Group IT
Follow up on IT committee final prioritization meeting with COO
Group Business Development &
Marketing Follow up on IT committee final prioritization
meeting with Executive Committee Member Executive Committee Table 8: Participants in the follow up interview round
For the follow-up interviews with IT committee participants, I created a short interview guide (Appendix C) based on the material that was send to the IT committee members before the meeting. In this material, there was detailed information about the meeting agenda. This helped to focus the questions on the different points of the agenda and
thus to acquire rich accounts of what exactly happened in the meeting. The interviews were semi-structured in nature and the main focus was on asking for examples and narratives of how the meeting unfolded.
The additional background material included other prioritization documents, PowerPoint presentations, meeting minutes, forms, spreadsheets, reports, organization charts and memos. These documents enabled me to gain more insights into the organization’s prioritization process and to triangulate the data sources with the interviews and the meeting observations (Lee 1999). In particular, the meeting documents and presentations helped to create the interview guides and stimulated my interpretation of the themes and concepts that emerged later during data analysis.
Figure 10 presents an overview of the overall data collection process. The next section presents the data analysis process.
Figure 10: Data collection process