Qualitative Interviews

4 Quantitative data

Online survey questionnaire among webmasters (N=541) to investigate constructs of websites success and the impacts of user testing

Descriptive analysis and Pearson correlation analysis to explore the relationships among constructs of success, and the impacts of frequency on user testing

Section 4.4.2

Table 5. Summarising the data collected and analysis performed.

Details regarding the empirical components of the thesis are provided below, in addition to issues concerning validity and reliability.

2010), and to understand explanations of website quality in this study, eight qualitative interviews with webmasters in public sector organisations were conducted between 2007 and 2009. The interviews involved eight face-to-face meetings and each interview lasted for about an hour and a half or even two hours.

All the respondents were located in or around Oslo (Norway), and therefore, the issue of time was not a subject of concern in this case. Open-ended interviews were conducted in order to establish, to a feasible extent, a situation or a conversational setting in which the webmasters could explain how, from their point of view, they would identify website quality aspects. An interview guide containing 15 open-ended questions was utilised with the purpose of guiding the conversations. Thus, the webmasters’ explanations of website quality were not restricted by using a set of pre-defined categories, thereby rendering it possible for the webmasters to be open-minded and convey meaningful explanations.

The interviews presented opportunities to clarify vague concepts and confusing issues, through personal interactions between me, as a researcher, and the respondents. Since the webmasters were able to follow the questions put to them and in fact, appeared to definitely relate to the subject matter, there arose no requirements to elaborate on the questions. The webmasters were approachable and receptive during the interviews. I strove to establish a comfortable interview setting, which further encouraged trust in sharing information and knowledge.

Confidentiality was the chief component emphasised as important by all the participants, and I emphatically guaranteed their anonymity during the interviews.

All the respondents participated voluntarily in the interviews. The webmasters’

identities were not published in the research paper (Paper 1), although the paper was firmly grounded on the findings of these qualitative interviews.

The participants were selected based upon their role and function in the organisation and I exercised complete control over the situation. The aim was to strive for a comfortable interview setting, which would enable the participants to provide in-depth and exhaustive explanations in response to the questions put forth. Before the actual commencement of the interviews, the respondents were briefed about the background and the purpose of the interviews. During the interview, if respondents were unable to follow the questions and were uncertain how to proceed, their doubts were immediately clarified. In actuality, none of the respondents expressed any concerns about the questions, although the questions were relatively open and there was room for interpretation. However, this was also the intention since website quality is an undefined term which can be approached and explained from different angles.

Once the interviews were completed, they were transcribed and sent via e-mail to the respondents. This aim was to clarify any misunderstandings or errors that had occurred during transcription. Some of the respondents did make further comments by adding a few minor and insignificant remarks. Primarily, this was not related to any errors in transcription, but rather additional information that the respondents wished to contribute to the survey. To ensure that the coding was done correctly and accurately, and to ensure thorough consistency throughout the analysis, collaboration was established in the research team. This was a safe method to ensure a common understanding of what was investigated and also the means by which the data were specifically dealt with in the analysis process. The tool utilised for this activity was the software NVivo. Making use of this tool enabled sharing of files via e-mail; in spite of two different locations in terms of the countries (one was located in Norway and one in Denmark).

In relation to the analysis of the data a grounded theory approach was applied by using “open”, “axial” and “selective” coding. Utilising a software tool proved to be extremely time efficient and beneficial for the analysis of qualitative data (Kvale, 1997). In this case, the software not only contributed to the opportunity for collaboration, but also helped to code the data, organise the data and perform various type of analysis. Other alternatives for analysis of qualitative data were Post-it notes (by using pen and paper), but Post-it notes would not present similar opportunities to collaborate during the coding process and for the analysis, since the researchers were located in different countries. Face-to-face meetings (within the research team) were also arranged to discuss the data and the resulting analysis to be performed on the qualitative data.

Conducting the interviews for this study was advantageous, in that it presented an opportunity to focus directly on the topic of interest (website quality) and helped to attain valuable insights that provided perceived causal inferences and productive explanations (Yin, 2009). An additional advantage of the qualitative data obtained by this method, during the course of this research, is related to the possibility of gaining valuable insights and exhaustive and in-depth knowledge of website quality, in the quest to understand the topic from a practitioner’s point of view. There is a lack of such knowledge in prior research studies identified, and by taking this approach; the aim was to close a perceptual gap between users and service providers (webmasters). Response bias, with regard to badly framed questions and the influence exerted by the researcher on the respondents, could, according to Yin (2009) be cited as some of the limitations of the interviews.

Further limitations could be mentioned, such as conducting interviews, which in itself is an elaborate and time-consuming process, and analysing the quantitative data, which is the subject matter of qualitative research.

In view of the sample selection of webmasters, one implication could be that the interviewed webmasters represent elite websites, namely, websites that received national awards. It would definitely have augmented the value of the study if webmasters from other backgrounds were interviewed. It would also have unquestionably enhanced the analysis, if we (the research team) had included an added round of interviews and follow-up questions, on distinct issues relevant to website quality, such as investigating the degree to which webmasters are permitted to explore their freedom to design, or whether they are merely expected to perform the job assigned to them. This could be something to explore in upcoming research contributions. In regards to the use of respondents (webmasters in award winning organisations), it is also important to be aware of that the use of evaluation criteria in these awards not necessarily reflect and measure quality in a good way. The webmasters might also focus on other quality aspects compared to the awards, which are more critical perceived from a user’s point of view. We can also speculate whether these aspects (suggested by the webmasters) are covered in the website awards. Additionally, we must be aware that each of the awards has its own focus and the evaluations (use of quality criteria and methods applied during the evaluation process) are largely affected by this.