4.4 Methodological strengths and limitations
4.4.2 Limitations
This research is potentially compromised by some limitations. The most important are potential common method bias in the survey-based data, reservations about the validity of managers’ evaluations of their own firms, and the possibility of missing information in the case studies.
The same informants, namely the participant firms’ CEOs answered questions measuring both independent and dependent variables. This poses a certain threat to validity since managers might consciously or unconsciously seek to be consistent in their answers. This potential problem is considerably mitigated by the fact that the survey data is longitudinal. In statistical analysis of relationships, measures of independent variables were based on data from the first round of data collection and measures of dependent variables on the second round of data collection, one year later.
Using managers’ evaluations of their own firms is a potential weakness of the research. However, since the NTBFs studied were new and mostly unknown this was the only feasible method for collecting a comprehensive set of data for each firm. Perhaps the greatest weakness was in asking managers to evaluate their firms’ emphasis on aesthetic design. The potential pitfalls here ranged from managers’ overestimation of their firms’ aesthetic design emphasis to managers’
non-recognition of silent design. Since the evaluation of aesthetic design is central to this research, it was appropriate to seek means to validate these evaluations. This was done through expert evaluations of the design sophistication of the NTBFs’ web sites. The correlation between managers’
evaluations of aesthetic design emphasis and graphic designers’ evaluations of the design sophistication of the firms’ web sites provided a measure of reassurance that the managers’ evaluations were correct.
Other manager evaluations used in analysis deal with image, pricing and isolation mechanisms compared with the competition. Managers’ subjective evaluations of the goodness of their firms and offerings compared with their competitors’ are likely to be skewed in favor of the respondents’ firms. However,
as shown in Papers 5 and 6, the variables in question are reasonably distributed although admittedly there is poor representation in the less self-complementary ends of the scales.
Since the NTBFs studied were not publicly traded and thus are not required to make their yearly statements public it was not possible to verify the figures reported by managers. A measure of validity was insured by collecting the data at about the time of year when most small Icelandic firms hold their annual shareholder meetings and so statements for the previous year had already been prepared or were being prepared, and managers could be expected to base their answers on actual figures rather than guesswork. In fact, it was not uncommon for respondents to look up their answers in their firms’ annual statements during the interviews.
No discussion of limitations would be complete without mentioning the obvious, namely that the case studies may have overlooked important information. This is offset by the fact that the research as a whole is not based only on the case studies, but rather the case studies provide a descriptive bridge between the measurements of prevalence, on one hand, and the examinations of relationships with performance, on the other.
5 R ESULTS : SUMMARIES OF PAPERS
In a traditional monographic thesis this section would present the results of the research as a whole. Since this thesis is based on six papers, this section provides summaries of the findings reported in each of the papers. In addition, the connections between the papers and the papers’ use of the empirical data are discussed. The numbers of the papers correspond roughly to the order in which work on each paper was begun. Table 5.1 provides basic information about the papers, their publication status and presentations.
Table 5.1: Summary information about the papers included in this thesis.
N Title and Author(s) Journal publication status Presentations and publication in conference proceedings4
1 Design as an Element of Innovation:
Evaluating Design Emphasis in Technology-Based Firms (Marina Candi)
Published in The International Journal of Innovation Management Volume 10, December 2006
NFF PhD Conference in Aarhus, August 2005
Symposium of Social Science Research in Iceland in Reykjavik, October 2005 (published in proceedings)
2 Oil in water?
Explaining differences in aesthetic design emphasis in new technology-based firms (Marina Candi &
Rögnvaldur Sæmundsson)
A resubmitted version revised according to reviewer comments is under consideration for publication in an academic journal
European Marketing Academy Conference in Athens, Greece, May 2006 (published in
proceedings)
Annual Meeting of the Academy of Management in Atlanta, Georgia, August 2006 3 The Role of Design in
the Development of Technology-Based Services (Marina Candi)
Published in Design Studies Volume 28, November 2007
The Scandinavian Consortium for Organizational Research seminar, Stanford University, August 2006 Reykjavik University School of Business research seminar, March 2007
4 For all the papers, presentations listed are of various versions of the papers, in some cases under
N Title and Author(s) Journal publication status Presentations and publication in conference proceedings4
4 How Different?
Comparing the Roles of Design in Service Innovation in Nordic and American New Technology-Based Firms (Marina Candi &
Rögnvaldur Sæmundsson)
Under review for publication by an academic journal
R&D, Innovation and the Dynamics of Economies (RIDE) seminar series at the Chalmers University Department of Management & Innovation Economics, September 2007.
Accepted for presentation at the Nordic Academy of Management Conference (NFF), Bergen, August 2007.
5 The relationship between aesthetic design as an element of service innovation and competitive advantage, fact or fad? (Marina Candi & Rögnvaldur Sæmundsson)
In the review and resubmit process following review for publication by an academic journal
The Product Development &
Management Association (PDMA) Conference Research Forum in Florida, September 2007 (published in proceedings)
6 Benefits of aesthetic design as an element of new service
development (Marina Candi)
Under review for publication by an academic journal
European Marketing Academy Conference in Reykjavik,
Iceland, May 2007 (published in proceedings)
14th International Product Development Management Conference, Porto, Portugal, June 2007 (published in proceedings)
In this section I have attempted to find a reasonable compromise between adequately doing justice to the results of each paper and replicating the entire contents of each paper. In doing this I inevitably run the risk that some readers will feel that more information and more detailed coverage is needed. These readers are kindly referred to the papers that are included, each in its entirety, including references and appendices, at the end of this thesis.