Chapter 3: Method
3.12 Conclusion
This chapter detailed the mixed methods used in this study in an attempt to maximize the information about the effect of music therapy on spontaneous communicative interactions of children with CI. Data collection was carried out using three different tools: the main one being video analysis, and the two secondary ones being the parent pre- and post-intervention questionnaires and semi-structured interviews. The emphasis in the method was on the use of a flexible protocol which the therapist followed during the two conditions implemented in this study: music therapy and play, in addition to the two different parts that each session consisted of- the part that was directed by the therapist and part that was not directed by her. The general principles and procedures of the protocol undertaken during the clinical trials were described as well as the therapeutic approach of the therapist.
The results of the three procedures- video analysis, questionnaires and interviews are presented in the next chapter. In the discussion, the appropriateness as well as the limitations of the research method will be reviewed and considered.
Chapter 4 Results
Introduction
This chapter presents the quantitative and qualitative analyses of the pooled data.
First, the analysis reporting observer reliability will be presented, as prerequisites to the other analyses. This will then be followed by the results of the analysis of the dependent variables related to the subjects’ behavior from the video data of music therapy sessions and play sessions, as well as additional supplementary analyses of parent pre- and post-intervention questionnaires and parent interviews.
The results reported below come from the analyses undertaken to answer the primary research question and sub-question that were derived from the overall hypothesis for this study, which stated that following cochlear implantation, young children who receive music therapy will exhibit an increase in frequency and/or duration of specific spontaneous communicative interactions (imitation, initiation, turn-taking and synchronization).
Terms and abbreviations
In order to avoid repetition and redundancy, the following glossary of terms, (previously defined in section 3.7) and their acronyms will be re-stated here as a key to understanding their consistent usage in the presentation of the results.
DM (Directed Music experience- directed by therapist): The music experience, in which the music therapist and toddler engage in music activities predominately initiated, guided, structured and encouraged by the therapist.
UM (Undirected Music experience- not directed by therapist): The music experience in which music therapist and toddler spontaneously engage in music activities predominately initiated and structured by the toddler.
DP (Directed Play- play directed by therapist): Play activities in which music therapist and toddler engage in games or playing with toys, predominately initiated, guided, structured and encouraged by the therapist.
UP (Undirected Play- play not directed by therapist): Play activities in which music therapist and toddler spontaneously play with games and toys which are predominately initiated and structured by the toddler.
Parent-Child Early Relational Assessment (Parent-Child ERA) (Clark, 1999;
Clark et al, 1980, 1984): An assessment designed to measure the quality of affect and behavior in parent-child interactions. In this study, part of this assessment was used to measure the music therapist’s affect and behavior in therapist-child interactions
Pre- and post-intervention questionnaires: Pre-intervention questionnaire is the questionnaire that the parents completed within the two weeks prior to the beginning of the clinical trials. Post-intervention questionnaire is the questionnaire which they completed within the two weeks after the trials were over.
4.1 Results: Reliability
Results related to observer reliability will be presented as follows:
4.1.1 Inter-observer reliability of target behaviors (dependent variables) 4.1.2 Inter-observer reliability related to the independent variable session part (directed vs. undirected)
4.1.3 Analysis related to therapist's behavior
4.1.3.1 Parent-Child ERA inter-observer reliability 4.1.3.2 Parent-Child ERA analysis
4.1.1 Inter-observer reliability of target behaviors (dependent variables)
As explained in section 3.9.1, a valid measure must be reliable. Therefore, this must be tested through observational studies, before any other analyses can take place.
Inter-observer reliability actually gives a score between 0 to 1 of how much consensus there is in the ratings or scores given by the observers.
In order to evaluate inter-observer reliability, 25% of each child’s video material (a total of 19 sessions of 20 minutes each, for all subjects) was randomly selected, observed and scored by an independent observer (see section 3.9.1.1). The researcher and the independent observer, separately, scored each target behavior as a frequency or duration sum score, and Intraclass Correlation Coefficients (ICC) were then computed. Table 4.1 presents the inter-observer reliability of all dependent variables.
Table 4.1. Inter-observer reliability of dependent variables in session analysis
Variable Agreement ICC
Spontaneous imitation 0.99
Spontaneous initiation 0.94
Spontaneous turn-taking 0.96
Spontaneous synchronization 0.93
Duration of spontaneous synchronization 0.76 Duration of spontaneous turn-taking 0.92 Number of events of spontaneous turn-taking 0.91
The correlations reported in this table show that with one exception (0.76), all the ICC’s were above 0.90. The very high inter-observer reliability on six variables out of seven indicates a high level of agreement between the researcher and the independent observer for these variables. For duration of spontaneous synchronization, the ICC of 0.76 still represents a relatively high level of agreement between the two observers.
In order to assure that a behavior could be described and identified as spontaneous synchronization, it was deemed necessary during the process of analyzing the results to establish a minimum time period of two seconds (or more) of this behavior taking place, so that the observers can be sure that it could be described and identified as spontaneous synchronization. The majority of events actually did last for more than two seconds. However, seven events of spontaneous synchronizations of two seconds were counted because it was noted that even in these short events, the child clearly attempted to simultaneously match the therapist’s action, e.g., the child tried to match either the rhythm of the music in the music therapy sessions or stretched out his/her hands to hold a toy/game together with the therapist.
4.1.2 Inter-observer reliability related to independent variable session part (directed vs. undirected)
In order to find out whether the therapist had implemented the protocol guidelines in relation to session part (directed or undirected), in all four conditions of this study (DM and UM as well as DP and UP), an independent observer watched the first two minutes from each session part, for all sessions randomly presented, and documented as accurately as possible whether that part was directed or undirected (see section 3.9.1.2). Cohen’s Kappa was used to compute inter-observer reliability for these nominal variables and was found to be 0.89. Landis and Koch (1977) consider Kappa
of 0.40 to 0.59 as moderate inter-rater reliability, 0.60 to 0.79 as substantial and 0.80 or above as outstanding.
4.1.3 Analysis related to therapist's behavior 4.1.3.1 Parent-Child ERA inter-observer reliability
As explained in section 3.1, the music therapist in this study was also the researcher.
In order to increase internal validity, a procedure was administered to find out whether the therapist consistently demonstrated similar affective and behavioral characteristics in UM and DM as well as in UP and DP. Before answering this question, a reliability analysis was undertaken. Two independent observers rated the therapist’s interactions with each child, by watching the middle five minutes from each half (directed and undirected) of one session randomly chosen for each child, and rating it on 14 variables taken from the Parent-Child Early Relational Assessment (Parent-Child ERA) (Clark, 1999; Clark et al, 1985) (The procedure is detailed in section 3.9.1.3.1). As the sample in this research was small, percentage agreement reliability was computed for their ratings, and found to be 72.14 % or 94.25% (see Appendix U), obtained by dividing the number of agreements by the overall number of observations and multiplying this ration by 100. These two different percentage agreements were calculated and presented to demonstrate two different computing methods found in the literature in relation to the Parent-Child ERA41.
4.1.3.2 Parent-Child ERA analysis
To further address the above mentioned issue of bias due to the therapist’s intentions in the dual role of therapist and researcher, one of the independent observers watched three additional sessions for each child, and rated the therapist’s interactions with the child as described in section 3.9.1.3.2 (see also Appendix V). In order to evaluate the presence or absence of a difference between the ratings in the play condition and the music therapy condition, means and SD’s were computed. The results are presented
41 According to one method, which resulted in the lower percentage agreement of 72.14%, an agreement exists if the observers use the same rating point on the 5 point scale for rating a specific item. The rationale for the other method, which resulted in the higher percentage agreement of 94.25%, is that it may be difficult at times to get an exact agreement as it is not a frequency count, time sampling approach. Therefore, 1’s & 2’s rating points, which are considered as ‘areas of concern’ on the PCERA, are accepted as an agreement. Same for points 4’s & 5’s, which are considered as ‘areas of strengths’. Ratings of 2’s and 3’s or 3’s and 4’s are not considered as an agreement (Clark, 1999; Harel, 1995).
in Table 4.2.
Table 4.2. Means and standard deviations for Parent-Child ERA analysis
Mean Standard deviation
Play condition 4.75 0.55
Music therapy condition 4.68 0.69
The results comparing the play condition and the music therapy condition are very close indicating almost no difference in the interactional behavior of the therapist in play and in music therapy.
In summary, the results for observer reliability showed very high inter-observer reliability for six variables and high for a seventh variable, attesting to the high reliability of the data.
4.2 Results: Effects of music therapy
The results of the analysis of dependent variables as well as the parent questionnaires and interviews will be presented in the following order:
4.2.1 Session analysis: Main effects and interactions 4.2.1.1 Frequency of spontaneous imitation 4.2.1.2 Frequency of spontaneous initiation 4.2.1.3 Frequency of spontaneous synchronization 4.2.1.4 Frequency of spontaneous turn-taking 4.2.1.5 Duration of spontaneous synchronization 4.2.1.6 Duration of spontaneous turn-taking
4.2.1.7 Number of events of spontaneous turn-taking
4.2.2 Supplementary analyses of parent pre- and post-intervention questionnaires 4.2.2.1 Analysis of the quantitative data
4.2.2.1.1 Analysis of the Likert-type questions 4.2.2.1.2 Analysis of the ‘yes/no’ questions
4.2.2.2 Analysis of the qualitative data (free-text answers) 4.2.3 Supplementary qualitative analysis of parent interviews 4.2.3.1 Thematic analysis
4.2.3.2 Comparison of fathers’ and mothers' responses
4.2.4 Case study narratives based on free-text answers from the parent questionnaires and interview material
For the purpose of clarity, each of the above topics will be re-presented below and followed by its relevant research question as well as analysis and results.
4.2.1 Session analysis: Main effects and interactions
The analysis of target behaviors (dependent variables) was carried out in order to answer the following questions:
Primary question:
Does music therapy enhance spontaneous communicative interactions of young children, following cochlear implantation?
Sub-question:
Among these children, is the frequency and/or duration of spontaneous communicative interactions significantly greater in undirected42 or directed music and play experiences?
The independent variables were:
1) Condition: music therapy vs. play 2) Session part: directed vs. undirected 3) Interaction of condition with session part
The dependent variables were analyzed by using the complete video recordings, for all five children (75 sessions of 20 minutes each; see section 3.9.2). The analysis included frequency data of all four variables (initiation, imitation, synchronization and turn-taking), duration for synchronization and turn–taking, and number of events of turn–taking.
In order to answer the primary question and its sub-question, all the dependent variables were analyzed according to seven questions formulated and presented in section 3.9.2. Each of these questions, which examine differences in frequency and/or duration, includes six comparisons between the two conditions, music therapy and
42 The undirected and directed approaches are explained in the Method chapter, section 3.7.
play. These comparisons are presented in Table 3.6.
All dependent variables involving counts and frequencies of events were analyzed by a Generalized Linear Mixed Model (GLMM). This analysis is analogous to the more traditional repeated measures Analysis of Variance (ANOVA), but allows for greater flexibility in dealing with various forms of distribution of data. Count and frequency data typically follow a Poisson distribution (Upton & Cook, 2002). Such distributions can be modeled with GLMMs, but not with traditional ANOVAs. In this study, GLMMs were fitted using Penalized Quasi-Likelihood (R Version 2.1.0, R Development Core Team, 2005; function glmmPQL in R package MASS, Venables
& Ripley, 2002).
As a sensitivity analysis, the researcher also calculated ANOVAs of more traditional Linear Mixed-Effects Models, based on a normal distribution (function lmer in R package lme4). The use of these models would not have altered the conclusions. Therefore, only the glmmPQL models using the more appropriate Poisson distribution were retained and are presented here.
Inferential statistical analysis was undertaken on the frequency and/or duration data points collected during the video analysis for each variable, from each of the subjects (75 sessions of 20 minutes each). Due to the small number of subjects, test power was relatively low, although the use of multiple measurements in each subject may partly compensate for this. A power calculation was not carried out because the main focus of the study was on observing the individual subjects' spontaneous communicative interactions. The low test power implies that inferential statistics may yield non-significant results even if the therapy was effective. Significant results, however, are valid with the nominal significance level. Because of the low test power, particular emphasis was placed on descriptive analysis, as opposed to relying strongly on the interpretation of p-values.
All analyses and significant effects will be now presented in detail. The values presented in the text all follow the publication manual of the American Psychological Association (2001) requirements of a maximum value to three decimal points (.001), but there are several examples where a much higher significance level was reported in the analyses. The reader may refer to Appendix X for the actual p-values.
4.2.1.1 Frequency of spontaneous imitation
The first analysis reports the differences in frequency of spontaneous imitation when comparing the music therapy condition and the play condition (six different indicates that the frequency of spontaneous imitation is significantly greater in the music therapy condition than in the play condition.
There was no significant main effect for the independent variable session part, nor was there any significant interaction between condition and session part.
Figure 4.1 depicts these results in a boxplot. Boxplots can be used for the comparison of several samples, and will depict all the followed significant results of the dependent variables; therefore, a detailed explanation of a boxplot will be now presented.
The boxplot (box-whisker diagram), is a graphic representation of numerical data, without any assumptions of statistical distribution. In descriptive statistics, a boxplot represents the five-number summary, which consists of the smallest observation, lower quartile, the median value, upper quartile, and the largest observation. The x-axis represents independent variables (either music therapy vs.
play, or undirected session part vs. directed session part). The y-axis represents the scoring ranges of the dependent variables (either frequency or duration data). Every rectangular box represents the results of the first to the third quartile (i.e. 25 to 75 percentile) which is the inter-quartile range (IQR); thus, the area in the box represents the middle 50% of the scoring range. The bold line inside the box indicates the
median value. Vertical tic marks, or ‘whiskers’, are then drawn extending above and below the box to the greatest and least observed values. The whiskers extend, at most, to 1.5 times the inter-quartile range. Any values beyond the ends of the whiskers are shown individually as outliers and are indicated by the presence of open dots. In other words, an outlier is a data point which lays more than 1.5 times IQR lower than the first quartile, or 1.5 times IQR higher than the third quartile (Everitt, 1996, pp. 30-32;
Upton & Cook, 2002, p. 47).
Play Music therapy
01020304050
Condition
Frequency of Spontaneous Imitation
Figure 4.1. Frequency of spontaneous imitation by condition
This boxplot (Figure 4.1) shows that the area of music therapy is greater than that of the play condition. From the dimensions of the music therapy box, one can see that its lower border is approximately at the same height as the upper box line for play. In addition, the median value is higher and the whiskers extend further in music therapy than in play. This result suggests that music therapy yielded a much greater incidence of spontaneous imitations than play.
4.2.1.2 Frequency of spontaneous initiation
Table 4.4 presents the results of a repeated measures ANOVA for frequency of spontaneous initiation and shows a significant effect (p < .001) of the condition variable. This indicates that the frequency of spontaneous initiation is significantly greater in the music therapy condition than in the play condition.
No significant effect was found for the independent variable session part (p = .06); however, the result may have reached significance, with a larger sample.
No significant effect was found for the interaction of condition with session part.
Figure 4.2 depicts the results in a boxplot.
Table 4.4. ANOVA table for the GLMM of frequency of spontaneous initiation
Variable
Numerator Degrees of Freedom
Denominator Degrees of
Freedom
F-value
p-value
Condition 1 141 74.65 ***
Session part 1 141 3.55 0.06
Condition x Session part 1 141 1..84 0.18
***p < .001
Play Music therapy
102030405060
COndition
Frequency of Spontaneous Initiation
Figure 4.2. Frequency of spontaneous initiation by condition
Viewing the plot for spontaneous initiation (Figure 4.2), one can see that the lower border of the music therapy box is approximately at the same height as the play’s upper box line. In addition, the music therapy area box is greater, the median value is higher, and the whiskers extend further than for play. This boxplot documents the greater frequency of spontaneous initiation in the music therapy sessions than in the play sessions.
4.2.1.3 Frequency of spontaneous turn-taking
Table 4.5 presents the results of a repeated measures ANOVA on differences in frequency of spontaneous turn-taking when comparing music therapy and play.
Table 4.5. ANOVA table for the GLMM of frequency of spontaneous turn-taking
***p < .001
A significant effect (p < .001) was found for the variable condition. This result indicates that the frequency of spontaneous turn-taking is significantly greater in music therapy than in play.
A significant effect (p < .05) was also found for the variable session part. This result indicates that spontaneous turn-taking occurred more frequently in the undirected part of both the music therapy and the play sessions, than in the directed part.
No interaction effect was found for condition with session part.
Figures 4.3 and 4.4 depict these results in boxplots.
Variable
Numerator Degrees of Freedom
Denominator Degrees of
Freedom
F-value p-value
Condition 1 141 45.95 ***
Session part 1 141 6.49 0.01
Condition x Session part 1 141 2.67 0.10
Play Music therapy
02468101214
Condition
Frequency of Spontaneous Turn-taking
Figure 4.3. Frequency of spontaneous turn-taking by condition
As with the previous two communicative interactions, the boxplot for spontaneous turn-taking (Figure 4.3) shows a noticeable difference between the music therapy and the play conditions confirming that spontaneous turn-taking occurred much more frequently in the music therapy than in the play condition.
undirected directed
02468101214
Session Part
Frequency of Spontaneous Turn-taking
Figure 4.4. Frequency of spontaneous turn-taking by session part
Although the lower border and median value for the undirected and directed session parts (Figure 4.4) are essentially the same, the much larger area of the
undirected box supports the conclusion that spontaneous turn-taking occurred much more frequently during the undirected part of the session than during the directed part for both the music therapy and play conditions.
undirected box supports the conclusion that spontaneous turn-taking occurred much more frequently during the undirected part of the session than during the directed part for both the music therapy and play conditions.