Musings on intra-individual variability

People change over time. Sometimes, the variability in a construct can be very meaningful, too. Think about someone who is really moody - going from super happy to super depressed on a daily basis. If you measure their mood once, you might get the impression of a very happy or a very depressed person, but it doesn't reflect at all what's really going on. Oh, this is a soapbox of mine. A big stonking soapbox built on a clash between methodological and statistical idealism (if only!) and pragmatism (we can't afford to do this / repeatedly prodding participants might annoy them or change the thoughts, feelings, and behaviors you're trying to measure).

The following two papers were written based on data from my honours thesis back in 2011, with some input from my supervisor Dr. Jay Brinker. While they were cute little stubs of ideas, I lacked the resources to gather enough robust data to really do much more with it. Also, there's some rookie honours shortfalls (entirely my doing!). I see them now with horrid clarity (academic equivalent of someone being presented with photos of their scrawny and/or acne-ridden ackward teen years)... But there's fondness, too, so I'll pop them here instead of stuffing them deep in a file draw.

A Variability Focused Re-conceptualization of State Measures

Intra-individual variability can be defined as “lawful but transient within-person changes in performance, such as trial-by-trial fluctuations on a reaction-time (RT) task or day-to-day variations of cognitive performance” (MacDonald, Li, & Bäckman, 2009). This definition reflects that, historically, intra-individual variability research tended to focus on the short term, such as across reaction time tasks (as in Ram, Rabbitt, Stollery, & Nesselroade, 2005), though more attention is slowly being paid to the role of intra-individual variability in the exploration of constructs in the longer term (Hedeker & Mermelstein, 2007), and this attention has significantly enriched research in a broad range of psychological fields (Nesselroade & Ram, 2004).

Trait questionnaires are attractive for producing standard values which may be compared across participants, and even studies. Yet, they are largely based on averages, making no provision for variability. Measures of cognitive functioning that do not account for variability are less sensitive than those that do (Bunce, Tzur, Ramchurn, Gain, & Bond, 2008). This may be because, as Hedeker and Mermelstein (2007) point out, both normal and abnormal thought processes are context-specific, and develop over time. Repeated measures of states can be analysed in multilevel frameworks thereby taking into account variability (for a comprehensive overview of this, see Hedeker & Mermelstein, 2007), but treat unexplained variability as error, rather than a potential phenomenon of interest. Intra-individual variability is a stable individual difference characteristic (Allaire & Marsiske, 2005) and can be extracted from standard repeated measures data.  The 'coefficient of variability', for example, is a commonly used measure of intra-individual variability drawn from the mean and standard deviation of each individual's set of responses.

Night-to-night variability in sleep can be as predictive of negative health and psychological outcomes as insufficient nightly sleep (Mezick et al., 2009; Vallières, Ivers, Bastien, Bealieu-bonneau, & Morin, 2005). Sleep is an attractive behaviour to measure when exploring issues of variability, due to its amenability to operationalization by self-reported repeated measures, and as variability in sleep has pragmatic meaning - sleeping patterns, rather than average hours slept, are an important factor in insomnia development and maintenance (Buysse et al., 2010). Verkuil, Brosschot, and Thayer (2007) noted that trait worry questionnaires adequately predicted average daily worry, but not fluctuations in daily worry. The current study intends to ascertain whether this is also true of the relationship between the Pittsburgh Sleep Quality Index (PSQI) and nightly sleep. This will be used as an example to demonstrate how choices regarding the treatment of variability in analyses can meaningfully impact on resultant conclusions.

Method

Participants

As part of a larger study, 63 (70% female) undergraduate psychology students aged 17 – 54 (M=20) were recruited from the Australian National University.

Materials

The Pittsburgh Sleep Quality Index (PSQI; Buysse, Reynolds, Monk, Berman, & Kupfer, 1989) is a self-report questionnaire containing multiple measures of sleep quantity and quality. For the current project, respondents reported for the previous month. Ranging from 0 to 21, a higher PSQI score indicates poorer sleep quality. Internal reliability was good (α=0.71).

Participants also completed a daily sleep diary via text message, recording sleep length the previous night in hours slept per night.

Procedure

Online completion of the PSQI  was followed by daily reports of previous night sleep length, sent  in response to SMS prompts received at 7:00pm for a week, beginning on a Monday, during a teaching period.

Results

A novel missing value imputation method was utilised prior to calculation of intra-individual variability coefficient. A standard normal error distribution was constructed around the mean self-reported hours of sleep per night for each participant, from which imputed values were sampled at random. In the interests of conservative validity estimates, imputed values exceeding the maximum or minimum number of self-reported hours slept by that participant were discarded.

Sixteen percent of self-reports of hours slept per night were missing. These missing values were replaced by the imputation method described above on a by-participant basis prior to the calculation of the coefficient of variability. Significant skewness of PSQI and coefficient of variability scores were raised to the power of .05 to correct for skewness revealed by D'Agostino tests. 

There were no significant correlations between PSQI, nightly sleep averaged across participants, or coefficient of variability (Table 1). Four participants with fewer than two daily responses were removed from analysis. PSQI did not significantly impact on the nightly sleep coefficient of variability, b=.022, t(57) =.0534, p=.595, explaining a non-significant proportion of the variance coefficient of variation.

Linear modelling, with nightly sleep averaged across the repeated self-reports for each individual, indicated that PSQI did not significantly predict nightly sleep, b=4.931, t(57)=1.101, p=.275. However, multilevel modelling, with nightly sleep nested by individual, revealed that addition of PSQI as a predictor to a null model significantly improved model fit, with an increase in PSQI score associated with fewer hours slept per night at α=.001, χ2(1)=119.4, b=-8.03.

Discussion

As in Grandner, Kripke, Yoon, and Youngstedt (2006), the PSQI global score was not significantly associated with time spent sleeping nightly when participant nightly sleep was viewed as an average. However, multilevel regression, which accounts for variability, did reveal a significant relationship between PSQI and nightly sleep, highlighting the impact that different choices in how variability is treated by analyses can have on results, and consequent conclusions.

Parallel to the relationship between baseline and state measures of worry in (Verkuil et al., 2007), the PSQI was not significantly associated with sleep variability suggesting in a non-clinical sample, the PSQI alone is insufficient to capture all pertinent aspects of sleep dysfunction. In the context of sleep, this supports Verkuil et al. (2007)'s argument that repeated state measures should be taken alongside trait measures in order to fully explore the construct in question.  

This paper demonstrates that in any situation where there is a possibility that variability itself may be a meaningful object of study, repeated state measures form a basis for calculation of the coefficient of variability, or a similar measure. The recommendation to include both trait and state measures, and to explicitly address variability as a meaningful component of a construct, is of particular relevance to sleep research, given the importance of sleep variability in pathological sleep outcomes (Mezick et al., 2009; Vallières et al., 2005; Buysse et al., 2010), but may be applied in any research setting involving a cognition, behaviour, or construct that naturally fluctuates over time. Future research could explore the issue of missing values further, as the validity of current study's suggested approach to imputing values requires further exploration in an empirical setting, particularly in situations with few measurement occasions, or in studies comparing two separate groups.

References

Allaire, J. C., & Marsiske, M. (2005). Intraindividual variability may not always indicate vulnerability in elders’ cognitive performance. Psychology and aging, 20(3), 390 –401. doi:10.1037/0882-7974.2 0.3.390

Bunce, D., Tzur, M., Ramchurn, A., Gain, F., & Bond, F. W. (2008). Mental health and cognitive function in adults aged 18 to 92 years. The journals of gerontology. Series B, Psychological sciences and social sciences, 63(2), 67 –74. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/18441267

Buysse, D. J., Cheng, Y., Germain, A., Moul, D. E., Franzen, P. L., Fletcher, M., & Monk, T. H. (2010). Night-to-night sleep variability in older adults with and without chronic insomnia. Sleep Medicine, 11(1), 56–64. doi:10.1016/j.sleep.2009.02.010

Buysse, D., Reynolds, C., Monk, T., Berman, S., & Kupfer, D. (1989). Pittsburgh Sleep Quality Index (instrument). Psychiatry research, 28, 193 –213.

Grandner, M. a., Kripke, D. F., Yoon, Y., & Youngstedt, S. D. (2006). Criterion validity of the Pittsburgh Sleep Quality Index: Investigation in a non-clinical sample. Sleep and Biological Rhythms, 4(2), 129–136. doi:10.1111/j.1479-8425.2006.00207.x

Hedeker, D. & Mermelstein, R.J. (2007). Mixed-effects regression models with heterogeneous variance: analyzing ecological momentary assessment data of  smoking. In T.D. Little, J.A. Bovaird, & N.A. Card (Eds.), Modeling  Contextual Effects in Longitudinal Studies. Erlbaum: Mahwah, NJ.

MacDonald, S. W. S., Li, S.-C., & Bäckman, L. (2009). Neural underpinnings of within-person variability in cognitive functioning. Psychology and aging, 24(4), 792–808. doi:10.1037/a0017798

Mezick, E. J., Matthews, K. A., Hall, M., Kamarck, T. W., Buysse, D. J., Owens, J. F., & Reis, S. E. (2009). Intra-individual variability in sleep duration and fragmentation: Associations with stress. Psychoneuroendochrinology, 34(9), 1346–1354. doi:10.1016/j.psyneuen.2009.04.005

Nesselroade, J., & Ram, N. (2004) Studying intraindividual variability: what we have learned that will help us understand lives in context. Research in human development, 1(1), 9-29.

Ram, N., Rabbitt, P., Stollery, B., & Nesselroade, J. R. (2005). Cognitive performance inconsistency: intraindividual change and variability. Psychology and aging, 20(4), 623 –633. doi:10.1037/0882-7974.20.4.623

Vallières, A., Ivers, H., Bastien, C. H., Bealieu-bonneau, S., & Morin, C. M. (2005). Variability and predictability in sleep patterns of chronic insomniacs. Journal of sleep research, 14(4), 447–453. doi:10.1111/j.1365-2869.2005.00480.x

Verkuil, B., Brosschot, J. F., & Thayer, J. F. (2007). Capturing worry in daily life: are trait questionnaires sufficient? Behaviour research and therapy, 45(8), 1835–44. doi:10.1016/j.brat.2007.02.004

Walsh, E.I., & Brinker, J. (in press). Evaluation of an SMS diary methodology in a non-clinical, naturalistic setting. Cyberpsychology, Behavior, and Social Networking.

 

Application Of A Pragmatic Measure Of The Relationship Between Sleep Consistency And Stress

Sleep difficulties, including overall insufficient sleep and variable sleep patterns, have detrimental effects on health, ranging from daily fatigue through to death (Lund, et al., 2010; Barnett & Cooper, 2008; Taylor et al., 2011). University students are vulnerable to both sleep difficulties, and high levels of stress (Lund et al., 2010). Stress can cause poor sleep (Berset, et al., 2010), and increase sleep variability (Mezick et al., 2009). Perceived stress predicts poor sleep with more power than  sleep schedule, or exercise frequency (Lund et al., 2010).

In a correlational study, Mezick et al. (2009) used actigraphy and neurotransmitter monitoring to explore intra-individual variability in normal sleeping patterns, and found that individuals with higher stress levels were more likely to have disrupted, shorter sleep than those under low stress. The effect of stress on sleep was greater in individuals with high negative affect. The current study extended this work by ascertaining whether the Pittsburgh Sleep Quality Index, a self-report measure that defines sleep quality more broadly, would be a viable alternative to the precise, but costly and labour intensive measures of sleep disruption used by Mezick et al. (2009).

The current study also aimed to clarify the role of affect in the relationship between sleep and stress. Positive and  negative affect are state dimensions that are related, but not orthoganal (Watson et al., 1988). They are not the inverse of one another, that is an individual may simultaneously have high positive and negative affect. In an exploration of the role of emotional reactivity in insomnia, Baglioni et al. (2010) found that positive sleep outcomes were associated with high positive emotions, but not necessarily low negative emotions, emphasising that a lack of negative thought is meaningfully distinct from the presence of positive thoughts in sleep literature. Mezick et al. (2009) measured only negative affect, operationalised as a rating drawn from depression and anxiety scales. The current study proposed that such a measure does not cleanly operationalise negative affect, and neglects the possible role of positive affect. Mezick et al. (2009) proposed that cognitive arousal may explain their finding that negative affect moderates the relationship between stress and sleep, such that those high in negative affect will experience more cognitive arousal due to stress, and thus have more sleep difficulties. Cognitive arousal induced at sleep onset (Ansfield et al., 1996), or earlier in the day (Valck, Cludyts & Pirrera, 2004), is associated with sleep difficulties. However, Mezick et al. (2009)'s data is not clearly interpretable given their broad operationalisation of negative affect which includes both depression and anxiety. Both depression and anxiety are cognitively arousing, associated with insomnia (Baglioni et al., 2010), and posses significant and distinguishable relationships with sleep (Mayers et al., 2009). The current study clarified the issue by using Watson et al. (1988)'s PANAS scale, which specifically addresses negative affect, and also measures positive affect.

 

Method

Participants

Sixty-three Australian National University undergraduate students (30% male) aged 17 - 54 (M=20) participated as part of a larger study.  They received course credit for their time.

Materials and Procedure

The Pittsburgh Sleep Quality Index (PSQI; Buysse, Reynolds, Monk, Berman, & Kupfer, 1989) is a self-report questionnaire containing multiple measures of sleep amount and quality. Participants reported on their sleep experience for the past month. Ranging from 0 to 21, a higher PSQI score indicates poorer sleep quality. Internal reliability was adequate for the current sample (α=0.71).

The Perceived Stress Scale (Cohen, Karmarck, & Mermelstein, 1983) has scores ranging from 0 to 50, with higher scores indicating more frequent experiences of stress. Internal reliability was adequate (α=0.75).

The Positive Affect Negative Affect Schedule (Watson, Clark, & Tellegen, 1988) has respondents rate how well twenty positive or negative adjectives describe them. Both positive and negative affect scores range from 10 to 50 with higher scores indicated greater affect. Internal reliability was high (positive α=0.86, negative α=0.83).

Using SMS methodology developed by Walsh and Brinker (in press), participants completed a daily sleep diary, recording sleep length in hours nightly for seven consecutive nights.

 

Results

Four of the 63 cases were excluded from analysis due to data missingness. PSQI skewness was corrected by raising responses to the power of 0.5. Positive and negative affect were not significantly correlated, supporting the need to measure both independently (Table 1). Perceived stress was significantly correlated with both positive and negative affect. It was significantly  positively correlated with PSQI, and thus poorer sleep quality at baseline. Average hours slept at baseline were significantly positively correlated with the average hours slept in the duration of the study.

Table 1

Correlations, mean and standard deviation of measures

 

1

2

3

4

5

6

M

SD

1 Perceived stress

-

-

-

-

-

-

29.3

5.88

2 PSQI

0.44**

-

-

-

-

-

6.52

2.73

3 Positive affect

 -0.53 **

-0.38*

-

-

-

-

29.09

6.24

4 Negative affect

 0.48**

 0.43**

 -0.16

-

-

-

23.08

6.12

5 Hours/Night (baseline)

-0.16

-0.43**

-0.01

-0.14

-

-

7.34

1.49

6 Hours/Night (nightly)

 0.04

 -0.18

 -0.14

 0.07

0.44**

-

7.43

0.94

7 Coefficient of variability

-0.02

0.06

0.06

  0.15

0.09

-0.09

0.2

0.1

Note. n=56. *pp

Hierarchical regression revealed that perceived stress score was not significantly associated with sleep duration at baseline, however, perceived stress was associated with an increase in PSQI score, and thus poorer quality sleep. Negative affect significantly predicted PSQI score when added to the model, but did not significantly interact with stress (Table 2).

Table 2

Stress, positive and negative affect as predictors of PSQI score.

Predictor

ΔR2

B

SE

t

p

Sleep duration as DV

 

 

 

 

 

  Step 1

.049

 

 

 

 

     Stress

 

-.053

.031

-1.682

.098

  Step 2

.015

 

 

 

 

     Positive Affect

 

-.035

.038

-.928

.358

  Step 3

.002

 

 

 

 

     Negative Affect

 

-.012

.037

-.312

.756

Regression Two

 

 

 

 

 

  Step 1

.049

 

 

 

 

     Stress

 

-.053

.031

-1.682

.098

  Step 2

.003

 

 

 

 

     Negative affect

 

-.015

.037

-.416

.679

  Step 3

.014

 

 

 

 

     Positive Affect

 

-.034

.038

-.878

.384

PSQI as DV

 

 

 

 

 

  Step 1

.266

 

 

 

 

     Stress

 

.045

.010

4.46*

 

  Step 2

.029

 

 

 

 

     Positive Affect

 

-.018

.012

-1.489

.142

  Step 3

.065*

 

 

 

 

     Negative affect

 

.026

.011

2.322*

.024

Regression Two

 

 

 

 

 

  Step 1

.266

 

 

 

 

     Stress

 

.045

.010

4.46*

 

  Step 2

.055

 

 

 

 

     Negative affect

 

.024

.011

2.086*

.042

  Step 3

.000

 

 

 

 

     Stress x negative

     affect interaction

 

 

.002

-.083

.934

Regression Three

 

 

 

 

 

   Step 1

1.266

 

 

 

 

    Stress

 

.045

.010

4.46*

 

   Step 2

.055*

 

 

 

 

      Negative affect

 

.024

.011

2.086*

.042

Step 3

.039

 

 

 

 

Positive affect

 

-.021

.011

-1.804

.077

Multilevel modelling, with nightly sleep nested by individual, revealed that addition of perceived stress as a predictor to a null model significantly improved model fit α=.01, χ2=50.8, b=.001. Addition of positive or negative affect led to significantly worse model fit (χ2=-5.4, b=.014 and χ2=-4.4, b=.019 respectively), indicating that neither positive nor negative affect was not associated with nightly sleep, when taken in the context of perceived stress.

Variability in nightly sleep was calculated for each individual as suggested in Shammi et al., 1998. Linear regression revealed that perceived stress was not significantly associated with the coefficient of variation, b=-.001, t(55)=.677, p=.501.

 

Discussion

Perceived stress was not significantly correlated with average sleep duration at baseline, but was associated with shorter nightly sleep over the study period. This is likely an artefact of analyses used. Regression analysis involving baseline sleep addressed only individual averages, while multilevel analysis involving nightly sleep addressed individual averages in the context of individual error. Multilevel analyses are thus more statistically robust, providing an account of the relationship between stress and sleep duration consistent with the literature (Berset et al., 2010). Building upon Mezick et al. (2009)’s finding that stress is associated with more disturbed sleep, perceived stress was significantly associated with poorer sleep as broadly defined by the Pittsburgh Sleep Quality Index. This supports the use of the Pittsburgh Sleep Quality Index to measure the effects of perceived stress on sleep quality in future research, particularly in populations vulnerable to both stress and sleep difficulties, such as university students (Lund et al., 2010).

Contrary to Mezick et al. (2009), stress was not associated with intra-individual variability in sleeping patterns. The current study's self-report measure of nightly sleep was prone to participant recall error and bias, which was not true of the objective sleep measures, such as actigraphy data, used by Mezick et al. (2009). Before the relationship between intra-individual variability and stress is revisited, the possibility that daily self-report data may be insufficient to capture nightly sleep variability should be investigated by comparison of actigraphy and self-report data within the same sample.

Negative affect significantly contributed to prediction of sleep quality at baseline independently of stress, while positive affect did not. This emphasizes the importance of negative, rather than positive, affect in future research addressing sleep. Contrary to Mezick et al. (2009), the relationship between stress and sleep was unaffected by negative affect. As the current study operationalised negative affect without explicit inclusion of anxiety or depression, this may indicate that it was those constructs, rather than negative affect, which were impacting on the relationship between stress and sleep.

 

References

Ansfield, M. E., Wegner, D. M., & Bowser, R. (2006). Ironic effects of sleep urgency. Behav Res Ther, 34, 523–531.

Baglioni, C., Spiegelhalder, K., Lombardo,C., & Reimann, D. (2010). Sleep and emotions: A focus on insomnia. Sleep Med Rev,14, 227-238.

Barnett, K. J., & Cooper, N. J. (2008). The Effects of a Poor Night Sleep on Mood, Cognitive, Autonomic and Electrophysiological Measures. J Inegr Neurosci, 7, 405–420.

Berset, M., Elfering, A., Luthy, S., Luthi, S., & Semmer, N. (2010). Work stressors and impaired sleep: rumination as a mediator. Stress Health, 27, e71 –e82.

Buysse, D. J., Reynolds, C. F., Monk, T. H., Berman, S. R., & Kupfer, D. J. (1989). Pittsburgh Sleep Quality Index: A new instrument for psychiatric practice and research. Psychiat Res, 28, 193–213.

Cohen, S., Karmarck, T., & Mermelstein, R. (1983). Perceived Stress in a Probability Sample of the United States. J Health Soc Behav, 24, 385–386.

Lund, H. G., Reider, B. D., Whiting, A. B., & Prichard, J. R. (2010). Sleep Patterns and Predictors of Disturbed Sleep in a Large Population of College Students. J Adolescent Health, 46, 124–132.

Mayers, A.G., Grabau, E.A.S., Campbell, C., & Baldwin, D.S. (2009). Subjective sleep, depression and anxiety: inter-relationships in a non-clinical sample. Human Psychopharm, 24, 495-501.

Mezick, E. J., Matthews, K. A., Hall, M., et al. (2009). Intra-individual variability in sleep duration and fragmentation : Associations with stress. Sleep Med, 34, 1346–1354.

Taylor, D. J., Gardner, C. E., Bramoweth, A. D., et al. (2011) Insomnia and mental health in college students. Behav Sleep Med, 9, 107–16.

Valck, E.D., Cluydts, R., & Pirrera, S. (2004). Effect of cognitive arousal on sleep latency, somatic and cortical arousal following partial sleep deprivation. J Sleep Res,13, 295 -304.

Walsh, E.I., & Brinker, J. (in press).Evaluation of an SMS diary methodology in a non-clinical, naturalistic setting. Cyberpsyc Behav Soc Netw.

Watson, D., Clark, L., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol, 54, 1063–70.