An exploration of the strengths and weaknesses of using text messaging as a tool for self-report data collection in psychological research
Are people able, ready and willing to become research participants using SMS?
We've established that SMS has been used for a wide range of purposes in the health and psychology literature. Though there are currently just 120 studies focussing on self-report data collection, the number of publications where SMS is used for research each year is increasing. This indicates a growing interest in the possibilities of SMS as a tool for data collection on the part of researchers. However, to be successful, a self-report research mode must be available and attractive to both researchers and their participants.
Ability to take part in research via SMS hinges largely on whether participants own a mobile telephone. If participants do not have access to a mobile telephone, it follows that a data collection mode requiring mobile telephones cannot be used. This raises questions regarding who may be expected to own mobile telephones, and whether the potential participant is capable of using SMS. As one major determinant of mobile telephone ownership and usage is age (Ling, 2008; Udine & Padova, 2002), the first paper in this section, Assumptions of age and mobile handset type, investigates mobile ownership in the general population, and what factors may help researchers estimate the likely mobile ownership in a given age group.
In cases where participants do not own a mobile phone, a researcher is faced with four options: not use SMS as a tool for their research; continue to use SMS and exclude non-mobile telephone owners from the study (e.g. Axén, Bergström, & Bodin, 2013; Devine et al., 2014); use a different data collection mode for non-mobile telephone owners (e.g. Macedo, Maher, Latimer, & McAuley, 2012); or provide a mobile telephone to participants for the purposes of participation (e.g. Reback, Ling, Shoptaw, & Rohde, 2010).
Assuming the researcher does not wish to take the first option, the second option is problematic if obtaining a representative sample is important, and non-mobile telephone owners systematically differ from mobile telephone owners. This may be the case if demographic factors influence mobile ownership, such as participant age (Ling, 2008), or geographical location (Haller, Sanci, Sawyer, Coffey, & Patton, 2006). The third option may be problematic due to cross-mode measurement variance, where responses are affected by the data collection tool in a way that systematically affects results. The fourth option, providing a mobile telephone for the purposes of participation, is a potential solution to the problem of participants not owning a mobile telephone. A researcher may want to do this to standardise the response experience across mobiles, or use an app in concert with SMS, it also solves the issue of participants not owning the necessary type of mobile telephone. Providing a mobile telephone would involve sending to the participant, or asking participants to collect a mobile telephone from the researcher, and typically then having them use the telephone in their own time, not in the researcher’s presence. Sending the mobile to participants by courier or post risks damage or lost to the handset in transit, but may allow access to otherwise unreachable participants. Meeting participants in order to provide the handset is unfeasible for large samples and could be costly if a research assistant wage is involved, but can be beneficial if used as an opportunity to train the participant in the use of the mobile device for the purposes of research participation. The second paper in this section, Should participants be given a mobile: Novelty vs Utility, examines the feasibility of providing a mobile phone to participants for the purposes of participating in self-report research, where the participant meets the researcher to collect the mobile telephone.
The first part of this section focusses on the physical infrastructure underlying SMS capability. This can be thought of as participant’s ability to participate, but it is not the whole story. Participants must also be willing to use SMS for the purposes of self-report research. The remaining papers within this section investigate how participants feel about the possibility of using SMS as a self-report data collection tool, and how this may affect participation behaviour. As with any other research mode or technology, it is possible that these feeling and behaviours will differ on the basis of demographic factors, most likely age (Ling, 2010). The first step is therefore to examine perceptions and response behaviours in general, and then to focus on pertinent demographic factors that may influence these perceptions. There is a multitude of factors and theoretical frameworks to describe participant perceptions of a research mode. Here, I focus on three meaningful factors that are particularly relevant to SMS: legitimacy, privacy, and convenience.
It is critical that participants perceive research as legitimate, and consequently be willing to engage with it (Koo & Skinner, 2005). Unfortunately the appeal of SMS as a communication tool has led to its use for less desirable purposes such as spam advertising. Many legitimate organisations are using SMS. For example, banks and financial institutions have begun to explore whether it can be used to integrate their communication with customers with their banking services (e.g. Amin & Ramayah, 2010; Nyeko, Moya, Kabaale, & Odongo, 2014). A recurrent theme within this literature exploring SMS for banking is whether the original informal nature of SMS communication, and the recent surge of using SMS for spam, diminish the perceived legitimacy of SMS usage for the purpose of banking. The same question is pertinent when considering the perceived legitimacy of SMS usage for the purpose of research. The third paper in this section, Perceived legitimacy of SMS as a psychological research mode, explores these issues.
Perceptions of privacy are associated with self-disclosure (Joinson, Reips, Buchanan & Schofield, 2010). The more private a situation is perceived, the greater the self-disclosure. If participants are asked to disclose personal information for research purposes, the perceived privacy of SMS will influence participants’ desire to take part. There is a large literature regarding perceptions of everyday SMS usage (e.g. Broaddus & Dickson-Gomez, 2013; Häkkilä & Chatfield, 2005a, 2005b), and its use as a support for clinical interventions (e.g. Curioso et al., 2009; Kunutsor et al., 2010; Rodrigues et al., 2012). There is some literature examining whether privacy could be a barrier to participation in self-report research using SMS (e.g. Ranney et al., 2014), and some research using SMS to collect data has asked participants to report on their perceptions of SMS privacy (e.g. Walsh & Brinker, 2012). However, there is no evidence to suggest that discussions regarding perceptions of privacy for particular research projects necessarily translate to perceptions of SMS as a research tool in general (or, conversely, that they do not).
The convenience of SMS is one of the major reasons for its global popularity (Leung, 2007; Liu, 2010). Communicating with participants in a way they find convenient can reduce the perceived respondent burden, and consequently may improve response behaviour (Sharp & Frankel, 1983). This general perception of SMS as convenient may translate to perceptions of how convenient it would be to respond to researchers using SMS. For example, when SMS has been used to support clinical interventions, positive perceptions of convenience are often reported (e.g. Gram, Holtermann, Bültmann, Sjøgaard, & Søgaard, 2012; Kuntsche & Labhart, 2012; Mahatanankoon & O’Sullivan, 2008). The fourth paper, Perceptions of privacy and convenience of SMS as a tool for self-report data collection, investigates these issues with a sample drawn from the general population.
Examining perceptions of SMS as a tool for self-report data collection in the general population is useful in establishing general expectations. But, there are different demographic sectors that may engage with SMS in different ways. The most noteworthy demographic characteristic that can impact on technology engagement is age. Accordingly, the fifth paper A tool for all the ages? SMS as a method for data collection with across a wide age range, an expectancy approach examines attitudes toward SMS in general, and as a data collection tool, in the context of age.
Another demographic of particular interest are the Deaf and hearing impaired. Just as in the wider population, mobile telephone ownership and SMS usage is common among the Deaf (Pilling & Barrett, 2008). Just as SMS has facilitated communication between the Deaf and hearing communities (Power & Power, 2004), its application to research with the Deaf could allow for more inclusive data collection strategies. The sixth and final paper in this section, SMS4Deaf – SMS as a mode for psychology research with the Deaf examine perceptions of SMS as a tool for data collection in the Australian Deaf community.
As mobile handsets become more sophisticated, they are capable of supporting increasingly sophisticated applications and software which can be used for mobile telephone based interventions, treatment, research and telecare. Though it is generally accepted that older individuals will own less sophisticated handsets, this brief study assesses age-associated factors relating to the type of mobile handset (cell phone, web phone, or smart phone) in more detail. Three hundred and twenty six Australian participants aged between 5 and 79 reported their age at first mobile purchase, and who initiated and made the purchase. Mobile handset type was significantly associated with both age and current everyday usage. It was concluded that current age, rather than age at time of purchase, was a simple and sufficient indicator of current phone handset type. [click here to read the full paper].
This paper was presented at the Go8-C9 PhD Forum on Population Ageing, December 2014, Sydney, Australia. Click here to read the slides.
Due to their ubiquity, mobile telephones may herald a great opportunity for ecological momentary assessment data collection. To access samples which do not own a mobile, or do not own a mobile that supports the preferred mode of response (i.e. apps), researchers may wish to provide participants with an appropriate mobile telephone for the purposes of participation. This often involves replacing a phone already in use. This study investigated the impact of providing a mobile telephone to participants for the purposes of participating in research, comparing the response behaviour of participants using their own mobile telephone against those using one provided by the researcher. Using iPhone 3s, 179 undergraduate participants completed a six-item questionnaire, 20 times over two day via app or text message. The three experimental groups consisted of those using their own iPhone, those using their own SIM in a provided iPhone, and those using a provided SIM in a provided iPhone. Results suggest that researchers seeking to conduct self-report research using mobile phones should be aware that the choice to provide a mobile telephone to standardise participant response platforms can impact on response behaviour. [click here to read the full paper]
This paper was presented at the Experimental Psychology Conference 23rd-26th April 2014, Brisbane, Australia. Click here to read the slides.
Perceived legitimacy of SMS as a psychological research mode
Participant perceptions of a data collection mode’s legitimacy is becoming increasingly important, as self-report data collection moves from interviews to methods using new technologies, where participants do not physically meet the researcher. Short Messaging Service (SMS) is a relative newcomer to self-report data collection. Understanding perceptions of the legitimacy of SMS may help to guide its application in a research context. Across two qualitative and one quantitative study (N=222), this paper developed a definition of ‘legitimacy’ specific to SMS as a self-report research mode, A communication method is legitimate if: (1) You are confident that communications come from a trusted source (2) it is appropriate for the topic being discussed (3) you feel you could use it to provide a comprehensive and accurate answer. This was applied it to gauge participant perceptions of the legitimacy of SMS as a tool for self-report research. Results indicated that participants feel the legitimacy of a research mode is important, and that the legitimacy of SMS is perceived as generally positive or neutral. However, SMS ranked poorly in legitimacy against other self-report research methods. These findings suggest researchers need to consider legitimacy in concert with other factors when considering SMS as a research methodology.
[Still trying to get this one published, so no full text for now]
Perceptions of privacy and convenience of SMS as a tool for self-report data collection
Short Messaging Service (SMS) is gaining traction as a tool for data collection in psychological research, but relatively little is known about participant perceptions of this methodology. Across three studies (total N=767), this paper provides insight into participant perceptions of convenience and privacy of SMS, with a focus on its potential as a tool for data collection in comparison with other possible research modes. Responses from members of the general population, undergraduate students, and individuals with and without experience of participating in research via SMS were compared. Overall, participants felt neutral, or positive, regarding the application of SMS as a tool for research. They had positive perceptions of SMS in terms of convenience, but were generally neutral in terms of the privacy of the mode. They were generally willing to participate in a hypothetical SMS study, but ranked SMS below email in terms of preferred research mode. This suggests that researchers should consider using SMS in appropriate study contexts.
[Still trying to get this one published, so no full text for now]
A tool for all the ages? SMS as a method for data collection with across a wide age range, an expectancy approach
Due to its ubiquity and convenience, Short Message Service (SMS) could be used as a tool for support self-report data collection in psychological research. To help researchers ascertain if SMS is suitable for particular samples, this paper explores the association between participant attitudes toward SMS for everyday purposes, and attitudes toward SMS as a tool for psychological research. These attitudes are discussed in the context of age because of the widely documented differences in technology uptake, and SMS usage, as a function of age, these attitudes were discussed in the context of age. Study 1 is a series of qualitative semi-structured interviews completed by 18 participants (aged 18-74). Study 2 is a quantitative online survey completed by fifty children (aged 5-17) and 269 adults (aged 18-79). Attitudes relating to general SMS usage were meaningful predictors of attitudes toward using SMS for the purposes of research. Increasing age was associated with lower self-efficacy, greater sensitivity to an external locus of control, and thus a less positive attitude toward using SMS in general, and specifically using SMS for psychological research. However, attitudes were not significantly associated with actual participation behaviour.
This paper was presented at the Australian Conference on Personality and Individual Differences (Melbourne, Australia, 22nd - 23rd November). Click here to read the slides.
[Still trying to get this one published, so no full text for now]
Text messaging (Short Messaging Service, SMS) is ubiquitous in Australia. It may prove a cheap and
convenient method allowing bidirectional communication between participant and psychological researcher. A
strength of applying SMS as a research tool is its inclusiveness, as it may be used to communicate with both hearing
and deaf participants. This paper explores how the Australian deaf community engages with SMS, and how this
engagement may be applied to using SMS to communicate with deaf participants in a psychological research
setting. Sixty six hearing impaired participants aged 20-89 years, ranging from moderately to profoundly deaf
took part by way of questionnaire (paper, online text, or online Auslan translation). At the end, they had the option to provide their mobile number and be sent a questionnaire via SMS. Most participants owned mobile phones, and used SMS daily. 60% believed that using SMS for research is a good idea. However, this did not translate into volunteering to participate in research using SMS – of the half who provided their mobile telephone numbers for subsequent participation, there was only a 17% response rate. Pearson's Chi-squared tests, Spearman's correlation, and logistic regression did not reveal any significant differences between those who did and did not offer their mobile telephone number in terms of mobile ownership, daily SMS usage, degree of deafness, or confidence with written English. Though many indicated willingness to participate in research via SMS by providing their mobile
numbers, a very low response rates to SMS questionnaires indicates that SMS may not be the most engaging
method for research with this sample. [click here to read the full paper]
This paper was presented at the Australian and New Zealand Communication Association Conference, July 2014, Melbourne, Australia. Click here to read the slides.
Amin, H., & Ramayah, T. (2010). SMS banking: explaining the effects of attitude, social norms and perceived security and privacy. The Electronic Journal of Information Systems in …, 41(2), 1–15.
Axén, I., Bergström, G., & Bodin, L. (2013). Using few and scattered time points for analysis of a variable course of pain can be misleading: an example using weekly text message data. The Spine Journal : Official Journal of the North American Spine Society. http://doi.org/10.1016/j.spinee.2013.08.035
Broaddus, M. R., & Dickson-Gomez, J. (2013). Text messaging for sexual communication and safety among African American young adults. Qualitative Health Research, 23(10), 1344–53. http://doi.org/10.1177/1049732313505712
Carrington, M. J., Neville, B. A., & Whitwell, G. J. (2014). Lost in translation: Exploring the ethical consumer intention–behavior gap. Journal of Business Research, 67(1), 2759-2767
Curioso, W. H., Alex Quistberg, D., Cabello, R., Gozzer, E., Garcia, P. J., Holmes, K. K., & Kurth, A. E. (2009). “It’s time for your life”: How should we remind patients to take medicines using short text messages? AMIA ... Annual Symposium Proceedings / AMIA Symposium. AMIA Symposium, 2009, 129–33. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3105604&tool=pmcentrez&rendertype=abstract
Devine, S., Ph, D., Bull, S., H, M. P., Dreisbach, S., & Shlay, J. (2014). Enhancing a Teen Pregnancy Prevention Program With Text Messaging : Engaging Minority Youth to Develop TOP Plus Text. Journal of Adolescent Health, 54(3), S78–S83. http://doi.org/10.1016/j.jadohealth.2013.12.005
Gram, B., Holtermann, A., Bültmann, U., Sjøgaard, G., & Søgaard, K. (2012). Does an exercise intervention improving aerobic capacity among construction workers also improve musculoskeletal pain, work ability, productivity, perceived physical exertion, and sick leave?: a randomized controlled trial. Journal of Occupational and Environmental Medicine / American College of Occupational and Environmental Medicine, 54(12), 1520–6. http://doi.org/10.1097/JOM.0b013e318266484a
Häkkilä, J., & Chatfield, C. (2005a). “It”s like if you opened someone else’s letter': user perceived privacy and social practices with SMS communication. Proceedings of the 7th International Conference …, 219–222.
Häkkilä, J., & Chatfield, C. (2005b). Perceived Privacy and Social Practices with SMS Communication. In 7th international conference on Human computer interaction with mobile devices & services. (pp. 219–222).
Haller, D., Sanci, L., Sawyer, S., Coffey, C., & Patton, G. (2006). R U OK 2 TXT 4 RESEARCH?--feasibility of text message communication in primary care research. Australian Family Physician, 35(3), 175–6. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/16525536
Koo, M., & Skinner, H. (2005). Challenges of internet recruitment: a case study with disappointing results. Journal of Medical Internet Research, 7(1).
Kuntsche, E., & Labhart, F. (2012). Investigating the drinking patterns of young people over the course of the evening at weekends. Drug and Alcohol Dependence, 124(3), 319–24. http://doi.org/10.1016/j.drugalcdep.2012.02.001
Kunutsor, S., Walley, J., Katabira, E., Muchuro, S., Balidawa, H., Namagala, E., & Ikoona, E. (2010). Using mobile phones to improve clinic attendance amongst an antiretroviral treatment cohort in rural Uganda: a cross-sectional and prospective study. AIDS and Behavior, 14(6), 1347–52. http://doi.org/10.1007/s10461-010-9780-2
Leung, L. (2007). Unwillingness-to-communicate and college students’ motives in SMS mobile messaging. Telematics and Informatics, 24(2), 115–129. http://doi.org/10.1016/j.tele.2006.01.002
Ling, R. (2008). Exclusion or self-isolation ?: Texting and the elderly users Intergenerational communication. The Information Society, 24(5).
Ling, R. (2010). Texting as a life phase medium. Journal of Computer-Mediated Communication, 15(2), 277–292. http://doi.org/10.1111/j.1083-6101.2010.01520.x
Liu, M. T. (2010). The influential factors of SMS usage among youth : an exploratory study to Western exchanging students in Asia. In 2010 International Conference on Networking and Digital Society (pp. 148–151).
Macedo, L. G., Maher, C. G., Latimer, J., & McAuley, J. H. (2012). Feasibility of using short message service to collect pain outcomes in a low back pain clinical trial. Spine, 37(13), 1151–5. http://doi.org/10.1097/BRS.0b013e3182422df0
Mahatanankoon, P., & O’Sullivan, P. (2008). Attitude Toward Mobile Text Messaging: An Expectancy-Based Perspective. Journal of Computer-Mediated Communication, 13(4), 973–992. http://doi.org/10.1111/j.1083-6101.2008.00427.x
Nyeko, J., Moya, M., Kabaale, E., & Odongo, J. (2014). Factors Influencing the Short Message Service (SMS) Mobile Banking Adoption : A Users’ Perspective in the West Nile Region in Uganda. European Journal of Business …, 1905, 34–45. Retrieved from http://iiste.org/Journals/index.php/EJBM/article/view/10912
Pilling, D., & Barrett, P. (2008). Text communication preferences of deaf people in the United Kingdom. Journal of Deaf Studies and Deaf Education, 13(1), 92–103. http://doi.org/10.1093/deafed/enm034
Power, M. R., & Power, D. J. (2004). Everyone here speaks TXT: deaf people using SMS in Australia and the rest of the world. Journal of Deaf Studies and Deaf Education, 9(3), 333–43. http://doi.org/10.1093/deafed/enh042
Ranney, M. L., Choo, E. K., Cunningham, R. M., Spirito, A., Thorsen, M., Mello, M. J., & Morrow, K. (2014). Acceptability, Language, and Structure of Text Message-Based Behavioral Interventions for High-Risk Adolescent Females: A Qualitative Study. The Journal of Adolescent Health : Official Publication of the Society for Adolescent Medicine, 1–8. http://doi.org/10.1016/j.jadohealth.2013.12.017
Reback, C. J., Ling, D., Shoptaw, S., & Rohde, J. (2010). Developing a Text Messaging Risk Reduction Methamphetamine-Using MSM : Research Note Intervention for. The Open AIDS Journal, 4, 116–122.
Rodrigues, R., Shet, A., Antony, J., Sidney, K., Arumugam, K., Krishnamurthy, S., … DeCosta, A. (2012). Supporting adherence to antiretroviral therapy with mobile phone reminders: results from a cohort in South India. PloS One, 7(8), e40723. http://doi.org/10.1371/journal.pone.0040723
Sharp, L. M., & Frankel, J. (1983). Respondent Burden : A Test of Some Common Assumptions. Public Opinion Quarterly, 47, 36–53.
Udine, U., & Padova, U. (2002). Young people and the mobile telephone. In E. A. Villar (Ed.), Revista de Estudios de Juventud (pp. 59–78).
Walsh, E. I., & Brinker, J. K. (2012). Evaluation of a Short Message Service diary methodology in a nonclinical, naturalistic setting. Cyberpsychology, Behavior and Social Networking, 15(11), 615–8. http://doi.org/10.1089/cyber.2012.0189
|Experimental Psychology Conference||(Brisbane, 23rd-26th April)|