Reading: Planning for a mobile future: a user research case study from the California Digital Library

Download

A- A+
Alt. Display

Articles

Planning for a mobile future: a user research case study from the California Digital Library

Authors:

Rachael Hu ,

California Digital Library
About Rachael

User Experience Design Manager

X close

Alison Meier

California Digital Library
About Alison

Mobile Intern
(Summer 2010) 

X close

Abstract

In the summer of 2010, the California Digital Library (CDL) undertook a mobile device user research project to understand how the proliferation of mobile devices with internet access amongst the general public and the explosion of mobile tools and products in higher education affected CDL constituents and services. CDL wanted to understand if they would need to support users in a mobile capacity.

In order to answer these objectives, findings from online surveys as well as in-person interviews were analyzed. General findings revealed that while mobile devices played a large role in con- stituents’ personal lives, it was not a device that they turned to when taking part in their academic work. This conclusion lead CDL to develop a mobile strategy that involved building a mobile tracking and evaluation infrastructure to ensure continued assessment and monitoring of mobile usage and trends among their users.

How to Cite: Hu, R. and Meier, A., 2011. Planning for a mobile future: a user research case study from the California Digital Library. Serials, 24, pp.S17–S25. DOI: http://doi.org/10.1629/24S17
  Published on 09 Nov 2011
 Accepted on 05 Sep 2011            Submitted on 18 Jul 2011

Introduction

The California Digital Library (CDL) serves the ten campus libraries of the University of California (UC). CDL licenses and builds digital tools, services, and collections that support UC academic users in their scholarly pursuits. In this case study, academic users are defined as students, researchers and faculty.

In the summer of 2010, CDL undertook a mobile device user research project to understand how the proliferation of these devices as well as mobile tools and products in higher education (HE) and libraries affected CDL constituents and services. The main objectives were to understand if CDL needed to support users in a mobile capacity and if there were opportunities to develop new ways to meet user needs. Based on findings from this inquiry, specific and general strategic recommendations were developed in order to guide CDL in supporting and developing mobile access to its services.

Methodology

The first step in this study involved a literature review. An initial survey was distributed via social media outlets and received a small sampling of 27 responses from librarians. A second survey, distributed to a random sampling of the academic population at UC Berkeley, resulted in 286 responses. Survey respondents held a range of different academic occupations that included faculty, graduate students and undergraduate students. The quantitative results in this article are based primarily on the findings of this second academic survey. All percentages are calculated based on the number of respondents to that particular question.

Finally, 14 interviews, made up of both information professionals as well as academic users, were conducted to gather qualitative data to better understand user motivations and workflow processes. Due to project timeline and budget constraints, only a limited number of UC campuses were a part of the survey and interview process. Future work would ideally capture data from all UC campuses and constituent groups.

Literature review

The goal of the literature review was to establish a baseline understanding of general user behavior and preferences surrounding mobile devices. Another important goal of the review was to learn how HE institutions and libraries were supporting mobile development.

General adoption and use

Mobile phones have become the most prevalent tool for media consumption and communication worldwide, with 4.6 billion mobile phone subscriptions in use at the time of the study. Compared to the 1.2 billion personal computers in use (including laptops), that number is staggering.1 Eighty-three percent of American adults own a mobile phone, and this number is even higher for American adults aged 18–29 (93%).2

Internet on mobile devices

Smartphones and other mobile devices with internet access allow people to access the web on the go. Some estimate that more users will access the internet from mobile devices than desktop computers within a few years.3 Pew Research Center finds that almost a third of American adults have used mobile devices to access the internet.4

EDUCAUSE surveyed undergraduate students regarding their use of the internet using mobile devices. They found that half of the student respondents own a mobile device that can access the internet, but only a third of the students actually access the internet from this device.5 The report cites easy access to the internet through other means, the expense of data plans and lack of desire to use the internet as the key reasons for the difference between ownership of capable devices and actual use. At the time of this study, there was not much research that focused on academic populations besides undergraduate students, such as graduate students and faculty.

Pew Research found that for people aged 18–29, access to information ‘on the go’ is a more important aspect of mobile technology (60%) than staying in touch with others (57%).6 One study completed in 2008 limited subjects' internet use to mobile phones. Although mobile internet has improved since 2008, their findings are still relevant. The study authors report that their subjects often found using the internet from a mobile device frustrating because the experience is different than using the internet on a full computer. They likened desktop internet use to scuba diving, where search can be ‘immersive’ and ‘invites exploration and discovery’. In contrast, mobile internet use is closer to snorkeling, where ‘shallow dipping in and dipping out of content for quick checking of key content is desired’.7

Mobile services in higher education

In addition to creating mobile content and applications, some institutions are taking advantage of text messaging services. One out of seven universities reports using text messaging to communicate with students, particularly for emergency notification.8 Higher education survey respondents in the UK ‘were more positive about accessing information through text messages than through the mobile internet’.9 Latimer finds emergency text messaging services problematic, however, because text messaging ‘suffers from several disadvantages including inherent design problems, the opt-in process, character limits, and vulnerability to abuse’.10 Latimer also notes that universities have difficulty maintaining up-to-date contact information because cell phone numbers change frequently.

Mobile services in libraries

Agrowing number of university and public libraries are offering mobile services. Libraries are creating mobile versions of library websites, using text messaging to communicate with patrons, developing mobile catalog search, providing access to resources, and creating new tools and services particularly for mobile devices.

Many recognize that mobile devices are playing a large role in people's lives, particularly for information finding. However, there are some who are less confident in the success of the wave of library mobile services. Bruce Washburn of the OCLC is pragmatic about the usefulness of mobile library applications:

“There is certainly much enthusiasm and interest in the library community about mobilizing library resources … But … there isn't corresponding evidence of widespread adoption and use of the results of those development efforts. At best, these applications may be partially displacing use that would have otherwise occurred in a desktop application. At worst, they may be either frustrating users by not delivering fully on the expected promise, or being otherwise missed or ignored.”11

Washburn believes that it is important to hold mobile library applications to usage metrics in order to judge success. Metrics should measure increased usage, either by the same people with more frequency or by more people overall.

Mobile devices and academic workflow

One area that has potential for academic use is reading on mobile devices. Although smartphones have become common devices for reading personal content, there seems to be less buzz around reading academic content on mobile devices. One reason for this difference is that reading for academic purposes is significantly different than reading for personal reasons. Reading is one component of an academic cycle of work that involves research, reading, analyzing and creating. Because reading is only one part of this cycle, tasks such as taking notes, comparing documents and saving citations are particularly important.

A pilot program at Princeton University reported that students found ‘the classroom experience was somewhat worsened by using e-readers, as study and reference habits of a lifetime were challenged by device limitations’.12 One of the primary complaints that these students had was the inability to take notes. The Kindle does allow users to take notes and make highlights that are then stored online, but students soon learned that they could only highlight up to 10% of the entire book, which was insufficient for their needs.

CDL mobile inquiry findings

The following findings reflect the snapshot of behavior and preferences that were found during the inquiry conducted at a small sampling of University of California campuses during this project.

Electronic equipment and mobile device ownership

First, the survey establishes the context of how ownership of these devices stacked up against ownership of other electronic equipment. As shown in Figure 1, laptops are clearly the most ubiquitous electronic device owned by academic survey respondents. Over 98% of survey respondents owned a laptop or netbook. After laptops, portable media devices such as MP3 players were most likely to be owned (78% overall).

Figure 1 

Ownership of electronic equipment, by occupation

Slightly more academic respondents own mobile phones without internet (61%) than mobile devices with internet (53%). In this survey, mobile devices with internet refer to a category of small mobile devices that does not include tablets or e-readers. Survey respondents overall are already well ahead of the general American population in terms of mobile device with internet ownership (23% of American mobile consumers have smartphones).13

Of academic survey respondents who own mobile devices with internet, the majority own iPhone (53%) or iPod Touch (20%) devices. The next highest device was BlackBerry (10%), and then Android (9%). Windows Mobile trails with (4%). (See Figure 2.) (Note: ‘mobile devices with internet’ is meant to indicate small mobile devices and not larger tablet devices.)

Figure 2 

Small mobile device with internet ownership, by occupation

Conclusions on electronic equipment and mobile device ownership:

  • although many own mobile devices that are capable of accessing the internet, a good portion of respondents do not
  • although the iOS platform (including iPhone, iPod Touch, and iPad) has the highest user group, other smartphone platforms – BlackBerry and Android in particular – are in use and should be considered when testing and supporting platforms.

Internet access and data plans

Some who own internet-capable devices do not actually subscribe to data plans that would allow them to access the internet. Some respondents, particularly undergraduate students, may share phone plans with family members and not have much flexibility in choosing their feature options. Of survey respondents who do own an internetcapable mobile device, 23% do not have data plans. Most cite cost as the primary deterrent to subscribing to a data plan. This finding is similar to the EDUCAUSE report that a third of students with mobile devices that support internet do not actually use the internet from their devices.14

Cellular vs.WiFi

Of those who do subscribe to data plans, 78% of survey respondents reported using the cellular network frequently to access the internet, whereas only 49% report using WiFi frequently.

Academics using cellular connections instead of campus wireless networks may encounter significant barriers when accessing licensed resources due to campus authentication requirements. Some struggle with configuring access to library resources even when on the campus wireless network. A staff clinician at UCSF who is a heavy user of mobile devices told us: “I do sometimes use the library for journals [on my mobile device], but I haven't gotten around the VPN issue to actually be able to pull up the articles.”

Conclusions on cellular vs. WiFi:

  • not everyone who owns internet-capable devices actually subscribes to data plans. Cost is a significant deterrent
  • many use cellular networks to connect to the internet rather than WiFi, which complicates accessing authenticated library resources
  • services that do not require authentication may have more use than those that do
  • since many respondents prefer using cellular networks, ways to connect to authenticated resources via cellular connections should be explored. This would be a complicated task because of various third party and vendor requirements and campus library infrastructures.

Activities on mobile devices with internet

Mobile devices with internet access are used for a variety of purposes. As shown in Figure 3, some of the most common uses include finding information and accessing e-mail. They are also used for text messaging, watching videos and listening to music, reading, playing games and social networking. They are used less for academic purposes, such as accessing campus or library websites or completing coursework.

Figure 3 

Online activities on mobile devices with internet

Finding information

Mobile users are likely to seek quick hits of information, especially when there is a pressing need in the moment, such as getting directions. Of survey respondents, 35% reported searching for information several times a day. For example, one faculty interviewee told us how he will access a research database (that is not mobile friendly) when he needs to look something up right away. He also noted that he would use his mobile phone for short tasks: “I would use this for very quick things, like looking in the dictionary.”

Library resources for research on mobile devices

Most interviewees noted that they did not want to conduct actual academic research on mobile devices. Many see research as a difficult activity that would only be more difficult on a mobile device. As one undergraduate student noted: “It's a little complicated to [access library resources] on the computer so I haven't tried on the iPod Touch.”

Despite this disinclination to do heavy research on mobile devices, there does seem to be some interest in having the option to access library databases, catalogs and resources from mobile devices. About 53% of survey respondents said that they would like to search library databases from mobile devices. About 55% wanted to search the library catalog from a mobile device. Instead of using these tools to perform actual research, it is more likely that users will use library databases to retrieve known materials or find quick information. One undergraduate student told us that she would only seek out library material that she already knew was available: “I probably wouldn't look for something new on [my mobile device] just because it's kind of a pain. But if there was something that I knew was already there and was just looking for it, I might do that.”

E-mail and text messaging

E-mail is a commonly used tool on mobile devices, with many survey respondents who own mobile devices with internet reporting that they access e-mail from a mobile device several times a day (56%). Text messaging is also popular from mobile devices with internet; 50% of mobile device owners with internet report using their device to send text message several times a day.

Notification

Next, the CDL wanted to learn if their constituents preferred either of these two communication avenues for notification from library resources or services. Most of the interviewees preferred notifications by e-mail instead of text message. Some mentioned that text messages demand immediate attention, and they didn't find most notifications (particularly library notifications) to be that urgent. One person noted that e-mail is easier to organize: “I like e-mail because it's easier to store e-mails and look at them later.” Another stated that she takes e-mail more seriously than text messages.

Many interviewees indicated a strong inclination to maintain text messages as a channel for personal communication rather than academic or professional. These findings were echoed in survey responses to the question “How do you generally like to receive information?” E-mail ranked quite highly for both communication with friends and notifications (see Figure 4). On the other hand, text message ranked highly as a way to communicate with friends but much lower as a way to receive notifications (see Figure 5).

Figure 4 

Receiving communication vs notifications by e-mail

Figure 5 

Receiving communication vs notifications by text message

However, text messaging should not be ruled out categorically as a notification method. Some interviewees did note an interest in receiving notification by text message, particularly from those who do not use or do not have e-mail on mobile devices. Also, about 25% of survey respondents noted that in general, they do like notification by text message.

Organization and transfer of Information

While e-mail is traditionally considered a way to share information and files between different people, it has also become a way to transfer information and files between devices. As one interviewee noted: “If it's in an e-mail, it's in my inbox, in my device. It's automatically everywhere.”

In addition, both e-mail and text messages are used to save notes or send information to oneself. One student explained that she sends herself both text messages and e-mails, depending on which device she is using to send the information: “If it's something that I might need to access while I'm on the go, or something I want to have with me, I'll save it in a text message to myself. If it's something I want to access at home, then I'll e-mail it to myself.” E-mail is the more common way to send oneself information; 54% of survey respondents reported sending e-mails to themselves frequently, while only 15% said the same for text messaging.

Conclusions on organization and transfer of information:

  • people often prefer e-mails to text messages because they can be stored, organized and do not demand immediate attention. On the other hand, some people who do not have e-mail on their phones would like notification by text message.
  • currently, there are not clear systems for sharing notes and files between devices, so people have adapted e-mail and text messages to do this. (Though this may change with more cloud-based online personal content storage options that are becoming available)
  • keep e-mail notification as the standard method of notification for systems like interlibrary loan. Consider creating an optional text message notification option. Any text message notification should be an opt-in feature
  • make it easy for people to use e-mail to transfer research and resources to and from mobile devices.

Reading: mobile devices with internet

About 30% of mobile internet-using survey respondents report reading content such as news articles or blogs on their mobile devices. Fewer report reading academic content. Some noted that they prefer to read PDFs on their laptops, while others stated a preference for reading material on paper. One graduate student told us: “I can't imagine reading a whole science journal on my iPod Touch.” Another told us that she doesn't like reading on her phone for longer than ten minutes.

Another significant issue for interviewees was the ability to organize their readings and citations. Many had complex organization systems on their laptops, such as saving PDFs in folders, saving URLs as bookmarks, or saving citations in EndNote. A staff clinician noted: “I like to have my laptop with me because I have all of my stuff organized in a certain way there. And if there were a way that I could do it on my mobile device and then it would end up on my laptop eventually, I would do that. But that's really my big issue … If it's something that's pertinent to my work, I'd want to be able to go and get it and then save it on my computer so that I can reference it later.” These tasks would be much more complicated from a mobile device because of the lack of citation tools and file organization available on mobile devices.

Interviewees often expressed hesitation with reading on their mobile devices because of the difficulty of opening various file types. One PhD student noted that her phone's memory is too small to hold large files and it doesn't read PDFs well. Another student wasn't sure if her device could open PDFs: “When I'm using PubMed I'm looking for a PDF of the article, and I don't think … can you open PDFs on [iPod Touch]? I don't know actually.”

It is interesting that so many interviewees were opposed to or uncertain about using PDFs on their mobile devices because PDFs are clearly the medium of choice for reading academic content. Out of survey respondents who reported using a mobile device for academic reading, most reported reading from downloaded PDFs (74%), with 47% reading academic content from websites.

Conclusions on reading on mobile devices with internet:

  • laptops are often the right tool for the job in hand, but mobile devices fill in when computers or internet are not available
  • mobile devices are more likely to be used for personal uses than academic ones
  • many do not want to read academic content on mobile devices because of note-taking preferences and difficulty with file formats
  • citation tools and organization workflows are built around laptops, and it is difficult to integrate mobile devices into these systems
  • people do not want to do in-depth research from mobile devices, but they do want access to resources
  • keep in mind that high ownership of mobile devices with internet and high usage of personal apps or mobile sites may not translate to high usage of academic mobile tools
  • gear all mobile sites and apps toward the mobile user who is looking for quick pieces of information (rather than uploading or organizing content or performing complicated tasks)
  • make library websites, databases, catalogs and resources accessible on mobile devices, but do not provide all of the functionality of desktop versions.

Strategic recommendations

Based on the overall findings from this inquiry, CDL concluded that the majority of the users surveyed and interviewed had not quite reached mass adoption or use of mobile devices in their academic workflow. Thus, the general strategic recommendations for CDL were preparatory in nature:

  • set up testing practices and environments for most heavily used device platforms. (As of summer 2010, the top three mobile platforms are Apple's iOS, RIM BlackBerry OS and Android OS.) Testing on physical devices is preferred over emulation environments
  • in general since CDL services are digitally-based, prioritize and support mobile web access as opposed to building standalone mobile applications unless the content or the main functionality of the service is something that users would utilize in a mobile context
  • assess mobile experience for current web-based services as they stand to target the adjustments that need to take place to make them mobile-friendly
  • at the time of the study, none of CDL's usage tracking mechanisms tracked for mobile use. Thus, adopting and maintaining web analytic tools with accurate mobile device tracking and usage statistics for online services are a crucial component to future mobile use assessment
  • consider creating a mobile optimized version of service if usage statistics show a growing number of mobile device access hits or if service is something that constituents would use in a mobile context
  • continue to survey constituents and end-users annually or bi-annually to capture rapidly changing behaviors that may result from new mobile product releases as well as future growth of 4G cellular network capacity.

Recommendations revisited

Since the CDL developed a strategic plan for supporting mobile device use, they have implemented this plan in the following ways.

Testing practices

Testing practices for mobile devices were initiated using both emulation environments as well as hands-on testing of the three main mobile platforms (Apple OS, BlackBerry and Android) on physical devices. Developers began testing CDL-maintained websites to understand how the existing non-mobile-optimized websites were functioning in a mobile space. The majority of these tested websites could be viewed fairly well on mobile devices. However, not all functionality was available for the three main mobile platforms.

Designing mobile web access

Once the study was completed, the CDL organizational website was targeted as a good test case to build a mobile website counterpart to the desktop website. Existing desktop website content was re-shaped and cherry-picked specifically to serve the needs of a mobile user's information needs. The team prioritized quick hits of information that users would need in a mobile context, such as directions, staff directories and quick links to services.

Web analytics for mobile device use

One of the most important changes to occur at CDL post-study was the adoption of web analytic tracking of mobile device access. Since not all of CDL systems are configured in the same way, different mobile tracking mechanisms had to be adopted. Some services started use of Google Analytics. Others added in mobile filters to their tracking mechanisms to capture mobile use. Several service teams have made plans that they would implement more formal mobile development once mobile usage reached 1% of total access. Since the study in 2010, this new mobile tracking infrastructure revealed that, as the study had concluded, many UC users were not attempting to access CDL-maintained library resources with a mobile device. For the most part, usage was well below 1% of total access. Of note, iPad use is significant in comparison with the use of other mobile devices. In usage statistics for CDL services monitored in the first few months of 2011, approximately 75% of the below 1% mobile access attempts were attributed to the iPad.

Moving forward, an internal mobile summit will be called every six months to re-evaluate mobile usage and follow up on outstanding shifts in UC access of CDL services with mobile devices.

Note: Further details and comprehensive reporting of the contextual investigation that was conducted in support of this project can be found on a project site: http://www.cdlib.org/services/uxdesign/mobile_project/

References

  1. Hennig, N , It's a mobile world, where do you fit? MIT Libraries: http://www.slideshare.net/nic221/its-a-mobile-world-world-where-do-you-fit (accessed 10 June 2010).

  2. Lenhart, A, Purcell, K, Smith, A and Zickuhr, K , Social Media and Young Adults. Pew Research Center's Internet & American Life Project: http://www.pewinternet.org/Reports/2010/Teens-and-Mobile-Phones.aspx?r=1 (accessed 10 June 2010).

  3. Morgan Stanley, The Mobile Internet Report: http://www.morganstanley.com/institutional/techresearch/mobile_internet_report122009.html (accessed 09 August 2010).

  4. Horrigan, J , Wireless Internet Use, Pew Research Center's Internet & American Life Project: http://www.pewinternet.org/Reports/2009/12-Wireless-Internet-Use.aspx (accessed 08 August 2010).

  5. Smith, S, Salaway, G and Caruso, J , The ECAR Study of Undergraduate Students and Information Technology, 2009 (No. ERS0906), EDUCAUSE Center for Applied Research. Article: http://www.educause.edu/Resources/TheECARStudyofUndergraduateStu/187215 (accessed 10 June 2010).

  6. Horrigan, J , ref. 4.

  7. Hinman, R, Spasojevic, M and Isomursu, P , They call it surfing for a reason: identifying mobile internet needs through pc internet deprivation, In: CHI ‘08 extended abstracts on Human factors in computing systems, 2195–2208, Florence, Italy: ACM, doi:10.1145/1358628.1358652.

  8. Sheehan, M and Pirani, J , Spreading the World: Messaging and Communications in Higher Education (Research Study No. Vol. 2), 2009, Boulder, CA, EDUCAUSE Center for Applied Research: http://www.educause.edu/Resources/SpreadingtheWordMessagingandCo/168953 (accessed 07 June 2010).

  9. Mills, K , M-Libraries: Information use on the move, 2009, Arcadia Programme.

  10. Latimer, D , Text Messaging as Emergency Communication Superstar? Nt so gr8, EDUCAUSE Review, 2008, 43(3), 84–85.

  11. Washburn, B , OCLC Research: Library Mobile App Development, Current conditions and strategic choices. Webinar presented at the TAI CHI Webinar Series, 2010.

  12. E-reader Pilot at Princeton, Final report, summary version, 2010, Princeton University: http://www.princeton.edu/ereaderpilot/ (accessed 09 August 2010).

  13. Kellog, D , (2010, June4), iPhone vs. Android, Nielsen Wire. Article: http://blog.nielsen.com/nielsenwire/online_mobile/iphone-vs-android/ (accessed 09 June 2010).

  14. Smith, S , ref. 5.

comments powered by Disqus