At a time of economic constraint, accurate and comparable usage reports are essential in making a compelling case about the value of journal subscriptions. The Journal Usage Statistics Portal (JUSP)1 is a new service, built in response to demand from UK higher education (HE) libraries. It enables library staff to access their usage data quickly and efficiently, and supports publishers in providing access to statistics for their customers. Essentially, JUSP provides a single point of access for HE libraries to view usage statistics for their NESLi2 2 subscriptions. NESLi2 is a national initiative supported by JISC Collections3 to license online journals on behalf of the UK academic community. Content from leading academic publishers is procured on behalf of the community, resulting in financial savings. Many university, college and research libraries take advantage of the NESLi2 deals and find that they offer good value for money compared to the time and cost involved in setting up individual subscriptions.
JUSP is being developed by a consortium involving JISC Collections, Mimas4 at the University of Manchester, Evidence Base 5 at Birmingham City University and Cranfield University 6. Members of the consortium have significant skills and expertise following several years' research in the analysis of usage statistics.
In 2005, Evidence Base worked with COUNTER7 and Cranfield University on a JISC Collections study to analyze the value and usage of NESLi2 deals8. One of the main recommendations from this work was that there should be a usage statistics portal to provide a single point of access to data and avoid the need for e-resource librarians to visit each publisher's website to download usage reports.
Subsequent projects run by Evidence Base included the Analyzing Publisher Deals and Measuring Up projects in which over 20 HE libraries took part9. The projects, based on the NESLi2 study methodology, demonstrated that there was a real need and demand for such a portal from information services within the HE community.
In 2008, a prototype usage statistics portal was developed in response to a funding call from the Joint Information Systems Committee (JISC)10. Initially the JUSP team worked with a sample group of five libraries, three publishers and one intermediary to successfully establish the feasibility of the portal. Subsequent development of a usage statistics service commenced in April 2010, with funding currently granted to December 2011. Key to each phase of work is an understanding of user requirements, in order to deliver effective solutions within a changing landscape. This current phase of development aims to include all NESLi2 participating publishers and approximately 150 UK higher education institutions (HEIs) that subscribe to the NESli2 deals. Against a background of increasing interest in understanding the value of activity and attention data, and the importance of the shared services agenda, this is a timely development.
Standardized Usage Statistics Harvesting Initiative (SUSHI)
Usage data is collected from publishers on behalf of libraries, using the Standardized Usage Statistics Harvesting Initiative (SUSHI)11 protocol. SUSHI is a significant aspect of JUSP and has the potential to provide great benefits to libraries. The protocol is a machine-to-machine way of gathering usage statistics to provide greater efficiency and to replace the user-mediated process of data collection. Libraries participating in JUSP benefit through access to automated monthly updates, avoiding the need to access individual publisher websites to obtain data, saving both time and effort.
At 1 April 2011, data was being collected via SUSHI for eight publishers and intermediaries with more added each month. The JUSP team are currently working with a number of NESLi2 publishers to implement SUSHI throughout 2011; this includes organizations such as Elsevier who are working with the team to deliver usage data via SUSHI for the first time. Much of this work is pioneering, but success has been achieved through co-operation, collaboration and an ongoing dialogue with publishers.
A JUSP SUSHI server enables participating libraries to download their usage statistics from the portal. Benefits of using the SUSHI server are that JUSP data has been quality checked and assured prior to loading into the portal, and libraries only have to configure their SUSHI client to work with one server, which has been fully tested, as opposed to up to 19 Nesli2 publishers' servers.
JUSP: the library perspective
What are the benefits?
JUSP provides a number of benefits for libraries. It avoids the need to visit multiple publisher sites to download usage statistics and allows libraries to quickly and easily compare usage across various publishers and years. It also includes data from gateways and hosts to provide a truer picture of usage. As well as being able to view and download standard usage reports, libraries can access a range of added-value reports to help them analyze usage and establish value for money to help in purchasing and renewal decisions. JUSP also provides JISC Collections with reports on how the deals they negotiate are being used by participating libraries.
How do libraries participate?
Libraries joining JUSP are required to sign a library participation agreement allowing JUSP access to their usage statistics. Libraries also need to complete a short web-based form enabling their account to be created and statistics accessed; data is then automatically gathered from each publisher on a monthly basis and updated on their behalf. In April 2011, 70 UK HEIs had signed the agreement and had passworded access to their own data through JUSP, with additional libraries joining at a rate of around ten per month throughout 2011. All HEIs subscribing to NESLi2 deals will be invited to participate and their data included by December 2011. An up-to-date list of participating libraries and publishers is available from the JUSP website. As well as NESLi2 deals, the portal aims to include separate publisher deals negotiated by JISC Collections for the Scottish Higher Education Digital Library (SHEDL)12 and for the Wales Higher Education Libraries Forum (WHELF)13.
What usage reports are available?
Over the past few years, the success of the COUNTER Code of Practice has meant that libraries can now be very much more confident that publishers are recording e-journal usage in an accurate and consistent way. The JR1 report (number of successful full-text article requests by month and journal), and the separate JR1a (number of successful full-text article requests from an archive by month and journal) form the basis of JUSP, although a number of supplementary reports are also provided. There are currently around 14 usage reports including journal-level reports, a number of summary reports and some experimental reports that continue to be evaluated and adapted through discussions with libraries. Reports can be presented by calendar year, academic year or any period chosen by the user. Most reports can be sorted on different variables such as title and total number of requests. Usage data from the majority of reports can be downloaded in CSV format or viewed as charts and graphs.
JUSP: the publisher perspective
What are the benefits?
JUSP helps publishers and intermediaries ensure that libraries have accurate figures on which to assess the value of their deals. It makes the delivery and analysis of usage statistics more efficient for their customers, whilst the ability to view aggregated usage statistics from publishers and intermediaries provides a more complete picture of journal usage. By removing the need for libraries to maintain separate password administration routines to download data from each publisher, JUSP provides greater efficiency.
Providing assurance that the data is correct, libraries can see how well titles in a deal are being used, analyze trends over time for the whole deal or for individual titles within it, and provide evidence of value for money. Libraries can use this information to plan developments in their library systems and internal marketing that will aid resource discovery and increase usage.
There are instances when there are queries over data, or usage statistics need to be adjusted. JUSP can address these queries and adjustments centrally and on behalf of institutions, providing greater efficiencies for publishers.
How do publishers participate?
Publishers joining JUSP sign a participation agreement similar to the version signed by libraries. In April 2011, ten NESLi2 publishers and intermediary services had signed agreements indicating their participation in JUSP. Publishers and intermediaries include Elsevier, Oxford University Press, SAGE, Nature Publishing Group, Springer, American Institute of Physics (AIP) and Project MUSE/The Johns Hopkins University Press, EBSCO EJS, Publishing Technology (Ingentaconnect) and Swets. Negotiations with a number of other publishers are at an advanced stage.
Publishers have been supportive of JUSP development and happy to participate in a service that provides benefits for their customers. The JUSP team has received a great deal of co-operation from publishers, particularly over the implementation of SUSHI, and this ongoing collaboration has been key to success.
What data is collected?
At present, JR1 and JR1a usage reports are collected from publishers participating in JUSP via SUSHI. COUNTER compliance and the automated gathering of publisher statistics via SUSHI are fundamental to JUSP, and it is looking hopeful that all publishers will adopt this in the near future.
In order to respect both publisher and library confidentiality, the portal will not include details of how much an individual library has paid for a particular deal. JUSP does not contain information about cost per download; it only contains information about journal usage.
The JUSP portal is under continual development, and is version controlled, with fixed releases at regular intervals. The team use ‘agile’ software development processes, aiming to deliver customer satisfaction by rapid, continuous delivery of useful software. Community consultation is a key aspect of service development. Establishing and understanding user requirements is an ongoing process in a changing environment. JUSP builds on knowledge acquired by the team through previous studies and aims to deliver a user-centred solution to analyzing usage data. The team encourages regular, ongoing feedback from both libraries and publishers to influence development of the service and ensure it is closely aligned with user needs.
Feedback is captured through a variety of formal and informal measures such as e-mail, discussions, surveys and interviews. A user survey was conducted during November 2010 to gather feedback from participating institutions about available reports, technical aspects, interface design, strengths, weaknesses and suggested improvements.
It resulted in some useful feedback and subsequent changes were made to the portal in the light of suggestions. Respondents especially liked those reports that saved time and added value. Aspects of the portal considered most successful were:
- the ease of navigating the portal
- a clear user interface
- SUSHI harvesting of data
- the ability to compare and combine reports
- the fact that all usage data is loaded in one place
- the ability to look at both calendar and academic years.
Respondents indicated that they would also like to see more NESLi2 publishers (the aim is to include all by December 2011) and also non-NESLi2 publishers (this will be for later consideration). Subsequent discussions with library staff in March 2011 were used to augment feedback from the survey and explore use of JUSP within libraries.
JUSP use within libraries
All libraries are now placing emphasis on analyzing the ways that their publisher deals are being used, and whether they continue to offer value for money, and highlighting the implications of cancelling titles when budgets are stretched. Some libraries have a top-sliced budget for the ‘big deals’, others divide up the funds by subject departments or schools. The approach to this analysis differs, some libraries using statistics packages such as UStat or 360 Counter, others using their own systems and spreadsheets. The aim of JUSP is to help with this analysis whatever system the library uses.
Discussions with library staff explored how JUSP is being used currently and plans for future use. Libraries are regularly using JUSP for internal benchmarking purposes to analyze trends over time and to compare usage of publisher deals. They value the time efficiencies it offers, the aggregation of data and the increasing benefits of using JUSP as the variety of publishers is extended. An individual with responsibility for usage data within an organization can change over time. Consequently, JUSP is valued for its ability to provide a complete, constantly updated source of usage data unaffected by staff changes within the organization.
Because JUSP is a central service, changes in the COUNTER standard will be accommodated centrally and as necessary. This removes the need for institutions to monitor and adapt to these changes internally and this is recognized as an important feature of the service. If publishers ever need to rectify errors in usage statistics reporting, the errors will be rectified by JUSP, without any action on the part of the library.
User requirements and portal enhancements
Aprocess of ongoing consultation informs development of the portal with feedback logged and reviewed at regular intervals. Consequently, the team is able to respond quickly to changing requirements. Two examples of reports and features implemented as a direct result of user feedback are:
- individual search and usage by title or ISSN
Following discussions with library staff about their use of JUSP, the team developed a journal title or ISSN search. The facility enables an individual title, keyword or ISSN search with options to select single or multiple results. This is proving to be a popular and frequently used reporting option.
- time series data
In response to requests, JUSP provides usage data back to 2009 for all new libraries and publishers joining JUSP. This allows libraries to track usage over a longer period of time. With some deals covering a two- to three-year period, and university courses changing over time, this can be a useful aid to renewal, where one year's usage may be misleading.
Future development plans
The number of publishers participating in JUSP is increasing steadily; inevitably JUSP becomes more useful to libraries as this number expands throughout 2011. Institutions have indicated that they would like to see coverage extended beyond the NESLi2 deals to a wider range of publishers and this is an area that the team hopes to address in the future.
Scheduled enhancements to the portal include additional usage reports, and a range of time-efficient, value-added services. Tutorials and support materials will be developed throughout the course of the year to help library staff understand and interpret their usage data and integrate JUSP data with other applications.
An important aspect of the portal is that it will save libraries the time and effort involved in collecting usage statistics, and will present them with some basic analysis. It is hoped that the portal will help all libraries to gain a clearer picture of their usage of NESLi2 deals and to present their own reports based on the data in the portal. For publishers, JUSP provides increased efficiencies for their customers.
A significant aspect of JUSP development involves ongoing consultation with both libraries and publishers. Regular communication is key to development of a responsive service and community resource.
As JUSP continues to develop and expand as a JISC Collections service, we expect that lessons learned from development of the portal as a service to UK HE can be successfully applied to the wider library and information community.