BIODOC: A Preliminary User Analysis

Cranfield University's BIODOC project aims to explore a number of key issues m the current debate over access versus holdings. This is an interim study of user perceptions and responses to an experimental service in which a number of existing Biotechnology journal subscriptions were replaced by a wholly access-based service.

The BIODOC project was conceived, in part, as a way of providing the library with an opportunity to experience and evaluate a number of the critical issues and challenges that the transition to more access based information services are likely to present to library customers and staff. The focus of the following study is concerned with the user impact of the project.
Ultimately, the pro* will enable the library to reach some conclusions as to the applicability of an access-based service to a wider customer base across the campus and the university, and to offer some indication of its potential transferability to the academic library sector as a whole.
The results described in this analysis are based largely on research carried out six months into the project. A user focus group conducted after the end of the first year of BIODOC has confirmed many of the findings described in this paper. The article attempts to address the question of what we have learnt generally from BIODOC; the emphasis is very much on the library customer.

Background
A full description of the design and operation of the first twelve months of the BIODOC project is given by Evans, Bevan and Hanington,19%.' BIODOC is a practical research project which is exploring the proposition that, in certain circumstances, a university library can offer a better service to customers by providing access to the information held in journals through electronic current awareness and document delivery services than by holding collections of purnals in house. The project is a collaboration between Cranfield University Library and Cranfield University Biotechnology Centre. Phase I of the Project ran from January 1995 to January 1996 and Phase I1 runs from February 1996 to January 1997. Phase I of the project was a research oriented project with a high level of library intervention and monitoring. Phase 11 is a more practical trial including less monitoring with a view to making its procedures a normal part of the library service.

Aims
to see if the BIODOC experiment is a cost effective approach to journal supply to see if it provides a better service for Biotechnology to compare document suppliers

Reasons for the project
The nature of Cranfield's work as a postgraduate teaching and research institution in leading edge technology meant that the library was already a heavy user of interlibrary loans, and the project was a logical extension of this activity. The recent spiralling cost of journals had resulted in a number of cancellations which meant that Biotechnology holdings were down to 16 titles. The department had very diverse research interests which could not be covered by this. For instance in 1994, pre-BIODOC, 559 different journals were consulted through interlibrary loans. The growth of electronic current awareness services and on-line document delivery services provided an opportunity to change the method of journal provision.

Operation
The library cancelled its subscriptions to all the journals which were bought for the Cranfield Biotechnology Centre in January 1995 and replaced them by a new service. This consisted of an electronic table of contents service, UnCover Reveal, backed up by a fast interlibrary loan service. Biotechnology staff selected journals of whose contents they wanted to be kept aware, and these were e-mailed from UnCover to their desk tops. Users also searched the UnCover database and the BIDSISI service. Downloaded references were then e-mailed to the library, who entered them on the LIBERTAS library interlibrary loan system. A number of different document supply services were used with different delivery times and methods of transmission. Services used were Uncover, Delft University of Technology, Royal Society of Chemistry, BIDSISI and BLDSC.
Training and support of users were vital components of the project.

User analysis
The analysis is based on information derived from a number of sources. The initial focus group meeting and questionnaire provided useful data on the information seeking behaviour of the library's Biotechnology customer group. The ongoing monitoring and evaluation of usage, supplier performance and expenditure, has provided much quantitative data on how the service has functioned. More qualitative information has been obtained through regular meetings with customers, and the presence of the Biotechnology representative on the Progress Group has helped to ensure a regular supply of feedback.
To date, however, the most substantial single analysis is the user study conducted over the summer of 1995 by Emma Nicholls2 as part of her University of Sheffield MSc research. The analysis offered in this present paper therefore draws heavily on her work.
The value of the insights given by the Nicholls' study to the overall evaluation of the project are underlined by the fact that the author worked as part of the ILL team at Cranfield for twenty months prior to undertaking her studies at Sheffield.
The primary objective was to evaluate customer perceptions as to the acceptability of BIODOC, in comparison to the in-house collection of core journals and the conventional interlibrary loan service which it replaced.
A sample of twenty-three BIODOC users (see figure 1) were questioned using a standardised interview saipt that elicited fixed and open  responses. The transcripts were then subject to a content analysis in order to provide a qualitative assessment and to examine whether there were significant differences in the perceptions expressed by different categories of user (MSc, PhD, staff).
The study looked at many aspects of the BIODOC service, including current awareness, information retrieval, and document supply; attitudes to browsing; service procedures, and the effectiveness of training and support provided by the library. The CASIAS function (current alerting serviceindividual article supply), performed in the project by Uncover, was the subject of detailed scrutiny. The study also includes valuable quantitative data as part of the comparative analysis of document supplier performance, which forms one of the project objectives.
Clearly from the view point of the library customer the cancellation of existing journal subscriptions by the library will only be acceptable if the access-based service offers clear advantages and benefits to the customer. In terms of enhanced current awareness, such benefits must include access to a wider range of material, and in a time frame which is significantly quicker than if the customer had to wait for the library to acquire current journal issues for stock. Document supply services must be able to ensure fast, reliable delivery, and request procedures must not create bamers which involve customers having to do more work in order to obtain their documents. Customers must also have confidence in the ability of the library to provide adequate training, advice and ongoing support.
Our interim studies show that a number of factors appear to have influenced customers' perceptions as to the effectiveness of the BIODOC approach. Not surprisingly, these include differences in user needs and also variations in expectations and previous experience.
Interestingly one of the reasons for selecting Cranfield's Biotechnology Centre for the BIODOC project was that the staff and students were thought to have a relatively discrcte and homogenous set of subject interests, in comparison to some of the larger engineering and management schools on the campus. This has very largely proved to have been a misconception, for the project has shown that BIODOC has been used to access a much wider range of information sources than expected, and the information has been sought for a great diversity of purposes. Although customers in each of the three categories (PhD, MSc, staff 1 on the whole have been shown to have broadly similar views, usage of the three components which make up BIODOC (current awareness, database searching and document delivery) has been shown to differ according to user category, in a number of significant ways. These will be discussed in more detail as they relate to the specific aspects of the BIODOC service. A general point worth emphasising, however, is that one very valuable benefit from the project is the much better understanding that the library now has of the information seeking and using behaviour of one of its most important customer groups.

General user perceptions
Overall the results so far seem to indicate that, for Cranfield Biotechnology Centre at least, the information support offered by the BIODOC project has been more effective than the provision which preceded the project.
Nicholls found in her study that 80% of respondents prefer to have access to a larger range of journals off-site than a smaller selection on-site. There has been much additional ad hoc evidence to support this conclusion. A member of the centre's staff commented recently that he has "never been so widely read" and that, in his admittedly very impressionist view, literature searches for thesis research conducted by students since the begmning of the project appeared to be more comprehensive both in terms of number of references and range of sources cited.
It is important, however, to view these opinions in their proper context. Nicholls certainly found much evidence that the users expectations, and the very poor existing in-house purnals collection, has certainly influenced user perception of BIODOC. For instance, she comments that more than twice as many staff favour the access option as MSc respondents. This could imply that customers, already familiar with the existing limited library stock, were more likely to respond positively to the new service than an MSc student, who perhaps comes to Cranfield fresh from completing a first degree course elsewhere, and who had become accustomed to having access to a library with a much more extensive collection of appropriate journal titles. There is also evidence to suggest that the nature and range of purposes for which staff require information implies a need for access to a much wider and more diverse set of sources.

Current awareness
Among the most interesting and, in some respects, the most disappointing findings of the BIODOC project so far, has been customer reaction to the current awareness elements of the service and the use of the Uncover Reveal profiling service in particular.
Anticipation amongst the project team of a very positive customer response had been encouraged by a completely unsolicited comment, at the initial focus group meeting, from one user. He suggested that the most helpful provision that the library could make was a service that gave him ongoing access to current journals information tailored to his individual research interest. There was some disappointment when the Nicholls' study revealed that only 28% of respondents regarded the Reveal Service as an effective method of current awareness. 44% felt that it was satisfactory but 28% concluded that it had been less than satisfactory for their needs.
Some possible explanations for this response is suggested in the replies of a number of interviewees that hint at a gap between promise and reality, a typical example being: "it's a good idea in theory, but in practice it isn't". An analysis of the reasons given adds further support to the view that the basic approach of the Reveal Service is indeed very sound in principle, but in certain situations customers are experienang some difficulty in realising the potential benefits.
The problem for the customers seems clearly to be one of information management, leading to what Nicholls describes as a typical example of information overload.
Common themes in the negative responses include the number of e-mailed contents pages received, the time taken to read through these, and the inadequacy of the existing campus e-mail system. 'The e-mail messages pileup because I don't have time to look through them...", was one response from a somewhat disillusioned customer.
Again the needs and experience of the users themselves can be seen to influence perceptions. An experienced researcher commented recently that it took him only 20 minutes to review a week's worth of references from his Reveal Profile. He admits, however, that his experience and familiarity with his own areas of research had enabled him to steadily refine his profile so that it eventually included only those titles which gave him references to the information on subjects he required.
Despite warnings given during user training, it is apparent that MScs who set up profiles were too ambitious in the number of titles they selected. They have also experienced considerable difficulty in identifying which titles are most likely to cover their interests. This too often seems to have resulted in what one respondent in the Nicholls' study termed "a scatter-gun approach". Factors adding to this problem include: unfamiliarity with the subject and particularly with the jargon, or language, of the literature, and, critically, with the lack of abstracts available in the Uncover Reveal Service, to assist the more inexperienced user.
This information overload was further aggravated by the Library's provision of a standard package of Uncover Reveal titles. This consisted of the contents of 14 cancelled journals and five additional titles from which articles had been most frequently requested by Biotechnology users through interlibrary loans in the previous year. Since this time, many Biotechnology users have asked to be removed from the mailing list for this package, citing too large a volume and too many irrelevant references as the reason for their decision.
Nicholls found that whereas all of the PhDs interviewed had used the Reveal Service, only 40% of MSc respondents had set up profiles. Two PhD respondents indicated that they preferred to continue to use alternative, more familiar, approaches to current awareness, such as regular use of the BID/ISI service, to check the contents of the new titles. Having looked at these conclusions the BIODOC team has made a number of recommendations which are intended to improve the effectiveness of the current awareness service and specifically to address the issue of 'information overload'. These include the following: discontinue the provision of a standard package of e-mail contents pages; introduce more effective integration of e-mail training, as part of the overall BIODOC training; adopt and promote the use of a new campus e-mail facility which provides individuals with an easy mechanism to set up a number of different e-mail directoriesreferences from profiles can then be routed into an appropriate directory, e.g. by subject, which should make the process of managing information more effective; appropriate library staff proactively to advise and assist users in the selection of the most relevant journal titles to include in profiles.
The recommendations on training and counselling are compatible with the envisaged new professional role for library staff which a number of commentators have begun recently to describe as Network Learner S~p p o r t .~

Database searching
In view of the well-established popularity of the BID/ISI databases in the Biotechnology Centre, it was expected that UnCover would be used primarily for the Reveal Service and that the use of the UnCover database for ad hoc searching would probably be somewhat limited. To a large extent the evidence collected by the BIODOC team to date has tended to confirm this hypothesis.
Nicholls found that only about half of her interviewees had tried to use UnCover for literature searching. Amongst those who had used the database, there was some difference of opinion between categories of users as to how easy the service was to search. In this instance, it is the MSc students who appear to have been more willing than the PhDs to give the UnCover database a try.
Problems cited by respondents include log-on difficulties, poor response times (particularly in the afternoons), dissatisfaction with the standard UnCover interface, and again the lack of abstracts. 36% of respondents were ultimately dissatisfied with the results of their literature searches (compared with 55% who were satisfied and 9% who were very satisfied).
There is evidence to suggest that the negative perceptions were more the result of the access and interface problems rather than a comment on the quality of the data. Indeed, the Nicholls' study shows an overwhelmingly positive response from users of the Reveal Service and the database to the currency of the data. 92% of respondents perceived the data to be 'very up-to-date', and in comparison to the pre-BIODOC service, one customer commented, '7 think you probably get what you want there (Uncover) more quickly than you would if the journals were being sent to the librarf'.
Throughout the prop3 to date, the BIODOC team has found other evidence which supports this very positive view on the cdrrency of the UnCover data. For example, one problem that the project has encountered in terms of document delivery is that users have been requesting items retrieved from UnCover that are at the time unavailable from our UK and European suppliers. There is much impressionist evidence to suggest that in comparison to BID/ISI users, information on current journal issues appears considerably more quickly on UnCover. The BIODOC team is conducting a number of studies at present and are intending to quantify this difference in performance.
At this stage, therefore, the BIODOC team believes that there is sufficient evidence to indicate that the use of UnCover for database searching should be encouraged, and are recommending a number of initiatives to assist in making the usage more effective. These include: to investigate the acquisition of a customised gateway to UnCover in the hope of easing existing access difficulties; to improve search facilities by standardising the Personal UnCover Navigator (PUN), and to press UnCover for the further development of this interface; to encourage improved end-user searching through more effective training (discussed in more detail below). Vol.9, no2, July 1996 Hayrington, Evans and Bevan BIODOC: a preliminary user analysis

Browsing
One particular aspect of information retrieval that the team set out to explore was the impact on users of the reduction in browsability implied in the move towards access based services. Evidence collected in the Nicholls' study revealed a clear split in opinion between the users who feel that browsing is important and those who do not.
Amongst the concerns expressed by the former group included, once again, the lack of abstracts in the Uncover service; and the inability to access 'miscellaneous' sections of journals, such as, editorials, news comments, and, particularly, job advertisements. For some users, too, serendipity represents an important source for new ideas and innovation and one bemoaned that "to do a search you have to know specifically what you want, which is fine for finding things you know about".
Here again previous experience and practice appears to have coloured these view points. Interviewees accustomed to having access to a wider collection of in-house journals than had existed at Cranfield, were more likely to be fearful about the loss of browsing opportunities.
Other respondents were more pragmatic in outlook. Many believed that time constraints made browsing something of a luxury for them. Several staff respondents noted that they maintained personal subscriptions which fulfilled their browsing requirement. A number of BIODOC users were even more positive in claiming that the use of Uncover Reveal in particular has enabled them to look at a whole range of esoteric and more specialist journal publications for which it would have been impossible to make a justifiable case for the library to acquire in-house. Overall, therefore, user perceptions on browsability appear to be very much a matter of taste, practice and personal experience.

Document delivery
To date evidence collected on the document delivery element of BIODOC is rather mixed. A full comparative analysis of supplier performance, including speed of delivery, fill rate, cost and reliability, will be conducted at the end of twelve months. Customers will have to be satisfied with both the speed and reliability of the service if they are to trust the concept of access instead of holdings.
In her study Nicholls focused on percephons of ideal delivery speed and actual experience. The average delivery speed achieved by the four suppliers at roughly the half-way point in the project is shown in figure 2 I llnCovcr BIDS RSC Dcllt BLDSC Supplier Figure 2: Document supply petfofmancle Nicholls found that the majority of respondents (62%) were satisfied with this performance. A further 25% were very satisfied. Balanced against this were 13% who felt that this level of performance was not sufficient for their needs.
These views, however, appear to vary according to user needs, expectations and experience of conventional interlibrary loan services. There also appears to be some variation in opinion between the different categories of users. In response to the question on ideal delivery speed, staff and MSc students seem to have greater expectations that their PhD counterparts. This may be a reflection on the high number of research contracts, with attendant deadlines, on which staff are employed for much of the time. Similarly MSc students, having to complete a number of assignments and a research thesis, within twelve months, are also under considerable time pressures. Nicholls discovered satisfaction with document delivery speed amongst MSc students muld change over the course of the academic year. One respondent recalled, ' 7 think mine came within a week which wasn't too bad. But now, because I' m getting towards the end of it (the MSc), I want it a bit faster." While BIODOC, in terms of delivery speed, also seems to be meeting the needs of the majority of customers, there has been a major reliability concern over the quality of fax documents received from the premium services (Uncover and BlDS On-line Document Ordering System).
Problems include missing and only partially complete pages and poor reproduction of tables, graphs and diagrams. Although the suppliers involved have been very responsive in their willingness to retransmit documents, there have been many occasions on which a photocopy has been requested in order to replace a fax. It has proved difficult to pinpoint exactly the cause of these problems, but almost certainly it includes faults at both the transmission and receipt stages. The library's own fax machine has proven to be inadequate for the task, in a number of respects. So serious has this quality issue become that a number of BIODOC users have asked that in future all of their document requests be posted rather than faxed.
As a result, the BIODOC team is currently exploring a number of options to improve document delivery satisfaction. These include: upgrade library fax to plain paper/higher volume machine; route documents through fax machines in the Biotechnology Centre; improve user training, especially for MSc students, in order to more effectively stress the need to plan research in advance, and to provide more accurate expectations of delivery speed; further investigate the use of FTP for document delivery, as well as the attendant copyright issues.

Non-usage
It was anticipated that some customers might have real difficulty in adjusting their informationseeking behaviour to the requirements of the project. Consequently, an attempt was made to simplify request procedures. The need to provide continual monitoring and evaluation of progress in any case ensured that library staff assumed much of this burden. Equally the library accepted the importance of providing effective training and ongoing support, so that users received a good introduction to the project, and could be helped through problems when these occurred.
On the whole, the library has been successful in minimising non-usage. The Nicholls' study showed that 75% of respondents judged their initial training to be satisfactory and 14% very satisfactory. The negative responses raised valid, and all too familiar, user training concerns with issues of timing and the problem of balancing content against differences in individual learning styles. Encouragingly, all respondents found the day-today support provided by the library either very satisfactory (57%) or satisfactory (43%).
Nevertheless, the library is aware that some users have been experiencing difficulties with various aspects of the BIODOC procedures. While Nicholls found that only two of her intenriewees had not used BIODOC at all, a number had attempted to bypass the document request procedures by routing these through the more familiar standard interlibrary loan service.
Nicholls discovered that 62% of her interviewees had found the BIODOC request procedures easy to master, while another 25% found them moderately easy to master. Of the 13% who had experienced difficulty, most cited some aspect of the use of technology, e.g. screen captures and email, as the cause of their problems. The two individuals who had not used BIODOC at all gave contrasting explanations. One (a technician) believed she had no use for the service, as her only regular need was for information on test methodologies covered in standard handbooks or text books. The other (an MSc student) implied that his lack of computer literacy had been an important factor. Interestingly, a number of respondents expressed some frustration at having to route requests through the library and felt that the process could be made much more effective by allowing them to request documents directly from the supplier.

Technology issues
Already there is evidence to suggest that technology impinges on the customefs use of access based services in a number of different ways. In addition to the 'technophobia' felt by some of the respondents in the Nicholls study, a Serials -Vol.9, m.2, July 1996 Hanington, Evans and Beuian BIODOC: a preliminary user analysis number of hardware issues have arisen as a result of the project. These include concern in the Biotechnology Centre itself for the number of personal computers available, the requirement for disk space on the department server, and the ability of their local network to cope with the additional traffic attributable to BIODOC activities. These are all factors which the library will have to consider before recommending the application of a BIODOC-style service to other customer groups on the campus.

Areas for further work
The library is currently considering a number of further studies in order to increase its understanding and appreciation of the implications of access based services. These include the following: end-user satisfaction with documents supplied through BIODOC; impact of BIODOC on the quality of research theses p d u d &yAt% and PhD sfudents, based on an analysis of citations, the marks awarded by examiners for the literature search content and the reports made by supervisors and examiners.
In addition to these user studies the BIODOC team will learn more about how this approach will affect the library. This inevitably will involve a full cost analysis of the project in order to assess whether such an approach can actually give better value for money. Even at this stage BIODOC appears very unlikely to offer real cost savings and will, if anything, probably cost the library more than the resources which it replaced. Equally important is the likely impact on library staff. It seems inevitable that the nature of some existing jobs will change, and the library will use the BIODOC experience to identify new skills that need to be acquired, and existing skills that need to be reinforced and strengthened amongst the library staff as a whole.
Above all, however, this experience must be used to translate the current research project into an established library service, if the BIODOC approach is ultimately going to offer a viable alternate strategy.
At this point, the BIODOC team believes that its interim analysis of user feedback and behaviour described here provides encouraging evidence of the viability of access based servioes. The evidence suggests that under certain conditions enhanced current awareness and document delivery can provide a more effective strategy on which to base information support. What is far from clear at present is whether a strategy devised in response to the Cranfield situation of having to provide for a research oriented customer group, active in an area in which the library collection has no real traditional strength, can realistically be applied more generally in other academic library environments.
"BIODOC : Access versus Holdings in a university librarf'. Paper submitted for publication.

2
Nicholls, E. "BIODOC an interim evaluation of a rapid document delivery and electronic current awareness service". Unpub. MSc Thesis, University of Sheffield, 1995.