Methodology

Selection of Study Titles

To begin the research process print journals for which the University had also acquired licenses for digital access were identified (1); approximately 6000 journal titles from all disciplines. The Providers of these journals were then assessed as to their capability of providing relevant use statistics. Fifteen Providers were identified as capable of delivering use statistics. Eligibility of specific journals for the study also required that at least two or more campuses had equivalent print holdings as well as digital access. This was determined by the examination of the journal holdings statements in the CDL Periodical Titles database.

Applying these criteria for title participation resulted in a qualified pool of 2683 research journal titles. Following guidelines developed by the project's Operations Advisory Committee and using work sheets generated from the research database each campus selected titles from this pool for withdrawal from the stacks, based on their knowledge of collection use, faculty interest, subscriptions, and holdings. After this first round of selections, the CMI staff in consultation with campuses matched selections with those of other campuses to create a final list of Study Titles. The final list of Study Titles totals 285 journals.

Examining Use of Print Versions of Study Titles

The number of uses of Study Titles is needed to determine if placing journals in storage has an effect on the volume of uses of either print or digital formats and for providing use data to compare with the expressed preferences for print or digital gathered in the User Preference Survey. Use of the print Study Titles at Control Sites is measured by the Control Use Data Slip. Data slips are placed inside of each Study Title piece with the instructions to all appropriate library staff to record every instance of reshelving. These slips are examined and tallied on the Control Use Worksheet by individual title each quarter. The Control Use Worksheet is sent to the Project for inclusion in the use analysis.

Use of the print copy of a Study Title sent to storage by Experimental Sites is measured in two ways; by copies of the original Return Request Form for the item from the particular Experimental Site and quarterly Experimental Use Worksheets from the Storage Facilities. To assure that Study Titles remain in or are returned to the appropriate control or Experimental Sites all physical pieces of a Study Title are marked with an identifying external spine and cover label. The labels are color coded to distinguish between control and experimental volumes and include appropriate handling instructions.

Examining Use of Digital Versions of Study Titles

Use of the digital versions of Study Titles is measured through statistics collected by the Providers of the online version of any given Study Title. Participating Providers all provide on-line digital use statistics. Although there are minor differences in the way these statistics are gathered the pre-selection of Providers based on this capability eliminated Providers with significant differences. Each participating Provider site is visited and an analysis identifying equivalencies in reported data prepared. The analysis results in a measure for each Provider that counts access to Study Titles at the article, table of contents and summary level; using the number of views and downloads (PDF, HTML and special formats) to compare with the number of reshelving instances. It is also the basis for specifications to assure that over time the Provider data is consistently downloaded for comparison.

Examining Differences in Print and Digital Forms that May Influence Use

The acceptability of the digital form of a Study Title may be a result of the characteristics of presentation and content of the Study Title in the different media. To account for this a Study Title Characteristics Analysis will be undertaken. By using the Study Title Characteristics Worksheet, a profile of the print and digital features of each Study Title will be developed. This will result in a list of characteristics present or absent in both forms of the Study Titles. In addition, a subject classification of Study Titles will examine possible subject factors that may influence use. This information will be used along with Control, Experimental and Digital Use Data to determine if or to what degree these characteristics are related to the number of uses. These characteristics will also be used in the User Preference Survey to craft questions on preference.

Examining User Preferences

Three approaches will be used to examine user preferences, behaviors and attitudes. The first is the User Request Survey. This survey accompanies each filled request for a Study Title or article copy from a Study Title when returned from the Storage Location to the local library or user. It asks questions regarding the purpose of using the print rather then the digital version and examines user awareness of the availability of digital. This survey will provide Study Title specific preference data that can be compared to actual print and digital use of that title. The second qualitative method of assessing user preferences is through a User Preference Survey administered to faculty, students and staff throughout the University system. This survey will be based on Formative Interviews with representative journal users to determine the focus and scope of the survey. Results will be examined for specific characteristics of journals, users, purpose of use and the user environment that may correlate with a preference to use journals in print or digital form. The usage data and preferences from the User Request Surveys with the preferences from the broad User Preference Survey will describe patron behaviors and attitudes toward using digital. The third method that will contribute to the examination of user preferences is a analysis of volunteered comments. The Comment Card Analysis will help identify possible situational factors not recognized elsewhere. To solicit comments, comment cards will be supplied at all study locations. As will a comment feature on the CMI project and participating campus library web sites to facilitate patrons contributing their thoughts.

Examining Decision Making Processes

This research requires coordination and consensus between campuses concerning the selection, processing, implementation and operations of the experimental and control conditions. The CMI Project effected the required coordination by establishing new working groups, advisory committees and guidelines to provide the necessary level of consultation and joint decisions. This structure and process is examined through the Consultation Process Survey and a descriptive analysis of the results will contribute to policy and program recommendations.

Examining Costs and Benefits of the Experimental Treatment

The costs of selecting and relocating Study Titles to storage will be captured by the Phase I Cost Worksheets, prepared by participating campuses and storage locations using guidelines from the project. The costs of operations to campuses and storage locations during the Experimental Treatment will be identified through a series of Phase II Special Cost Studies including Storage Reports that will specify by Study Title the number of volumes per linear foot requirements of the Experimental Condition. Costs of Study Title print subscriptions and digital licensing will be examined through a special study of university - publisher agreements. This data will be examined for potential savings in, local library space, processing, binding, acquisitions and serials management.

Data Management and Reporting

This research effort will generate a substantial amount of raw data. To manage, analyze and provide reports to answer research questions and to provide access for possible subsequent studies, a database organized around Study Title will be developed. This will provide for rapid and repeatable tabular analysis at critical points in the research. Although much of the data is being provided in a manner that may be attributable to a particular library user or survey respondent the database will not record data in an attributable fashion. Use data for the Study Titles will be made available in tabular form at quarterly intervals through out the study. The project will maintain a web site and will post research plans, guidelines, data and analysis as it comes available.

<< Previous: Data Gathering Instruments | Next: Appendix A >>

1. California Digital Library, Key Indicators of Collections and Use, July 1, 2000 - June 30, 2001. Prepared October 15, 2001 by: Rosalie Lack, John Ober.