Institutional repositories trace their roots to disciplinary e-print servers established to facilitate early and open access to research literature by allowing authors to self-archive digital preprints or postprints of their own journal articles. Topical aggregations were necessary because each repository was a silo with its own search-and-retrieval system and there was no easy way for researchers to search multiple repositories at once. This situation changed with the 1999 release of the Santa Fe Convention, which subsequently became the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). Repository applications that supported OAI could export their metadata for harvesting into other aggregations, essentially uncoupling the repository itself from the discovery function. This allowed universities to establish institution-based repositories without disrupting disciplinary research.

The movement for institutional repositories has been carried along by a whole new set of goals. For some, they are a way for the university to promote itself by collecting and disseminating the works of its faculty. For others, they are a means of undermining traditional scholarly publishing and a weapon in the battle against skyrocketing journal subscription prices. To many, institutional repository is synonymous with preservation repository. An often-quoted article by Clifford Lynch calls the IR “most essentially an organizational commitment to the stewardship of these digital materials, including long-term preservation where appropriate, as well as organization and access or distribution.”¹ Unfortunately, in the United States, institutional repositories are underused and struggling to attract content. A 2007 study counted roughly 100 IRs in U.S. universities and colleges, but also found the median growth rate was only one item a day.² Growth occurs when deposit can be mandated—for example, for electronic dissertations—or done by administrative staff, as for departmental publications. Faculty use, however, is nearly always voluntary and nearly always low.

Most institutional repository systems support the same set of core functions. They provide a mechanism for submitters to register, and for registered submitters to log on. They provide a mechanism for uploading digital materials and forms for entering descriptive metadata, whether by the (presumably bibliographically unskilled) author or some other party. They allow submissions to be reviewed and edited before approval for ingest. They ingest, store, and provide some level of content management for the digital materials. They provide access to the materials using local search-and-retrieval features and support some methods of access control. They have the ability to expose OAI-PMH–compliant metadata for harvesting into other aggregations.

Institutional repositories may not be complete preservation solutions, but they do make preservation possible by allowing responsible curatorial organizations to capture content they might otherwise not have, and to secure appropriate rights from the authors. Viewed as preservation repositories, IRs have ingest and metadata capabilities that support the Availability and Identity goals of the core preservation activities. Because the content is managed centrally by skilled IT staff, the Fixity and Viability of the stored files should also be ensured. Where IR applications differ is in the extent of their support for Authenticity and Renderability.

DSpace, Fedora, and EPrints are the three dominant applications used for institutional repositories, although each can be used as a platform for other services as well. Each incorporates, or has plans to incorporate, different preservation functionality as noted below. DSpace, Fedora, and EPrints are also the focus of the annual Open Repositories conference, which features open user-group meetings for the three applications, followed by sessions on topics of common interest, often including preservation. Papers from Open Repositories 2007 are available online. The 2008 meeting will be held in Southampton, England, in April.

Open Repositories 2007 http://openrepositories.org/2007

Open Repositories 2008 http://or08.ecs.soton.ac.uk

DAITSS was developed by the Florida Center for Library Automation (FCLA) as the software application underlying the Florida Digital Archive, a preservation repository for the use of the public universities of the state of Florida. It is available for use under the open-source GPL license, but there is no active community of developers beyond the FCLA staff. DAITSS is meant to run under Linux or Unix, is written in Java and Ruby, and uses a MySQL database.

DAITSS is designed to conform closely to the OAIS model. It will accept SIPs and transform them into AIPs for storage. As part of the ingest process, DAITSS identifies the format of each incoming file, validates the file, extracts technical metadata, and creates normalized or migrated versions if necessary. The original files as submitted are always preserved, along with the “last, best” version created by DAITSS, and both the originals and “last, best” files are disseminated in a DIP on request.

Since all format transformations are done at the time of ingest, it is possible that content archived through the system may not be stored in current renderable formats at any given point. To get around that problem, when a dissemination is requested, DAITSS exports the AIP and transforms it into an SIP for re-ingesting. As the files are re-ingested, they are identified, validated, and normalized or migrated just like any other SIP contents. The newly updated AIP is then disseminated to fulfill the request. As such, the system can be said to perform “just in time” format transformation. Mass migration can be accomplished by using the reporting system to identify all AIPs containing files requiring migration and requesting their dissemination.

DAITSS can accept content in any format, but can ensure renderability only for supported formats, which are listed on the project Web site. A DAITSS repository is most suitable for text, image, audio, and video formats, which benefit from migration, as opposed to video games and other interactive multimedia more amenable to emulation. DAITSS records PREMIS-conformant metadata and maintains digital provenance in the form of PREMIS event records. There is an administrative interface for repository staff, but no access interface for end users. The system is designed to be used as a stand-alone preservation repository or as a backend to institutional repository systems, digital library systems, e-journal publishing systems, and the like.

An advantage of DAITSS is that it was designed from the start to implement format transformation as a preservation strategy and so provides a complete infrastructure for supporting migration. A weakness is that the system is difficult to install, configure, and manage, and is very difficult to contribute code to. Development on a new generation of the software, DAITSS 2.0, began in 2007 and will hopefully remedy some of these problems.

DAITSS Web site www.fcla.edu/digitalArchive

DAITSS wiki http://daitss.fcla.edu

LOCKSS (Lots Of Copies Keeps Stuff Safe) was developed at Stanford University as a way for libraries to capture and store their own local versions of digital content, according to the theory that the more institutions hold something, the less likely that something is to disappear. Originally designed for electronic journals, it can store any Web-accessible content. The freely available software was released in beta version in 2000 and in production in 2004.

LOCKSS is designed to run on stand-alone low-end microcomputer hardware. The installation/boot disk contains its own operating system and everything necessary to turn the machine into a “LOCKSS Box.” Installation and operation are very simple.

Before a publication can be harvested by a LOCKSS Box, the publisher has to give permission in the form of a “manifest” page on the Web site, and a LOCKSS plug-in has to exist for the publishing platform. The library running the LOCKSS Box configures it with a list of titles the library wants to store. The publisher's license generally allows the library's users to access the content from the LOCKSS Box in the event the publisher's content is unavailable, for example, when the publisher's server is down, or if the library discontinues its subscription to a title.

LOCKSS ensures the fixity of the contents in an innovative way. Multiple LOCKSS Boxes with overlapping content periodically participate in “polls,” where the message digest (checksum) of the content held in common is compared. If the content of one box is different from the others, it is assumed to be damaged and automatically replaced with a good copy. Details of the “majority voting, fault tolerant” system are described by Rosenthal and Reich.⁷

Although LOCKSS primarily guarantees Availability and Fixity, developers have begun to address Renderability by doing a proof of concept that forward format migration can be done at the time that stored content is delivered to the Web browser.

The LOCKSS Alliance is a library membership organization with annual fees ranging from just over $1000 to almost $11,000 depending on the size and type of institution. Alliance members get some incentives such as access to “premium” content, but participation is mostly a way of supporting the LOCKSS initiative.

Most of the publishers participating in LOCKSS are open-access publishers or university presses. In contrast, CLOCKSS (Controlled LOCKSS) is a smaller and more formal partnership involving eleven scholarly publishers (many of which do not participate in public LOCKSS), six libraries, and OCLC. By agreement with the publishers, the CLOCKSS libraries host LOCKSS Boxes comprehensively caching these publishers' journal output, and will make it available to everyone without restriction when “no publisher has current responsibility for, nor is providing electronic access to” the content.

Other projects are finding LOCKSS useful as a core or auxiliary technology. The MetaArchive of the American South and the Alabama Digital Preservation Network are using LOCKSS as the basis of a distributed preservation network infrastructure aimed at historical materials and locally created digital content, respectively. Other projects are using LOCKSS as a tool for replication and verification of file integrity as part of a larger preservation repository application.

LOCKSS Web site www.lockss.org/lockss/Home

CLOCKSS Web site www.clockss.org/clockss/Home

aDORe was developed at Los Alamos National Laboratory (LANL) by the Digital Library Research & Prototyping Team. Development of the software was partly funded by an NDIIP grant from the Library of Congress. It was released in 2006 for public use under the GNU Lesser General Public License (LGPL). Development is controlled by LANL. The system is written in Java and has been tested on Linux, Windows, and Mac OS platforms.

aDORe is not a complete preservation solution but is designed to address the problem of storage for digital repositories. Writing digital objects as individual files in filesystems is inefficient in terms of I/O (input/output) for storage, access, backup, and recovery. Concatenating objects together in “tar” or “zip” files is more efficient for I/O but impedes the use of standard tools for indexing and manipulating the contents. The aDORe approach is to separate the storage of data files (bitstreams) and their metadata and to package each in formats that can be easily accessed. For a given set of digital objects, the XML-based metadata files (for example, MPEG-21 DIDL or METS documents) are concatenated together in larger files called XMLTape files. The content data files themselves are combined into ARC files, a format developed by the Internet Archive (see Chapter 7). The XMLTape files and ARC files reference each other by means of identifiers.

An advantage of aDORe is that it integrates widely used open-source tools and standards. The metadata in the XMLTape can be indexed and accessed through the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). A datastream in an ARC file located through the OAI-PMH repository can be accessed via its OpenURL. The architecture allows for multiple aDORe archives to be easily federated.

Another benefit is that the standards-based approach should significantly lower the learning curve for implementers. The system is documented and includes a tutorial. A plug-in architecture will allow developers to build their own customized pre-processors for the content of XMLtapes and ARC files.

aDORe Archive Web site http://african.lanl.gov/aDORe/projects/adoreArchive