Chapter 3. Assessment

Your institution or organization may have already begun digital preservation efforts without knowingly advancing a digital preservation program. These efforts most commonly occur unintentionally in policy development or technological systems improvements. For instance, your collection development policies, mission statements, and access procedures could already mention digital materials. Your technology systems administrators often have already started along the road of information security, systems diversity, and more. Without a comprehensive assessment of your organization, there is no way to tell what, if any, digital objects your institution currently maintains, what resources your organization has to devote to the project, and what policies or procedures already exist that could help or hinder preservation efforts. Therefore, the first step for any new program or even an overhaul of an existing program is to do an assessment.

There are several stages of assessment to prepare for creating or updating a digital preservation program. The first stage is getting an overview of the current digital objects that need preservation in your organization. Not all digital objects created or held by your organization are worthy of expending resources to preserve. This is similar to the fact that not every piece of paper in the world is worthy of being arranged, described, and maintained in a traditional archival repository. The second stage is to determine what resources your organization has to preserve these digital objects. The assessment of available resources should include personnel, technological infrastructure, and monetary assets. The third stage is to survey the current policies and procedures of your organization that may relate tangentially or directly to future digital preservation efforts. Finally, your assessment should conclude with the development of realistic and aspirational goals for your organization’s digital preservation program. These goals will set the stage for policy creation and advocacy efforts focused on preserving existing funding and support potential increases in resources for your digital preservation program.

The first stage in your suite of assessments is to do an inventory of your institution’s current digital holdings. This inventory can be as high level as listing categories of digital objects only, or as granular as listing every piece of digital media in your archives, all of the accessions with digital materials, and all of the e-journals, databases, and e-books the library has purchased or subscribes to. Most likely your inventory will land somewhere between the two extremes. Figure 3.1 is an example of this. The purpose of the inventory is to determine what you need to preserve, if anything. In the unlikely event that there are no digital objects in your holdings at present, the inventory is a place to speculate on the digital material you would like to collect from potential donors or the digital material you would like to create or capture through digitization, digital scholarship projects, or the course of regular business. What you have, or will have, in terms of digital objects will determine the procedures you create and the tools you will use in your digital preservation program.

The inventory process is an ideal time to set up and implement your first workflow for your digital preservation program. In many organizations, digital media is separated out from the original donation during the accession and appraisal process. There are organizations that have discontinued the practice of separating materials in different formats—photographs, for example—to different storage locations in a repository because of newer archival processing guidelines like “More Product, Less Process.”1 Separation is still a necessary strategy for digital media due to the fragility of the digital carrier media. Also, separation allows you to prioritize the stabilization of digital materials as they arrive at your archives. The stabilization process will be covered in chapter 5.

The separation and inventory workflow I use starts upon finding a digital media object, such as a CD, DVD, flash drive, or floppy disk, in an accession. I immediately remove the item and place a flag or separation sheet in the place I found the item in the box or folder. The separation sheet includes the date of separation, the type of item, a description of the item, where the item has been moved to, and who performed the separation. The description of the item includes some kind of digital object identifier in case there are multiple digital media items in an accession. The identifier I assign to the digital media in the University of Montana’s archival collections is AccessionNumber_ObjectNumber, where the object number starts at 001. (You may have collections that have more than 100 digital media items, so plan ahead for this when numbering.) It is important to label the digital media in some way with that identifier as well. The “moved to” location should be a centralized location where you keep all digital media. All digital media objects that belong to the same accession should be kept together, either in a folder for a few objects, or in their own box for many objects, clearly labeled with the accession number.

The next stage in your assessment is to identify and document all the resources available to perform digital preservation activities. This part of the assessment will require that you connect with other people in your organization and discuss current and potential resources that could be assigned to a digital preservation program. It is important that you have a simple explanation ready of what digital preservation is and in what context you are asking these questions to create a common understanding of what you are trying to achieve. This is crucial for when you interact with your information technology professionals. They will use terms similar to those used by a digital preservationist, but the working definitions of those terms can be very different for the two fields. Other colleagues may not have any idea what digital preservation is or why you are asking for more information about resource allocation. An example of a simple, focused pitch about what you are doing and why would be: “I am trying to find out how the library backs up internally produced content to see how often we archive our content, how long that content stays in storage, and in how many different places the content is saved to determine how secure our digital content is for future use. Can you help me?” This is a long question with many pieces, but it is focused on specific aspects of digital preservation: storage diversity, length of storage time, and how often backups are overwritten. Simply asking the question will help you start the discussion about a digital preservation program with colleagues in your organization.

The most important resource to assess is personnel because personnel time tends to be the most heavily expended resource for any digital preservation program. Are there currently members of your organization who have digital preservation responsibilities as part of their job descriptions? If so, who are they, and how much time can they spend on the effort? The amount of current knowledge and available time personnel can spend on the digital preservation program will drive your implementation strategy. A successful digital preservation program could be one full-time person with a high level of knowledge working on the program with support from the information technology department and the bibliographic management department on occasion. Another successful approach could be one or two people from every department with a medium amount of digital preservation knowledge working on the program as they can. These are just two of many potential scenarios. The key is that whatever personnel time and expertise allocation you develop remains sustainable in the long term. The size of your organization will determine how granular this part of the assessment will be. The intent is to determine if your organization has these resources. A potential way to document this assessment is by using your institution’s organizational chart. List your potential collaborators by job title, and document, for each person, if they are willing to contribute to the program, and if so, how much time would they be able to spend on it and what skills they believe would be useful to a digital preservation program. In this way, you can map out the current abilities of your collaborators, whether additional training may be needed, and the amount of participation you can expect from your collaborators in helping you plan your digital preservation program implementation strategies.

The assessment of available resources continues with an inventory of your organization’s current technology resources and standard practices. One of the best tools for this type of assessment in the National Digital Stewardship Alliance’s Levels of Preservation.2 The Levels of Preservation are currently undergoing a scheduled revision to bring the text up-to-date with current practice and to make the document more relatable to practitioners new to the field and collaborators whose main area of expertise may not be archives. The structure of the document—“a tiered set of guidelines and practices intended to offer clear, baseline instructions on preserving digital content at four progressive levels . . . across . . . different functional areas . . . focused on specific preservation actions”—will remain the same.3 The emphasis in the Levels of Preservation is on activities, but by determining what activities you have completed, you can also make an inference about what technology resources you currently have available. If you do the assessment with a colleague knowledgeable about your technology resources, your colleague may be able to suggest resources that are available but not currently utilized for digital preservation.

Finally, you will need to determine and document what financial resources, beyond personnel time and existing technological infrastructure expenditures, are available to fund your digital preservation program. Do not be discouraged if the answer is none. When starting a digital preservation program, it is more important to have personnel and technological resources available because so much of digital preservation is in planning, policy setting, and workflow creation using existing resources. When your program has been up and running for a while, you will have a better understanding of the gaps in your digital preservation system and then be able to request specific funds to fill those gaps and have evidence to support your funding requests.

The next stage in your assessment is to look at your organization’s mission, policies, and procedures. This survey should be done with an eye toward where a digital preservation program will support the mission or fill gaps in existing collection development policies, access policies, and so on. If existing procedures or workflows in the organization produce digital materials, those may need to be integrated into the new digital preservation program. For example, if your organization is creating permanent born-digital records, you need to know where those records are, how they are being saved in the short term, and what kinds of electronic formats they are being saved in so that you will have a better understanding of the preservation needs of the records. Similarly, if your organization is already creating digital surrogates of analog materials, you will need to understand how that process works, what the final formats of those digital copies will be, and if your organization intends to invest in preserving the digital copies over time.

Having completed all of your assessments, it is time to develop two sets of goals. The focus of the goals should be to advance your digital preservation program toward sustainability and further compliance with the standards I talked about in chapter 2. One way to create your goals is to map your assessments to one of the certification checklists or to the Digital Preservation Capability Maturity Model (DPCMM).4 For those just starting out, I suggest mapping to the DPCMM because it has only fifteen areas of performance, each explicitly requiring conformance to OAIS requirements, versus the one hundred plus requirements in TRAC or the data-specific nature of the CoreTrustSeal.

A capability maturity model, the heart of how the DPCMM is structured, is “a set of structured levels that describe how well the practices, processes and behavior of an organization can [reliably] and sustainably produce desired outcomes . . . [using] a series of associated activities and baseline metrics used to measure performance in a given area.”5 The DPCMM used the OAIS and TRAC ISO standards (14721 and 16363 respectively) to develop the performance measures for each of the fifteen areas. For each area, a digital preservation program can fall between Level 0 (nominal) and Level 4 (Optimal). There is a break between Levels 2 and 3 that requires the digital preservation program must fully conform to ISO 14721 in a sustained manner before Level 3 can be achieved. The fifteen areas are broken into two sections, Digital Preservation Infrastructure, which speaks to organizational commitment, and Digital Preservation Services, the processes required to actively preserve digital material. Using the Digital Preservation Capability Self-Assessment Scorecard, you can map your assessment results to the DPCMM and receive a scorecard with an overall score of the stage of your digital preservation program.6 Figure 3.2 is an example of a final scorecard. The Digital Preservation Capability Self-Assessment Scorecard was originally created and structured for records managers, so you will have to be a little flexible when answering the questions if that is not your institutional context.

You can now, using this map from assessment to requirements, set goals for raising your capability level in those areas you deem most necessary and most achievable with your current resources. This could be a set of fifteen goals, or it could be set of four, depending on your organization’s particular needs and abilities. Sustainable digital preservation programs require a balance between organizational infrastructure, technological infrastructure, and sufficient resources. When you are creating your goals, make sure to not emphasize one of the areas over the others. If you already have sufficient technology to meet your current needs, focus on goals that improve organizational policies and resource allocation. Digital preservation is all about how much risk you are willing to accept to your digital materials. If you are willing to accept a high level of risk, that willingness will be reflected in fewer technological goals. If you are willing to accept only a very low level of risk, then you will have a high number of technological goals.

When setting each goal, define a clear metric or set of metrics that, when achieved, will prove that the goal has been met. For example, if one of your goals is to go from DPCMM Level 0 (no access to digital preservation expertise) in Technical Expertise to Level 1 (minimal access to expertise), your metric could be to have one or more employees successfully complete a set of digital preservation courses.7 These goals and metrics will become the road map you use to create your digital preservation program implementation plan. The DPCMM maturity stages are cumulative. You do not truly move up a stage until you are able to implement and sustain all of the requirements of the lower stages. To truly build a sustainable program and to get the most out of the DPCMM as an assessment tool, I suggest your goals reflect moving all of the DPCMM categories to the same level. If some of your categories are at Level 0 and some are at Level 1, an achievable goal would be to move every category into Level 1 and sustain your abilities at Level 1 for a period of time before trying to move any of your categories to Level 2.8

The second set of goals will differ from your first set only in that you are creating aspirational goals. If you had more people, more time, and more money, what would be your ideal goal for each requirement and its associated metrics for achievement? It may not be achieving a Level 4 for every requirement. Remember, your digital preservation program needs to work for your organization and your community of users. This may mean a less complicated and less resource-intensive program than the ideal espoused by TRAC, the CoreTrustSeal, or the DPCMM. Instead, your aspirational goal may be that you meet the requirements for an end-to-end system where all digital content is being preserved and users have a path to independently access that content. This would mean that your goals are to achieve the minimum necessary for preservation but the maximum necessary for access. Another organization may put more time and resources into the preservation end because its policies require long-term restrictions on content before users are able to access the materials. In this case, the preservation piece is much more intensive because it is more difficult to recognize a possible preservation problem when materials are not being constantly accessed.

Your aspirational goals will set the stage for how you decide to advocate for your digital preservation program in the future. These goals and metrics will help you plan for future expansions to your digital preservation program. This plan will help you create an advocacy campaign to request new resources for your program. You can go to your organizational leaders with evidence of what you have already done and a plan for where the new resources will be expended with potential outcomes already laid out. You can also use your assessment of the organization’s mission and goals to show where the digital preservation program supports your organization’s leaders’ initiatives and plans for the overall organization. Your assessments and goals can also help you determine where collaborations with other organizations will be most beneficial and effective, especially if other organizations that you routinely work with have done a similar assessment. In fact, I encourage you to do these assessments at the same time as your partner organizations because you can benefit from the lessons others in your group learn from internal discussions with information technology groups and resource allocators in their own institutions. Common goals that result from these shared assessments may allow you to pool resources with other institutions to fill common gaps in everyone’s digital preservation program.

This is something that I have done with other academic libraries in the state of Montana. We worked together using a much leaner version of the assessment series that I have talked about in this chapter. We have been focusing on building the knowledge of member librarians and slowly increasing their organizational infrastructure to the point where they can request support for a digital preservation program. The result of the common assessment was a common need for digital preservation policies at every institution, either new or an updated version. This is the next step in creating a sustainable digital preservation program, and thus the next chapter of this report.

Notes

1. Mark Greene and Dennis Meissner, “More Product, Less Process: Revamping Traditional Archival Processing,” American Archivist 68, no. 2 (Fall/Winter 2005): 208–63, https://doi.org/10.17723/aarc.68.2.c741823776k65863.
2. Megan Phillips, Jefferson Bailey, Andrea Goethals, and Trevor Owens, “The NDSA Levels of Digital Preservation: An Explanation and Uses,” Library of Congress, 2013, www.digitalpreservation.gov/documents/NDSA_Levels_Archiving_2013.pdf.
3. Phillips et al., “The NDSA Levels,” 1.
4. Charles Dollar and Lori Ashley, Digital Preservation Capability Maturity Model (DPCMM), v. 2.7 (San Francisco: Tournesol Consulting, July 6, 2015), www.securelyrooted.com/dpcmm, under “DCPMM Background and Performance Metrics v2.7.”
5. Dollar and Ashley, Digital Preservation Capability Maturity Model, 8.
6. The Digital Preservation Capability Self-Assessment Scorecard is a service that you will need to register for and requires that you begin your self-assessment within seventy-two hours of registering. You may re-register at any time (DigitalOK login page, accessed June 5, 2019, www.digitalok.org); Dollar and Ashley, Digital Preservation Capability Maturity Model.
7. Dollar and Ashley, Digital Preservation Capability Maturity Model, 20.
8. Dollar and Ashley, Digital Preservation Capability Maturity Model, 8.

Figure 3.2

Sample Digital Preservation Capability Self-Assessment Scorecard. This is an example of a completed scorecard with the fifteen areas of performance individually scored and an overall score for the entire program.