Next Steps Preservation Plan for the Archeology Program Office

OVERVIEW

The Archeology Program Office of the Prince George’s County Department of Parks and Recreation was established in 1988 to excavate, preserve and protect archeological sites in county parks. It is part of the Maryland-National Capital Park and Planning Commission. As part of its mission, the program curates millions of artifacts, and over the years, related documentation has been created in various formats and on disparate media. Documentation is primarily in the form of digital and/or physical copies of reports, catalogs, slides, print photographs, negatives, drawings, maps, and videos. All artifacts and documentation are stored in the same facility. Their goal is to have all digital content centralized on one shared network which is currently in development.

This report outlines recommended next steps to preserve their digital content following a framework designed by the National Digital Stewardship Alliance (NDSA). NDSA’s Levels of Digital Preservation presents a series of recommendations based on five areas of concern for digital preservation – storage and geographic location, file fixity and data integrity, information security, metadata and file format. Level 1 recommendations relate to the most urgent activities need for preservation and serves as a prerequisite to the higher levels.

STORAGE AND GEOGRAPHIC LOCATION

Establish a central storage system for all digital content and maintain at least one complete copy stored in another location.

Digital content is currently stored locally on five desktop computers, two laptops, a back-up drive of files from former staff, approximately 300 compact discs and about 700 3.5” floppy disks. Staff have been working on a shared drive to organize content in one place. While staff are backing up their own drives locally, there is no complete copy of all digital content.

Following NDSA’s Level 1 recommendation, staff should move all files off disparate media and into a storage system and create a complete copy of files that should be stored in a different location. This serves a number of purposes:

– Transferring to a new storage system will guard against potential loss of data on old media.

– Minimizing number of storage locations will facilitate data integrity monitoring.

– Creating a complete copy will provide a back-up in case files are corrupted or lost.

– Storing a copy in a different location will guard against loss specific to one location such as damage to equipment as a result of severe weather or a catastrophic event.

– This process is also a necessary first step in achieving a goal for the program, which is to have all files organized and accessible without extensive searching.

1. Create a complete copy of what is accessible right now.

The development of the shared network in coordination with the program’s IT department will likely serve as their established storage system. A portion of files are currently inaccessible on floppy disks. However, the bulk of digital content is stored locally on various hard drives.  Any content that is currently accessible on hard drives should be copied and stored in another location to safeguard against loss. See “File Fixity and Data Integrity” below for checking data integrity and creating a file manifest. In the short term, storage can be on an external hard drive that is kept in another location in a secure place. Alternatively, staff could use a cloud storage service such as Dropboxwhich would have the added benefit of storing files offsite. Once files are integrated into the shared drive, a copy can be created from this set of content. Staff can work with the IT department to see if they can use a back-up storage system that is already in place with other departments.

2. Establish a file structure in the shared directory

Staff can begin integrating current files that they work with into the networked drive in order to develop a workable structure that will be practical for finding files. Historical documents can then be organized into that structure. Once a structure and naming system has been established, staff should document and follow this structure. This can be done concurrently with the following step.

3. Copy files from compact disks and 3.5” floppy disks using an unnetworked workstation

It will likely take a while to sift through decades worth of files. Given the urgency of potential loss from old media and a potential risk of viruses from disks used by former staff, copy over all the files from compact and floppy disks first onto a an unnetworked drive and run a virus scan before combining with current files.

a. Drive space needed: Assuming approximately 700 MB of data per CD and 1.44 MB per floppy disks, there is approximately 211 GB of data if all the disks are full.

b. Install anti-virus software on unnetworked workstation: Speak with the IT department about installing antivirus software so that it can run in the background while copying over files. Establish a schedule to run full virus scans depending on how regularly files are being copied over.

 c. Install drivers for 3.5” external floppy drive on unnetworked drive: At the time of the survey, staff had an external drive to read 3.5” floppy disks but lacked the software to run the drive. This step is necessary in order to access any of this information.

d. Install fixity software on unnetworked and networked drives: See the section “File Fixity and Data Integrity” below.

e. Transferring files from CDs and floppy disks to unnetworked hard drive: Currently staff use context such as file extensions, file location, file name, and the author name to find content. Therefore, it’s recommended that they preserve these elements and related descriptive information that may be written on the disks as much as possible until they can sort out all the content. Disks were labeled in a number of ways. The majority of floppy disks where organized in boxes of about 10 disks with labels on the boxes suggesting that there were groups of related disks. Some disks had vague or no labeling. Related files kept together on disks or boxes of disks should be copied into a directory folder with the descriptive information from the label. This can either be transcribed onto a text file or a picture can be taken of the label and included in the directory.

f. Timing: While each floppy disk may hold a relatively small amount of data, the process of copying over hundreds of disks can be labor intensive. However, during this window, the work is not being backed up. Therefore, consider either setting a short window of time to copy over all of these files, or setting a schedule where small batches of files are copied over from disks and scanned for viruses before being copied to another location. This can either be added to shared drive or another external drive that is stored in another location. Once files have been safely transferred, they can be integrated into the file structure on the shared drive.

4. Set a schedule of back-ups to maintain a complete copy.

METADATA

The NDSA Level 1 recommendation it to create an inventory of digital content and storage location. Like the digital content itself, the inventory should be backed up and stored in another location. As mentioned in the Section “File Fixity and Data Integrity” below, AVP’s Fixity software includes a function to generate a manifest of file paths that will assist with inventory. Staff can maintain this inventory as they continue to develop the file structure on the shared drive.

For file and directory naming, consider creating a controlled vocabulary and syntax to make it easier for staff to find files. This can include specific terms for archeological site names, document type (e.g., site form, report), and a version, year or other modifier (e.g., draft, final) when needed.

FILE FIXITY AND DATA INTEGRITY

File fixity is a way of ensuring that files have not changed. It is recommended to run fixity checks whenever files are transferred (Owens, p. 110). This will generate an alphanumeric string called a checksum that can be compared before and after the transfer. Changing the content of the file including the format will change the checksum value. If the IT department does not already use fixity software, Fixity is a free tool from AVP that can be used to generate and compare checksum values and ensure that all files have been transferred. The software also generates a manifest of file paths along with the checksums that could prove useful in establishing an inventory of digital content.

The Level 2 recommends virus checking high risk content while Level 3 recommends virus checking all content. Virus checking high risk content is addressed 3b of “Storage and Geographic Location.” Staff should have antivirus software installed at their workstations and run scheduled scans.

Level 3 also recommends checking fixity as fixed intervals to ensure data integrity over time. Consider establishing a yearly schedule of validating fixity. Any corrupt or missing files can be replaced with a copy that passes fixity validation.

INFORMATION SECURITY

This step will outline who has access to the content and what they can do with it. This will prevent files from getting deleted or changed by unauthorized staff. NSDA Level 1 recommends to identify who is authorized to read, write, move or delete individual files. Related to this, Level 4 in the section on file fixity also recommends that no one is authorized to have write access to all copies. This reduces the likelihood of changing or deleting all copies of one or more files.

Staff have taken initial steps in the process by creating three different directories with different levels of access for their users: one directory for onsite archeology staff, one directory for the rest of the Prince George’s County Parks Department, and one directory for Dinosaur Park, another program of the Prince George’s County arm of the Maryland National Capital Park and Planning Commission that shares the same workspace.

However, more levels of access may be necessary if only one person in the office is allowed to have read or write permissions. In addition, staff should clearly delineate working files from historical files that should not change. This will help to prevent the document from being changed or deleted. It will also help with fixity validation since working files will likely involved changes in content which will change the checksum value. This can be accomplished by setting permissions to specific subdirectories or to specific sets of files. Document access restrictions and store in a location that all users can access.

FILE FORMAT

 File formats can become obsolete. In some cases, once the format is obsolete the file might not be able to be opened in another format or will not be rendered in exactly the same way. The purpose of this section is to minimize these problems by using formats that are less likely to become obsolete, or that can be effectively rendered in another format. Widely used formats are generally considered to remain accessible because there will be a demand to either keep them accessible or develop a means of migrating them (Owens, 121).

Since some media have not yet been accessed, a current inventory of file formats that have been used is not available. Formats currently in use are jpegs andfiles generated from different versions of Microsoft Word, Excel and Access. Mapping files are created using GIS (geographic information systems) technologies. Older files have been created using WordPerfect, CAD (computerized aided drafting) software, and Paradox relational database. Staff are currently having trouble with opening Paradox files since the database is no longer supported and cannot be opened using current versions of Access or Excel.

Formats such as jpeg and Microsoft Word and Excel are commonly used, although the latter two undergo regular updates which may render slight changes if a file is opened in a new version. As the files are incorporated into the new directory structure on the shared drive, staff should develop an inventory of formats that they are using, work with the IT department to monitor them for obsolescence, and be prepared to migrate as needed.

FUTURE STEPS

NDSA Level 2 for storage recommends creating a third copy of the content and Level 4 recommends at least three copies stored in locations with different disaster threats. The Archeology Program Office could combine this recommendation with a means of sharing some of their content. This could be through a subject-specific repository for archeology or something more general like the Internet Archive.

Levels 2-4 address steps to maintain storage media so that files continue to be accessible in the long term. Staff should work with their IT department to document storage used for their shared drive and back-up copies, monitor for obsolescence, and have a plan in place for updating systems.

Staff also expressed an interest in resuming digitization of their physical documentation. Some reports and slides have already been digitized. As a starting point, staff can discuss their experiences with past efforts and lessons learned to establish goals for the program and how this will fit into the file structure that they are creating for current digital content. The Still Image and Audiovisual Working Groups of the Federal Agencies Digital Guidelines Initiative can be a good resource to establish best practices for digitization.

Survey of the Archeology Program Office for Prince George’s County Parks and Recreation

Overview

 The Archeology Program Office of the Prince George’s County Department of Parks and Recreation was established in 1988 to excavate, preserve and protect archeological sites in county parks. It is part of the Maryland-National Capital Park and Planning Commission. As part of its mission, the program curates millions of artifacts, and over the years, related documentation has been created in various formats and on disparate media. The office has been located in a house in Upper Marlboro, Maryland, since around 1999/2000 where all artifacts and documentation are kept. It is currently staffed by three archeologists who are the primary users of the documentation.

The goal is to have all digital content centralized on one shared network which is currently in development. Ideally physical copies would be digitized. This content would likely have variable levels of access; some files could be made available to the public through a research portal while some would remain for internal use only.

Scope of Holdings

Documentation is primarily in the form of digital and/or physical copies of reports, catalogs, slides, print photographs, negatives, drawings, maps, and videos. The following includes a description of digital content and some of the physical holdings since there is interest in digitizing them.

Digital storage media

Digital content currently resides on five desktop computers, two laptops, and a back-up drive of files from former staff. The three current staff people also keep back-up drives at their desks. Content is primarily kept and backed up locally, but the staff are just beginning to test a shared network.

Additionally, files are also stored on approximately 300 compact disks and 700 3.5”floppy disks. This may be an underestimate as some disks are kept with hard copy reports. The disks are labeled but not necessarily enough to discern the contents. Approximately 600 of the 3.5” floppy disks are kept in boxes with a consistent labeling format. Staff have an external floppy drive provided by an offsite IT department to read the 3.5” disks but the drivers have yet not been installed.

Reports and catalogs

While the exact contents of the storage media are not known, staff anticipate that many reports and catalogs were originally written with WordPerfect. More recent reports are written using Word. Versions will vary. Catalog data was originally kept using Paradox relational database software and staff currently have no means of reading these files. Current catalog information is kept on both Excel and Access. There are approximately ten shelves of archeological site reports (25 linear feet) and four filing cabinets of site-related files.

Photographs, contact sheets, negatives, and slides

Digital images are saved as jpegs.These are both born-digital and digitized from a portion of the photograph and slide collection. There are two shelves (approximately 5 linear feet) of binders containing photographs, contact sheets, negatives and slides. Descriptive information is written on the dividers and on the backs of some photographs. There is also a flat file drawer of matted photographs.

Maps and drawings

Mapping files have been created using CAD (computerized aided drafting), and more recently, GIS (geographic information systems) technologies. In addition, there are fourteen flat file drawers of maps and drawings.

Video

There are approximately twenty VHS tapes the contents of which are unknown.

Current Management of Digital Holdings 

Digital content on floppy disks is inaccessible at the moment. Current staff are relying on institutional memory and lengthy searches to find files from former staff stored on a back-up drive. This is based on an understanding on what the person’s role might have been and when they worked there. At times, searching based on file extension can provide a hint at the content.

Most digital content is still stored and backed up locally, but staff are testing a new shared drive with directories allowing different levels of access for onsite staff and the rest of the department. Within the directory for onsite staff, subdirectories currently relate to specific archeological sites, but the exact organization has not yet been determined since content can be classified in more than one way. Implementing this shared network offers an opportunity to integrate disparate digital holdings and would facilitate creating a back-up copy in a different location.

Perception of the State of Digital Content

The perception from the archeology staff is that there is digital content that will add either known or potential value to their mission. This content is currently hard to find, on inaccessible media, or in unusable formats. Identifying and moving these files to one drive would: save staff time by making the files easier to find, save space currently taken up by heterogeneous storage media, eliminate duplicates, offer an opportunity to keep all information about specific sites or artifacts under one subdirectory, and prevent irreversible loss.

Resources

The Archeology Program Office will be able to benefit from the technical expertise of the Prince George’s County Parks and Recreation IT department, which provides and maintains the staff’s equipment. In addition to the equipment mentioned above and the shared network, the staff have two scanners that have been used to digitize slides, files and print photographs in the past.

The IT department also regulates information security, one of the key preservation activities that will be addressed in the coming preservation plan. This does, however,  limit the types of software staff can download themselves – the drivers for the external floppy drive mentioned above, for example. Additionally, they may serve as a resource in implementing another of the key activities – data integrity checks.

The program also benefits from a small but dedicated staff who are vested in maintaining the integrity of this digital content. The effort is being spearheaded by the Assistant Archaeology Program Manager. The level of effort will likely be limited to a couple of hours per week until a student intern or volunteer can be hired and trained to do the work.

National Archives, The Digital Vaults

The National Archives is probably the most well-known archive in the United States. However, most people only ever see its most famous documents on display, The Constitution, The Declaration of Independence and the Bill of Rights. While these are the foundational documents of our country, the archives housed thousands and thousands of other pieces that can tell us a lot about the history of our country. Most of these documents are collecting dust, only seen by a lone researcher every decade or so.

                However, recent digitization efforts for the collections at the National Archives are changing this situation. Digitizing records allows greater access for researchers who may be otherwise restricted by travel and financial considerations. Additionally, digitization really opens up the archives for the general public. It allows those people who would normally only enter the National Archives through the front to digitally march through the researcher entrance and explore what they can find.

                The National Archives Digital Vault allows visitors to browse their collections through tagging. Records are organized by tags and you can follow one to another through these connections. In fact, this is the premise for their pathways game, which starts with a record and provided clues to help to find a related document. It is meant to highlight the different ways that documents can be connected to one another. The site also has the option to create your own collection. Any records of personal or research interest can be dragged into a separate space and saved as a collection.

                In addition to tagging, you can filter through documents for type or time period. This is helpful for visitors with specific types of documents or subjects in mind to easily find items. I also really liked this feature because it maintains some of the importance of physically visiting an archive. Oftentimes researchers stumble upon records at archives that they may not have been specifically looking for, but are relevant to their research topic. By showing a number of records related to the one someone is looking at prevents this from disappearing entirely.

                The site also offers visitors the opportunity to create their own products. For example, one could create their own pathways challenge. Visitors can also create movies or posters using the documents that they have saved in their own collections.

                I feel that these types of tools serve multiple purposes. They open archives up to the general public and allow them to explore records they would most likely never see otherwise. In this sense, the design is very accommodating to browsing without a specific topic in mind. However, it also has the features necessary for a more focused search, thereby allowing the utility of the digital collections to be extended to serious researchers as well.

Digitization, Digital Preservation, and File Formats

So I successfully scared everyone off from blogging about the The NINCH Guide to Good Practice in the Digital Representation and Management of Cultural Heritage Materials. If you get a bit lost in this reading I suggest reading over mwest24’s post on it from last year.

Along with the post I would suggest the following vocabulary terms that we should all become familiar with. I will weave these terms together into a bit of a how computer files and storage work discussion that will help set us up for Kirschenbaum’s book.

As this is technical info, I am just going to crib most of it for Wikipedia. Don’t get too lost in the details, but please read these over and click the links out to Wikipedia if you have no idea about some of the terms.

Key Terms for File Characteristics 

Dots per inch (DPI) is a measure of spatial printing or video dot density, in particular the number of individual dots that can be placed in a line within the span of 1 inch (2.54 cm). The DPI value tends to correlate with image resolution, but is related only indirectly.

character encoding system consists of a code that pairs each character from a given repertoire with something else—such as a bit pattern. In our work, the most useful things to know about are ASCII and Unicode.

Data compression, source coding, or bit-rate reduction involves encoding information using fewer bits than the original representation. Compression can be either lossy or lossless. Lossless compression reduces bits by identifying and eliminating statistical redundancy. No information is lost in lossless compression. Lossy compression reduces bits by identifying marginally important information and removing it.

Embedded Metadata: Metadata that is embeded inside a given file instead of existing outside the file in some other database or something. Examples include ID3 tags for MP3 audio files and Exif for image files.

Storage Terms

binary file is a computer file which may contain any type of data, encoded in binary form for computer storage and processing purposes; for example, computer document files containing formatted text

bit (a contraction of binary digit) is the basic unit of information in computing and telecommunications; it is the amount of information stored by a digital device or other physical system that exists in one of two possible distinct states. These may be the two stable states of a flip-flop, two positions of an electrical switch, two distinct voltage or current levels allowed by a circuit, two distinct levels of light intensity, two directions of magnetization orpolarization, the orientation of reversible double stranded DNA, etc.

The byte (play /ˈbt/) is a unit of digital information in computing and telecommunications that most commonly consists of eight bits. Historically, a byte was the number of bits used to encode a single character of text in a computer[1][2] and for this reason it is the basic addressable element in many computer architectures.

disk image is a single file or storage device containing the complete contents and structure representing a data storage medium or device, such as a hard drivetape drive,floppy diskoptical disc, or USB flash drive. A disk image is usually created by creating a complete sector-by-sector copy of the source medium and thereby perfectly replicating the structure and contents of a storage device.

File Types:

Documents: Of particular importance for us are .txt .doc .pdf .xml and .html

Images: Of particular importance for us are .jpg .tiff  and JP2000

Audio: Of particular importance for us are .mp3 .wav

Digital Video Encoding: this one is tricky, we will talk about .mov .mpg .swf .mp4 and .avi

http://www.digitalpreservation.gov/formats/fdd/descriptions.shtml

Print Project Proposal: 9/11 and Online Archives

The print project that I am proposing for this course stems from my interest in the role of digital media in the evolution of cultural memory. The central question driving my attention to this area of study lies in discerning whether or not technology is having a significant impact on public discussion and collective understanding surrounding the remembrance of historical events. By exploring more recent history, in particular the destruction and loss of lives in three American cities on September 11, 2001, it becomes possible to explore how the expanding digital humanities movement is changing our understanding of the archive. The proliferation of born-digital content leading up to this national tragedy has resulted in numerous online archives dedicated specifically to this event, as well as the availability of materials in special collections as part of larger projects. These archives espouse vastly different purposes, aggregate varying types of content, and originate from a variety of civic, federal, commercial, and individual sources.

In surveying the websites yielded by appropriate keyword searches, I hope to create a clearer picture of how the archive is being enacted online. For each site, I will attempt to provide a concise summary of what the stated goals for the project are, who funds and maintains its offerings, how content is collected, whether content is limited to a specific subset or leans toward universal collection of relevant artifacts, how content is organized and presented, whether user-generated content is allowed or encouraged, what types of policies are in place regarding access and responsibilities for long-term upkeep of the collections, and whether the site appears to curate their materials with or without bias. It will also be useful to explore whether or not these online archives appear to have implied audience, either professional or amateur, and whether the low cost of online broadcast opens up the mnemonic discussion to minority voices such as conspiracy theorists and 9/11 deniers.

I am further interested in seeing what role the traditional physical objects associated with historical practice have found in online archives. Not having yet delved into the research, I would assume that these sites are dominated by born-digital content that is easy to upload and manage while tangible objects languish under the same time and resource constraints limiting how quickly they can be documented and processed for viewing online that are currently affecting the digitalization of historically offline archives of pre-digital artifacts. Where relevant in the case of websites that simultaneously offer physical access to collections, as is the case with the National Archives, I will discuss this divide between offline and online at additional length by looking at differences in policies for access of materials and how much of the total content is available online.

Finally, the diversity of online archives presenting content relevant to this particular historical event includes some that allow user comments and/or reviews of specific content, as well as usage or download statistics. Whenever possible, this information will be included and discussed in hopes of sketching out how the content is being used and how those users or site visitors identify themselves in relation to the material.

This paper will work in concert with another paper that I am preparing this semester that will look specifically at policy issues surrounding user-generated content in the online archive. Hopefully, these attempts to create a tentative framework of how online archives currently function will underwrite future research into what effect broader access to these primary materials has on the shape of the public discourse of cultural memory.