Project Overview
Phase 1 resulted in a functional digital data archives system at the Shoreline Historical Museum (SMH), stored in an Adobe Lightroom database system. This system supports full search and image processing functions, and greatly streamlines and empowers research of the archives. It integrates existing metadata with the existing digital image archives at SHM, establishes a system for updating and correcting this metadata, and performs these updates on a significant portion of the existing archives. Phase 1 was accomplished over a period of 6 months.
Phase 2 made all the digital assets contained in the archives available online via the SHM website. Users are now able to search online, pan and zoom, and request high resolution versions of any image. This part of the project was developed by Barry Hansen with Electromagnetic Software.
Project Origin
I became familiar with the Shoreline Historical Museum while working on an interactive public art project with David Francis and 4Culture. We were creating an augmented reality experience where a user could move their phone around at Shoreview Park in Shoreline, WA to show historical photos that corresponded with where the phone was pointed.
However, during the process of working on this project, we found that the photo retrieval system at the Shoreline Historical Museum was in need of help.
I was hired to manage the project, study the photo collection, and establish a keywords hierarchy to make the photos searchable from every imaginable angle. I produced a “data dictionary” to assist everyone in cataloging the photos, and trained five volunteers to use Photo Mechanic to process metadata.
Imagery from the augmented reality art project
Problems with the Original System
The original systems for ingesting, storing, and displaying digital imagery at Shoreline Historical Museum evolved organically over many years, and without adequate planning or budget. This is a typical situation for many small non-profit museums, but it was a pressing problem for the SHM, since so much of their collection was in the form of historical photographs.
The related processes of document intake, scanning, and meta-data collection were mixed between three separate computer systems, together with a hard-copy intake master catalog. Because of this situation, activities that normally would proceed quickly became very time consuming and error prone. For example, the metadata (information about each photograph) was on a stand-alone system with no graphics capability. That meant that volunteers had to enter metadata for a photograph without the benefit of seeing a digital image on the screen at the same time. Instead the operator must rely on information that was entered in the hand-written master catalog, where it might have been logged by a person not familiar with the photograph or the historical significance. There was no method for checking or updating the metadata in this standalone system without retrieving the physical document from storage, an activity which would have been extremely time consuming.
There was a separate machine for storing and displaying images to both museum staff and the public, and that machine was not connected to the system containing the metadata for the images or to the master catalog. When a researcher arrived at the museum, they first sat at the metadata computer and searched for photos using a spreadsheet system, and without being able to view the photos.
Photo numbers (which were sometimes duplicate or inaccurate) were then written down and taken over to the computer containing the digital images, and then these had to be searched for manually using the Windows OS. Usually, a SHM staff member accompanied the researcher because it was possible for a user to inadvertently overwrite or delete photos on the computer.
In short, the system for storing, maintaining, and making historical photos available to the public at SHM was very much in need of a redesign and an update.
The original intake workflow at SHM
Approach
Our approach can be summarized in 8 basic tasks, as follows:
1. Implement the automated backup procedures.
2. Develop the metadata plan. This was a key task and was performed by Ken Winnick working in close consultation with former museum director Vicki Stiles. A proper metadata design ensures an effective searching capability for the thousands of records housed at SHM.
3. Install the software on the three SHM machines.
4. Bulk process the existing metadata, and attach to the existing image files. This step took advantage of the advanced automation abilities of Photo Mechanic. We knew that the error rate of the existing metadata was high, but it was a good starting point nevertheless.
5. Once the metadata was attached, the entire archive was imported into Lightroom. (It should be noted that an archive of 3,000 – 10,000 images is considered “small” for this system, so archive size was not a problem.)
6. Develop and set up the manual metadata updating workflows on each of the SHM’s three computers
7. Working with volunteers, process the bulk of images contained in the archive. We processed about 2,000 images in the 6 months of this project. Processed images included complete metadata and have been updated in the Lightroom database.
8. Establish the tools and workflows for capturing new materials coming into the archives. These workflows replaced the existing methods.
Workflow
To start the brainstorming phase, Vicki Stiles, Executive Director of the Shoreline Historical Museum, and I created a keyword hierarchy using a "Mind Mapping" software. Following Information Retrieval Best Practices, we created four primary categories (Attributes, Actions, Agents, Entities) and expanded upon them based on the content of the collection and what people might search for.
In the screenshot below, you can see examples of nested keywords in the hierarchy: clicking on the "+" next to AGENTS (upper left) opens a category for Domesticated animals → Livestock → Cows.
Brainstorming a controlled vocabulary of nested keywords to import into Photo Mechanic for taggers to use
Once we developed our keyword hierarchy, we exported it as an Excel spreadsheet to maintain the structure and imported that to Photo Mechanic. Thus, photo taggers are able to categorize images with a "taxonomy" or controlled vocabulary, which keeps tags from getting disorganized or unintentionally divided. (For example, a controlled vocabulary keeps photos tagged with "cat" from being separated from photos tagged with "cats", "CAT", "felines", etc.)
Example of using Photo Mechanic to add metadata to a photo, tagging it with "Technologies", "Vehicles", and "Automobiles" as well as details about the type of photo, location, clothes the person was wearing, etc.
Linking three workstations with Photo Mechanic to feed metadata into a shared, searchable Lightroom repository
SOFTWARE
1. All metadata processing used the industry standard Photo Mechanic software, by CameraBits. This software is known for its speed, capability, and low resource requirements. It is the go-to software used by virtually all news reporters worldwide. We customized the software, especially with respect to a “controlled vocabulary” for metadata. (With the previous system, when a user searched for “Lake Forest Park”, they were presented with images that have any of those words—Lake, Forest, or Park. With the new system, they will only get what they expect, that is, images from the city of Lake Forest Park.)
2. Adobe Lightroom is used for the image processing and image database system. This is an extremely capable and widely supported system, and fully meets the needs of SHM. The image processing functions such as image enhancement, sharpening, and JPG creation are all useful. The database element offers an extremely powerful search and organization tool, including for creating “virtual collections”.
The software listed above were selected to satisfy the following key requirements for the system:
- Ability to incorporate and correct metadata that is currently stored separately from the images
- Ability to automate and distribute the metadata and image capture processes
- Ability to fully search the archives based on any number of parameters
- Ability to support research projects as collections of files, without disturbing the underlying database
- Ability to generate appropriate JPG files quickly and easily for use on the web (Phase 2)
- Easy to use, easy to maintain, cost effective, capable, and widely supported
- Ability to automate and distribute the metadata and image capture processes
- Ability to fully search the archives based on any number of parameters
- Ability to support research projects as collections of files, without disturbing the underlying database
- Ability to generate appropriate JPG files quickly and easily for use on the web (Phase 2)
- Easy to use, easy to maintain, cost effective, capable, and widely supported
Open-source Web Server Code
Many other museums have found that existing photo archive systems are expensive and cumbersome. We are excited to share the system we developed with other museums/cultural institutions to use FOR FREE as an Open-Source project: Lightroom Nested Keywords.
Future Goals
You can currently filter by a time period, such as the 1960s, to understand what life looked like in Shoreline at the time. However, it's limited to images in the SHM archive.
Can we build a system that allows users to simultaneously search photo archives from the Shoreline Historical Museum and other local institutions, like the Black Heritage Society? This would add more diversity to the SHM's archives to give a more complete idea of what was happening in different communities at different points in time.
(This time period relates to my Edwin T. Pratt Legacy Archives project, as Pratt was doing local civil rights work in the 1960s.)
Filtering by the 1960s in only the SHM archive
What if we could search multiple local archives?
SPECIAL THANKS
Vicki Stiles, previous Executive Director at Shoreline Historical Museum, for her astute guidance and never wavering confidence in this project.
Barry Hansen, for developing the web interface for the project and offering it as an open-source project.
Gary Carlson, Principal at Factor Firm in Seattle, for generous collegial input to the project.
It was an honor to receive a generous grant from the Pendleton and Elisabeth Miller Charitable Foundation. The Foundation makes grants primarily to civic and cultural institutions that serve the general public, with a focus on Pacific Northwest history. By providing an opportunity for us to do the work of reaching a larger body of researchers with better collection accessibility, they have truly demonstrated their commitment to community heritage.
Volunteers: Howard Fox, Barry Hansen, Laurie McCarthy, Karen Mogster, and Sally Yamasaki
Me (plaid shirt, middle right) with the dedicated volunteers at the Shoreline Historical Museum
TRY IT YOURSELF!