In the 1970s, the United States Environmental Protection Agency sponsored Documerica, a large-scale photography initiative to document environmental subjects nation-wide. While over 15,000 digitized public-domain photographs from the collection are available online, most of the images were scanned from damaged copies of the original prints. We present and evaluate a modified histogram matching technique based on the underlying chemistry of the prints for correcting the damaged images by using training data collected from a small set of undamaged prints. The entire set of color-adjusted Documerica images is made available in an open repository.
Many of the most important environmental laws and federal agencies in the United States came
into existence during the large-scale political and social environmental movement that formed
during the 1960s and 1970s [
Documerica was an EPA-funded project running from 1972 to 1977 that aimed to
“photographically document subjects of environmental concern in America during the 1970s” [
Unfortunately, a damaged set of Documerica prints was used for digitization. The scanned
photographic prints have an intense red shift, causing everything from the sky, to lakes, to
trees to have a red/orange hue in place of their normal expected colors. The sharpness of
the images has not been significantly afected indicating that the damage was due to a slow
degradation over time through a combination of heat and light. The color shift is sufÏciently
strong to reduce the aesthetic and rhetorical power of the images that are available online [
In this article, we present and evaluate an algorithm to automatically correct the color of the
damaged digitized Documerica images using a small set of the undamaged prints as training
data. Our goal is to restore the aesthetic qualities of the images rather than the impossible task
of perfectly matching the colors in the reference images [
The photographers employed by the Documerica project took photographs using the
Kodachrome color reversal film produced by Eastman Kodak [
The National Archives of the United States has three copies of prints from Documerica: (1) archival copies held in cold storage and available on special request, (2) a non-circulating set held in their print room, and (3) a damaged set of prints that were used for the digitization process.1 The authors visited the National Archives and selected a set of 23 slides from across the collection to manually re-scan from both the cold-storage and non-circulating print set. We selected images that contained a variety of diferent colors. After determining that the cold storage images were indistinguishable from the print room copies, we decided to only re-scan the latter. We then manually cropped each of these images to match the digitized prints. One selected image had already been replaced with a corrected scan online. This left a training set of 22 images for which we had manual reference scans from the print room and the damaged, digitized copies on National Archives website. 1It is not entirely certain how the latter set was damaged or why it was selected for digitization. The third set was previously used as a circulation copy, and was likely damaged due to its continued circulation. Personal communications with the current archivists suggest that because the digitization was done of-site, someone selected the circulating copy as to send of before realizing how much it had become damaged.
Color adjustment of digital photographs is a well-studied topic and an essential element of
the workflow of modern photographers. For example, a common technique for professional
photographers is first to photograph a color reference card containing a variety of boxes with
known hues. Then, commercial software can use this reference image to learn how to adjust
any other images taken with the same equipment in similar lighting conditions [
In addition to commercial software for color adjustment, there has also been considerable
academic research on the specific task of matching the color between two images, a process
known as color mapping or image color transfer. In one of the earliest studies of color
mapping, Reinhard et al. introduced a technique consisting of a simple standardization of color
intensities in a specific color space [
Our approach to image color mapping with the Documerica collection is closely related to
the well-known histogram matching technique. In this technique we attempt to match the
distribution of colors in one image with the distribution of colors in a reference image through
a monotonic transformation of the pixel intensities [
First of all, through our understanding of the materiality of the prints, it is likely that the
color shift that damaged the prints likely afected the images along the dimensions of the three
developer colors. The contributing elements of heat, light, and/or humidity would afect each
of these dyes diferently according to their chemical composition. Our qualitative analysis of
the consistent red shift in the digitized images further indicates the need for a color correction
applied individually to each of the CMY color channels. So, in contrast to the standard strategy
for histogram matching that uses color spaces adapted to the sensitivity of the human eye, we
will apply our directly to the RGB/CMY pixel intensities [
The second diference from a standard histogram matching algorithm is that we want to avoid overfitting to the distribution of a single image. In fact, one of the goals of our transformation is to restore the diversity of lighting and colors that are in the source materials. Instead of perfectly matching the distribution for a single image, we want to create a transformation by averaging the histogram transformations across the entire training set of 22 pairs of images.
The goal of our adapted technique, which we will refer to as median histogram matching (MHM), is to learn three monotonic functions ̂ , ̂ , and ̂ that each map an input color intensity (from 0 to 1) into an output color intensity on the same scale. Applying these to the cyan, magenta, and yellow components of a damaged image should result in a transformed image that more closely represents the undamaged form of the print. In addition to being motivated by the chemistry of the prints, the transformation mirrors tools available in popular photo editing software such as the Photos application of macOS and Adobe Photoshop.
Diferences in the technology available for digitizing the undamaged prints—including the lighting, orientation, overscan size, and print-specific artifacts—made it infeasible to line up our reference images pixel-by-pixel. Following related work on color correction, we focused on lining up the general distribution of each color channel intensity. For each image and color channel , we computed the percentiles for the color intensities of both the damaged image and the scanned undamaged print. Then, we constructed a function ̂ by matching the two sets of percentiles to one another. For example, if the 20th percentile of the damaged image had a cyan intensity of 0.6 and the undamaged percentile was 0.82, ̂ (0.6)would be set to 0.82. We then set ̂ (0)to 0 and ̂ (1)to 1 and filled in the intermediate values through linear interpolation. Then, the final predictions for each color channel ( ̂ ) are given by taking the median across all of the images in our training collection. These transformations are guaranteed to be monotonic and to be well-defined for all input values.
outlined in the previous section. The grey lines show the estimates ̂ for each image , with the darker colored lines showing the final learned transformations based on the median across the entire training set. These transformations were also applied in the final column of Figure 1.
The cyan color channel transformation is the most diferent from an identity mapping. The transformation suggests that the amount of cyan intensity should be increased, which aligns with our qualitative analysis of the collection has having a general red tint (the intensity of cyan can be computed by taking the inverse of the red pixel intensity). The transformation indicates that the cyan dye as degraded at a rate faster than the other two dyes. The magenta and yellow curves, on the other hand, are much closer to the identity transformation. While the 2The representation of a pixel in CMY space can be computed by simply subtracting the RGB representation from 1. So, matching the CMY distribution is equivalent to matching the RGB representation. cyan magenta yellow algorithm does indicate some changes for individual images, these are relatively minor when we take the median across the entire training set.
We see that there are considerable diferences across training images. These diferences are less pronounced than they first appear because over half of all the pixel intensities in the digitized images have intensities over 0.8. Looking at the upper part of each curve, we see that these are much more stable than the rest of the transformation. Additionally, the transformations on the bottom of the curves correspond to parts of the image that have a small amount of the given color dye. Often these are very bright parts of the image that are close to white, and the diferences indicated on the chart have a small visual efect, as we see in the following subsection.
The diferences across training images highlight two points of caution. First, the adjustments are nowhere near perfect. The final exact colors can still be quite diferent than the original prints, the negatives themselves, or the light reflected by the objects being photographed. Secondly, in order to find a good transformation for another collection. Taking the median across several hand-selected transformations will likely work well, whereas applying a filter trained on only one image to a larger collection can quickly lead to overfitting.
We can use quantitative measurements to see how well the learned median transformations
work on the training data itself. In order to do this, we use a leave-one-out cross-validation
technique in which we compute the median transformations with the th image removed—
which we denote by −̂ () —and then compare this to the transformation given by using only
the th image itself [
To get have a sense of the scale of this metric, we will compare it with using the identity transformation to estimate ̂ . This metric treats all input color intensities equally. We also compute a weighted version of the metric weighted by the density of the colors in the image itself to provide a more accurate measurement of how much the two transformations difer on a specific image.
The results of the leave-one-out cross-validation are shown in Table 1. The cross-validated median estimator has an error rate about three times smaller than the error from using the input images. The errors are slightly smaller when weighted by the empirical distribution of the intensities, but the relative relationships remain similar. Only 1 of the 22 training images has a cross-validation error larger for the median estimator than the identity transformation, with every other image being improved by a factor of at least 50%.
One of the motivations for the work in this paper was our perceived ability to make the digitized, damaged Documerica prints look much closer to what we expected through manual color adjustments in commercial photo editing tools. One editor on the English version of Wikipedia, posting under the username Hohum, had a similar experience with the Documerica photographs. In January 2023, they uploaded manually corrected versions of 14 Documerica photographs to Wikipedia. No description of the method used was given. It seems unlikely that the user had access to the undamaged prints; they have uploaded thousands of other corrected images under a large variety of categories on Wikimedia Commons. While we should not treat the edits by Hohum as a gold-standard transformation that we should aim to replicate perfectly, comparing our method to the manually edited photographs ofers further quantitative evidence for the efÏcacy of our automated technique.
As we did in comparing the transformations in the leave-one-out cross-validation, we will look at the average distances between the reference image (here, the corrected version uploaded to Wikimedia Commons) to both the uncorrected image and our automatically corrected image. Here, though, we look at the average Euclidean distance at the level of individual pixels. Because we are interested in measuring the visual perception of the transformation, we will ifrst convert the images into two diferent color spaces designed to represent human visual perceptions: CIELAB and the CIELUV [15]. We report the results with and without considering the lightness dimension of the colors.
Table 2 compares our transformation to the independent, manual transformations performed by Wikipedia user Hohum. Across all four selected color spaces, the median transformation method is roughly twice as close to the manual adjustment than the original image. For all the photos, the distance from the median transformation is closer to the manual transformation in the CIELAB and AB color spaces. Only one image, which consists almost entirely of a ifeld of monochromatic corn shot from above, has a worse CIELUV distance in the median transformation.
By looking at a subset of the transformations applied outside of our training set, we can qualitatively describe how well the algorithm produces more realistic colors in the images. Creating more realistic images that are more aesthetically interesting and engaging is the ultimate goal of our work. Figure 3 shows 18 Documerica images and result of applying our transformation. The red shift in the original images is very noticeable in the first and third columns. The transformed images lose this red tone and become more color-appropriate. The image of the two people fishing and the group of people on a bus both show no elements that still have any red hues. On the other hand, images such as the sunset and the rocky clif with a bridge still retain their expected red characteristics. In other words, we have not overcorrected the images by making them all too blue. Instead, the updated collection shows a diverse set of colors that help to re-establish the true scale and scope of the Documerica project itself.
The Documerica project produced a large set of historically important documentary
photographs [