| Home | Syllabus | Next |

1. Imaging Defined
2. Imaging Defined Again
3. Optical Imaging

An image is a homomophic representation of an object. Representations range from abstract ("my computer") to iconographic () to precisely homomorphic. An image differs from a representation which simply identifies an object in that an image contains and presents spatial information drawn from an object. The icon is not an image of my computer because it contains no information drawn from my computer. The concept of space is as important as the concept of information to the definition of imaging. My computer is 17 inches by 16 inches by 7.5 inches and it has green indicator LEDs. This is information drawn from my computer, but it is not an image because it does not exist in a space and is not presented spatially.

Let us formally define imaging as follows:

A topological space is a space in with a concept of neighborhood. Such a space is necessary to define distances between points and to have a concept of near and far points. "Well-ordered" means that the spatial arrangement of object information in the image must be drawn from the spatial arrangement of information in the object. (Under some projection or transformation, points that are near neighbors in the image must be near neighbors in the object.) Note that the image space need not be the same as the object space and that the mapping from the object function to the image function need not be one-to-one, invertible, or uniquely determined from object data.

Based on the above definition, are the following examples images?

The Census of the United States (Answer)

A Distorted Photograph of My Computer (Answer)

René Magritte's Clairvoyance (Self-Portrait) (1936, oil on canvas, Galerie Isy Brachot, Brussels) (Answer)

René Magritte's The Forest (1926, oil on canvas, Musée de l'art wallon, Liège.) (Answer)

The heart rate as a function of time of people named Joe. (Answer)

The heart rate as a function of time of people named Joe standing in line for basketball tickets. (Answer)

Why is such a precise definition of imaging necessary? The history of image capture and creation has three major stages:

1. Images were first captured by human observation and created by drawing or painting.
2. Images were captured by analog physical processes, as in a camera, and created by tracing or photo-chemical processes in film.
3. Images were captured by automated sensor arrays. The image capture space may correspond to the image space, as in a camera, but this is not necessary. In many cases, such as magnetic resonance imaging or x-ray imaging, the capture space has a Fourier relationship with the image space. Digital processing is used to transform captured information and create the image.

In progressing through these stages, the concept of an image changes. The imaging system in the second stage may capture views not directly observable by humans (such as colors outside the spectral range of the eye). Imaging systems in the third stage may capture views not directly observable by any direct observer, such as the dynamics of processes in the human brain (which can be imaged by magnetic resonance.) Our definition of imaging is meant to distill the common theme of these different approaches.

Until the nineteenth century, artists were the primary imagers. Precise imaging is not the primary goal of the artist. Artists are more interested in creating new objects to highlight abstract ideas or in using apparent images to highlight non-physical attributes of existing objects. The history of art and artistic representation can be explored at a number of sites on the web. The History of Art Virtual Library is a good starting point. Art is also a useful vehicle for exploring the difference between objects, images and digital images. Non-artistic imaging focuses object data capture. Once the data is captured on a negative or in a computer, the image is regarded as infinitely reproducible. An artistic image, in contrast is regarded as unique. The focus in this sense is on the image as an object.

Recently, there has been considerable interest in digital capture and distribution of art. Digital reproductions may be very precise images of a piece and are easily distributed. While a digital representation of a painting may contain exact information about a painting, however, as an object it is quite dissimilar from the painting itself and is less useful in studying the painting than an artistic reproduction. A nice discussion of relationships between digital imaging and art is presented in Introduction to Imaging at the Getty Information Institute.

The invention of photography changed imaging profoundly. For the first time, images where reflected physical processes, rather than human observation. According to  A History of Photography From its beginnings till the 1920s by Robert Leggat, the first successful photographic process was the Daguerreotype, which was announced on 7 January 1839. True photographic processes built on earlier artistic homomorphic recording processes, such as the camera obscura. Over the 150 years since the invention of photography, continuing advances in film speed, color recording and sensitivity, and optical equipment have dramatically improved the fidelity with which photography replicates the world, but the basic idea behind photography is the same today as it was then.

Four particularly profound advances in remote imaging have been introduced in the 20^th century: movies, video, holography, and tomography. Here is some perspective the significance of these techiques:

• Movies are images of space-time phenomena. They represent the first, but not the only, example of 3 or more dimensional mechanical image recording.
• Video is important not for the cultural destruction it has wrought, but as the first example of electronic imaging. It is interesting that electronic imaging was invented to allow television, not to allow image processing. Early systems provided no means of storing the transmitted image. It is even more interesting that in the age of multimedia, digital systems are likely to replace television. Discussions of the history of video are available from a number of sources. A particularly interesting version is on line as the Farnsworth Chronicles.
• Holography is the most profound demonstration to date that there is more to see in the optical field than the human eye detects. The History and Development of Holography is briefly covered on line by Holophile, Inc.
• Tomography is the most powerful and least well defined of the four major advances. The etymology of the word comes from the standard "graphic" root and the Greek word for slice. While the most common use of the word remains rooted in x-ray computer tomography, a number alternative uses in magnetic resonance, radar, and seismology have arisen in recent years. In the past five years, "optical coherence tomography" has also been developed, although this is not really a slice-selective technique. The importance of tomography is that it finally breaks the link between image formation and feature capture. Tomographic systems in many cases are completely computational and form homographic projections without analogy.

In the twenty first century, one may expect a revolution in imaging arising from the combination of the four major advances listed above. The keys to the revolution are dramatic improvements in video image capture devices, holographic and computational algorithms, and computational hardware. The invention, in the mid-1970s, of the charge coupled device (CCD), caused a quiet but profound shift in optical imaging. Prior to the CCD virtually all still images were captured by photochemical films, but CCDs and similar sensors are now the ubiquitous choice for scientific imaging. Scientists choose CCDs because digitized data from an electronic sensor array can be mined for quantitative understanding much more effectively than analog patterns on film. The shift of non-technical imaging markets to digital technology has proceeded slowly as CCD and computer technology has improved. High quality digital cameras have appeared in consumer market just in the past year. As the availability and power of image analysis computers and image transmission networks increases, one expects CCDs to replace film in technologically advanced countries. To learn more about electronic imaging hardware, one might visit Kodak, Agfa, Ricoh, or Canon.

There is a deeper significance to the CCD revolution than simply replacing film, however. Just as electronic text has shifted from the serial structure of printed documents to hypertext, computer images have much greater potential complexity and information capacity than printed images. An interesting example of the potential of electronic images is provided by the photobubbles created by Omniview.

An imaging system consists of three components: the object, a medium for transferring object information, and image detection/creation hardware. These three components are shown schematically below.

As shown in the figure, the information transfer medium may correspond to the optical or x-ray field or other propagating wave. This physical view of imaging corresponds closely to the second stage process defined above. In general, the three imaging system components need not exist in the same space, in which case a figure like the one shown above is not possible.

At the most abstract level, imaging is synonymous with following inverse problem:

This definition maintains the idea of the imaging system consisting of an object, information transfer and image formation. In the traditional case of a film based optical imaging system, both the measurement and the source are continuous functions of space. The source is typically a three dimensional scene, such as the Grand Canyon. The measurement is a 2D pattern on film. The transformation between the source and the image in this case may be written

,

where is the intensity radiated by the canyon at point and is the pattern recorded on film at point . In conventional imaging systems, we are satisfied to look at the pattern on film and say, "That's the Grand Canyon," meaning either that the source distribution represented in the image is the 3D Grand Canyon, , or that our sensory experience looking at the recorded pattern is similar our sensory experience looking at .

Imaging is a subset of inverse problems. Inverse problems are often ill-posed or ill-conditioned. Imaging problems are essentially always ill-posed but hopefully seldom ill-conditioned. Ill-posed means that a problem has no exact or unique solution. Ill-conditioned means that solution or estimated solution varies widely in response to small variations in the measurements. Imaging is ill-posed because the information contained in the source is almost always greater than the information contained in the measurement. The information contained in a function or object can be defined to be the number of binary bits necessary to completely specify the object. For example, the question "Is the source red?" is one bit of information. If the source consists, as solids typically do, of approximately atoms per cubic centimeter, it might require or so bits of information to specify it. (The nth bit might the question "Is the nth atom carbon?") Note that the amount of information is the number of yes/no questions one must answer to specify the object and not the number of different answers to those questions.

For the imaging problem, the amount of information in the source may exceed the amount of information in the measurement for a variety of reasons but the most typical and most profound is that the transformation from the source to the measurement usually has finite information capacity. In the case of optical imaging, the finite capacity arises from the finite aperture of the imaging system, which means that some fields radiated by the source are not detected by the measurement system and from the finite wavelength of the field, which means that subwavelength source features cannot radiate. Note that the fact that the source is three dimensional and the measurements are two dimensional is not necessarily a problem from an information theoretic perspective. (From a practical perspective, the fact that the source is 3D may mean that the source has more potential for spatial complexity and information than the measurement.) We will speak of these issues a great deal later in the course, for now it is simply important to recognize that the resolution imaging systems is always finite and that finite resolution implies loss of information regarding the source.

While the reconstruction of sources from optical data is ill-posed, optical imaging to the limits of its resolution is reasonably well-conditioned. The primary reason for this is that transformation from the source to the measurements in optical systems is linear. We will discuss linear transformations in detail later in the term. For now it is sufficient to note that linear transformations have linear inversions to the limits of their resolution and that linear transformations of weak additive noise yield weak additive noise at the output. This weak additive noise does not therefore yield ill-conditioned inversion for linear transformations.

This is a course in optical imaging, which means after this note we leave the philosophy of imaging and focus on specific instruments. Optical imaging technology has developed in the following stages:

1. Observation of objects with the eye.
2. Observation of transformed objects with the eye. This technology developed over approximately the past 1000 years with the invention of lenses and curved mirrors.
3. Capture of images of objects. The first means of doing this was tracing images on a wall using a camera obscura.
4. Mechanical capture of images. This is photography, invented in the 19^th century.
5. Sequential capture of time varying scenes. This is a movie.
6. Electronic capture of images.

It is important to recognize that the nature of images and imaging changes with each of these steps. Only with the last step does imaging as an inverse problem become interesting. Prior to this last step, the inversion problem was implemented by the observer or by analog processing with lenses or pinholes. The class of transformations one can implement with lenses and mirrors is highly constrained. More complex inversion algorithms require digital processing of digital data. Nevertheless, lens based imaging transformations remain the main processing element of optical imaging systems. The reasons for this are discussed later in the term.

To day optical imaging systems include a wide range of instruments, from conventional cameras, microscopes and telescopes to advanced systems like scanning near-field systems, hyperspectral imagers, lidars, coherence tomography systems, laser fluorescence imagers, interferometric telescopes, surface profilometers, and many more. Despite this variety, all optical image sensors can be analyzed by considering two principal questions:

What is the spatial and spectral bandwidth of the imager? This question is essentially the same as asking what is the information transfer capacity of the transformation between the source and the sensed measurements.

What are the noise sources for the imaging system and what is the measurement signal to noise ratio? In digital imaging systems, optical component design (lenses, etc.) should be optimized to give the highest SNR for the target image.

This course answers these questions for a number of imaging systems. Before answering the questions, however, it is important to understand them in some detail. To obtain this understanding, the first quarter of the course is dedicated to understanding the optical field, how it propagates, and how it is detected. The second quarter of the course defines the spatial and spectral bandwidth of the field and considers how optical elements can transform the field. The third quarter considers specific components of imaging systems and how they can introduce error and noise to measurements. Only in the last quarter of the course do we consider specific systems and return to the two critical questions listed above.

| Home | Syllabus | Next |

David J. Brady, dbrady@duke.edu