The present invention relates to a system or systems for spectrally multiplexing a plurality of source images so as to provide a composite image, rendering the composite image, and demultiplexing of such a composite image to recover one or more of the source images.

Spectral multiplexing, as used herein, refers to a process for encoding plural source images in a composite image. Composite image rendering refers to a process for rendering the composite image in a physical form. Spectral demultiplexing refers to a process for recovering at least one of the encoded source images from the rendered composite image, such that the recovered source image is made distinguishable from, or within, the composite image, by subjecting the rendered composite image to a narrow-band illuminant that is preselected to reveal the source image.

Accordingly, the present invention is directed to methods and systems for spectrally-encoding plural source images and for providing the spectrally-encoded plural source images in a composite image; for rendering the composite image by imagewise deposition or integration of selected colorants on a substrate; and for recovering at least one of the encoded source images from the rendered composite image. The spectral characteristics of the colorants, illuminants, and sensors are employed in mathematical determinations to optimally encode the source images in the composite image such that at least one selected source image is uniquely recoverable under predetermined settings and operational characteristics of a selected image capture device.

Each source image is spectrally encoded by mapping values representative of each source image pixel to a corresponding pixel value in one or more of a plurality of colorant image planes. The contemplated encoding, in its simplest form, may include the conversion of each source image to a monochromatic, separation image, which is then directly mapped to a corresponding colorant image plane in the composite image. A plurality of source images can thereby be mapped to a corresponding plurality of colorant image planes in the composite image. An illuminant that is designed to particularly interact with a given colorant is said to be complementary, and vice versa.

The plural monochromatic separations are designed to be combined in the composite image, which in turn will control the amount of one or more pre-selected colorants to be deposited on the substrate. In one possible example, each colorant is assigned to a respective colorant image plane of the composite image, and the colorant values in the respective colorant image planes represent the relative amounts of colorant deposited in the rendered composite image. For example, a rendered composite image may be rendered using cyan, magenta, and yellow colorants that are deposited over a given area on a substrate by a rendering device.

A composite image file composed of the plural colorant image planes may be stored or transmitted as composite image file. The composite image may then be physically realized by delivering the composite image file to a rendering device with instructions for rendering the composite image on a substrate using the identified colorant or array of colorants. One suitable embodiment of a rendering device therefore includes a digital color electrophotographic printer.

In the contemplated encoding scheme, the mapping of each source image is performed according to mathematical determinations described herein for optimizing the effect of the following on the composition, rendering, and demultiplexing of the composite image: (a) the spectral absorption characteristics of the colorants selected for rendering the composite image, such spectral absorption characteristics especially comprehending the interaction of plural colorants when such are combined on the substrate; (b) the spectral radiance characteristic of the illuminant(s) that will be used to illuminate the composite image for recovering the source image; and (c) the spectral response characteristic of the sensors(s) in an image capture device that are used to sense the rendered composite image during such illumination, for recovery of the desired source image. The optimization is performed so as to provide at least one source image in the composite image that is uniquely revealed to an observer's eye (or recovered by a selected image capture device) according to the demultiplexing method described herein, and that is otherwise difficult or impossible to recover by other apparatus or methods.

In one embodiment of the method as defined in claim 8 the rendered composite image is captured with N sensors having respective sensitivities as a function of wavelength λ given by

In a further embodiment N values

In a further embodiment the method further comprising the step of performing gamut mapping to limit at least one of the source images to one or more gamuts available in the rendering step.

In a further embodiment the source image encoding step further comprises the steps of:

• converting at least one source image to a monochromatic separation image; and
• mapping the monochromatic separation image to a corresponding colorant image plane in the composite image.

For example, as illustrated in Figure 1, in addition to absorbing in its primary red absorption band, the typical cyan colorant also exhibits unwanted absorption in the green and blue regions of the visible spectrum. Likewise, the typical magenta colorant has significant absorption in the blue region in addition to its primary green absorption band. The typical yellow colorant has somewhat improved performance, in that it exhibits negligible absorption in the red region of the spectrum and a relatively minor amount of absorption in the green region; most of its absorption is restricted to the blue region.

The spectral absorption characteristics shown in Figure 1 are fairly typical of the commonly-available colorants used in printing and publishing. From the foregoing description, one will now recognize that the colorants that are most commonly available exhibit certain absorption characteristics outside of their primary absorption bands. Also, some illuminant sources produce illuminants that have partially overlapping spectra. If multiple source images are encoded without optimization and rendered using practical (i.e., non-ideal) colorants, the recovered source images may be found to interfere with each other and therefore may not be recoverable as separate images under the corresponding fields of illumination from a practical illuminant source.

With reference to Figure 2, for example, when a non-optimally rendered composite image is subject to CRT-based green illumination, due to the unwanted absorption in a cyan colorant in the green region of the spectrum, a first source image may be recovered along with a second source image having a primary absorption band in the green region of the spectrum. Likewise, under CRT red illumination, due to the unwanted absorption of a magenta colorant in the red region, along with the first source image, the second source image may also be recovered.

In consideration of this interaction between commonly-available colorants and commonly-available narrow band illuminants, the spectral characterizations of the colorants and the illuminants, coupled with a characterization of the spectral sensitivity of the observer (or in use of an image capture device, the spectral sensitivity of the sensors in the image capture device), may be predetermined, and accordingly selected, so that the interaction of these characteristics can be optimized and used to an advantage. With proper optimization, one can produce rendered composite images that, when subjected to a particular combination of illuminant and sensor, will promote recovery of only the correspondingly desired source image, whereby the recovered source image is substantially free of components of the other source images. However, when an attempt is made, without benefit of the optimization described herein, to generate, render, and recover a particular source image from a composite image, the recovered source image will be degraded or confused due to, for example, the unwanted presence of image components from the other source images present in the composite image. For example, the recovery of a degraded source image, or of image data that is not representative of the desired source image, thus indicates that the rendered composite image is improper and thus, in some applications, deemed counterfeit.

The methods and systems of the present invention may be directed to such encoding, rendering, and recovery of the desired source when the rendered composite image is to be subjected to illumination by one or more illuminants and the source image is then detectable by an observer or, in alternative embodiments, captured by one or more sensors operable in an image capture device. Such devices recover individual pixel signals in one or more sensor color channels under wide-band illumination; each channel's spectral sensitivity is predeterminable. If an image capture device is employed, having multiple sensor channels, the individual channels may be used in the demultiplexing process or a mathematical combination of the channels may be employed. The sensitivities of these devices may also be advantageously designed to optimize the recovery of a single source image without interference of the other source images. The colorants used for producing the images can also be pre-selected for their unique spectral characteristics and their relationship to the spectral sensitivity of the image capture sensor device. In a feature of the present invention, the source image presented in a rendered composite image is recovered when the composite image is illuminated by a controlled field of illumination of at least one narrow band illuminant having a selected spectral power distribution.

In another feature of the present invention, source image data representative of a plurality of disparate, pictorial source images may be spectrally encoded to form secondary image data representative of a composite image. The composite image may be realized as a single, complex, rendered pattern of deposited colorants, wherein at least one colorant is utilized for its particular spectral reflectance characteristic, and in particular for its narrow band absorption characteristic. The source images are accordingly spectrally multiplexed to provide a composite image which is recorded on the substrate by use of at least one of the narrow band-absorbing colorants. Typically, at least one of the source images is not easily recognizable to an observer who views the rendered composite image. That is, until the rendered composite image is subjected to the demultiplexing process, the rendered composite image is, to a certain extent, visually confused such that an observer is substantially unable to discern one or more of the source images by unaided viewing of the rendered composite image. Alternatively, one or more of the source images may be encoded so as avoid or reduce visual confusion and therefore be visually apparent in the rendered composite image when the rendered composite image is subjected to ambient white light or another wide band illuminant, and become confused or difficult to detect when the rendered composite image is subjected to a complementary narrow band illuminant.

In another feature of the present invention, a colorant selected for its narrow band absorbing properties may be employed to appear dark when subjected to its complementary narrow band illuminant, and to appear light when subjected to a differing illuminant having a spectral power distribution that lies substantially outside of the spectral absorption band of the particular colorant.

Accordingly, the present invention is directed to processing the plurality of source images in a digital image processing system utilizing specialized digital image processing and color science concepts, and rendering the resulting calibrated composite image using a calibrated rendering device. The rendered composite image can be subjected to at least one narrow band illuminant having a selected spectral power distribution, such that at least one source image is demultiplexed.

• Figure 1 represents reflectance spectra for a white paper substrate and colorants in the form of Cyan, Magenta, Yellow, and Black dyes (at 100% density) operable in a dye sublimation printer.
• Figure 2 represents the relative radiance spectra for the red, green, blue primaries generated by a typical cathode ray tube (CRT).
• Figure 3 is a block diagram of systems for spectral multiplexing and demultiplexing of plural source images, and for rendering a composite image having therein at least one encoded source image, constructed according to the invention.
• Figure 4 is a simplified schematic diagram of methods operable in the system of Figure 3 for spectrally multiplexing first and second source images in a composite image, rendering the composite image with use of respective first and second colorants, and for demultiplexing the rendered composite image.
• Figure 5 is a schematic simplified representation of the spectral multiplexing system of Figure 3, in which an image processing unit and associated peripheral devices and subsystems are employed.
• Figures 6A and 6B are simplified schematic representations of the spectral demultiplexing system of Figure 3, in which a controller and associated peripheral devices and subsystems are employed.
• Figure 7 is a rendered composite image generated using cyan and magenta colorants, having encoded therein first and second source images are suited for recovery under typical CRT red and green illuminants.
• Figure 8 is a rendered composite image generated using cyan and black colorants, having encoded therein first and second source images which are suited for recovery under typical CRT red and blue illuminants, wherein a spatially varying amount of black is provided so as to result in a uniform light gray background in the recovered image when subjected to blue illumination.
• Figure 9 is a rendered composite image generated using cyan, yellow, and black colorants, having encoded therein first, second, and third source images which are suited for recovery under typical CRT red, blue, and green illuminants, respectively.

The present invention exploits the interaction between certain narrow band illuminants and their corresponding (complementary) colorants (especially the colorants typically used for printing), and the manner in which the eye detects images illuminated with illuminants having narrow band spectral power distributions. The methodology described herein may be generalized to apply to an arbitrary number of illuminants and colorants, and for the purpose of simplicity the invention is described with reference to the cyan, magenta, yellow, and black colorants commonly used in color printing applications, and to the narrow-band red, green, and blue illuminants commonly generated by CRT-based light sources. This description thus makes reference to the handling of monochromatic and color source images encoded according to an array of colorants such as the CMYK color primaries. However, it will be apparent to one of ordinary skill in the art that there are alternative spectral schemes to be employed in the spectral multiplexing of the invention. An alternative would include a color system that employs primary colorants other than CMYK for color representations, such as systems that use RGB primaries or high-fidelity colorants such as orange and green. Still another alternative would be to employ the invention in a system that processes different types of multi-spectral data, such as source images encoded with respect to narrow band colorants responsive to illuminants generated from ultraviolet or infrared light sources.

As the present invention is directed to the multiplexing or demultiplexing of at least one source image encoded in a composite image, the composite image may be defined in a spectrally multiplexed (SM) image plane. This plane may have any number of different patterns of pixels, with a primary characteristic being that the plane is spectrally multiplexed. In general, at each location in the SM plane, a pixel value one or more spectral components may be present, and which spectral component is present depends on the gray level of the corresponding pixel in one of the source image planes. (The invention may also have applications to SM planes in which each pixel includes color values representative of color separation image data from more than one source image plane.)

The general theory of the invention may be understood with reference to a rendering device in the form of a color hardcopy output device, such as a printer, and to a mathematical framework that employs nomenclature similar to that used in conventional color imaging. Consider a color hardcopy output device with M colorants. Prints from this device are to be viewed under N different illuminants,

In the following description, we assume that a control value of 0 for a given colorant represents no printing of that colorant. This convention is not a requirement for the invention and is only adopted for notational simplicity.

The description herein is limited to the case of luminance characterization alone, because under narrow band illumination the eye primarily sees differences of luminance and is unable to distinguish most color differences. Note that luminance as described here agrees in concept with its standard usage, i.e., as a measure of the perceived light energy; however, it's definition is not limited to the conventional usage and is expanded to comprehend the special viewing situations also described herein. In particular, under narrow band illumination, specific visual effects may influence the perception of a source image. A specific instance of this is the Purkinje effect that causes increased sensitivity in the blue region of the spectrum at low light levels, which may be of particular relevance for viewing under blue light and CRT illumination in general. Some of the advanced concepts from photometry and colorimetry that are required in such situations are described for instance in G. Wyszecki and W.S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae, 2^nd Edition, John Wiley and Sons (1982).

The methods of the present invention are directed to the multiplexing, rendering, and recovery via demultiplexing of a source image encoded in a composite image. We assume that the one or more source images to be recovered are described by the spatial luminance distributions desired under each of the illuminants (although, in the alternative, any other equivalent specification that can be transformed to luminance/density may be used.) Thus, there are N images specified, with Y_i(x,y) being the desired luminance values that we wish to produce under the ith illuminant L_i where x,y denote the two spatial coordinates. For the purposes of simplifying the notation in the following discussion, the spatial dependence is sometimes dropped in the following description with the understanding that the discussion applies to each pixel location independently.

To examine the basic methodology symbolically, consider a simplified example of a composite image rendered in cyan and yellow colorants. In the simplified example below, additivity of "RGB" densities is assumed. This is for the purposes of simple illustration of the principles only and not intended to restrict the invention; in those situations where this approximation is invalid, more precise assumptions can be made. In this example: C, M, Y, K and R, G, B will respectively denote the colorants and illuminants; a superscript will denote illuminant; and a subscript will denote a colorant. Let:

• d^R = density of the image perceived under R illumination,
• d^B = density of the image under B,
• d_C^R = density C separation under R,
• d_C^B = density C separation under B,
• d_Y^R = density Y separation under R,
• d_Y^B = density Y separation under B.

When illuminated with a R or B illuminant, the total density perceived can be approximated as, d^R(x, y) = d_C ^R(x, y) + d_Y ^R(x, y) ≈ d_C ^R(x, y) d^B(x, y) = d_C ^B(x, y) + d_Y ^B(x, y) ≈ d_Y ^B(x, y)

Accordingly, this methodology exploits the characteristically low density of a colorant when subjected to a first illuminant and the characteristically high density exhibited by the same colorant when subjected to a second, differing illuminant. Thus, at least one perceptibly distinct source image (that is encoded in the rendered composite image by use of the particular colorant), will be imperceptible (or nearly so) to an observer when subjected to the first illuminant, but perceptibly distinguishable to an observer when illuminated by the second illuminant. Upon perception of the source image by an observer, the source image may be comprehended and the information embedded in the composite image, or the composite image itself, is thereby readily comprehended.

The example presented above assumed that colorant interactions can be entirely ignored. This assumption is not true with most practical colorants and additional considerations are therefore required.

Consider the case of a rendered composite image that is produced by using C and M colorants for subsequent illumination under red and green illuminants. For simplicity, in our illustration below we assume additivity for the red, green, blue band densities, as the general case for situations where this approximation does not hold is described subsequently. A first source image may be recovered primarily from the cyan component of a composite image, and a second source image may be recovered primarily from the magenta component; however, unwanted absorption by these colorants are preferably compensated to avoid artifacts discernible by an observer. The total density under red illumination at pixel location (x,y) can be approximated as d^R(x,y) = d_C ^R(x,y) + d_M ^R(x,y) and the total density under green illumination is d^G(x,y) = d_M ^G(x,y) + d_C ^G(x,y) where d_U^V (x,y) represents the visual density under illuminant V due to colorant U at pixel location (x,y).

The terms d_M^R(x,y) and d_C^G(x,y) represent the unwanted absorption. In the simplest case, it can be assumed that a colorant's absorption under its complementary illuminant is used for two purposes: 1 ) to recover the desired image and 2) to compensate for unwanted absorption by the other colorant(s) present in the composite image. So a magenta colorant may be used to produce the desired image to be seen under green illumination and to compensate for the unwanted absorption of the cyan colorant; a cyan colorant may be used to produce the desired image under red illumination and to compensate for unwanted absorption of magenta under red illumination.

The portion that is used to compensate for the unwanted absorption should combine with the unwanted absorption to result in a constant spatial density so as to make it "disappear". Let d1_C^R(x,y) represent the portion of Cyan density that is used to compensate for the unwanted absorption of Magenta under red, which is determined by d1_C ^R(x,y) + d_M ^R(x,y) = constant = q^R The remaining density contribution of Cyan under red illumination is d2_C^R(x,y) = d_C^R(x,y) - d1_C^R(x,y). Note that the total density can be written in terms of these components as

Therefore the overall visual density under red illumination corresponds a constant background density of q^R with the spatially varying density pattern of d2_C^R(x,y) superimposed. This spatially varying pattern is the one that is seen under red illumination and should therefore represent the first multiplexed image that is to be seen under red illumination.

In a similar manner the density contribution of magenta under green illumination can be decomposed into a component d1_M^G(x,y) that is used to compensate for the unwanted absorption of cyan under green illumination, given by d1_M ^G(x,y) + d_C ^G(x,y) = constant = q^G and the remaining component d2_M ^G(x,y) = d_M ^G(x,y) - d1_M ^G(x,y) which satisfies

Therefore the overall visual density under green illumination corresponds to a constant background density of K^G with the spatially varying density pattern of d2_C^R(x,y) superimposed. This spatially varying pattern is the one that is seen under red illumination and should therefore represent the second multiplexed image that is to be seen under green illumination.

Since the terms d2_C^R(x,y) and d2_M^G(x,y) represent the visual variations in density corresponding to the two multiplexed images, we would like to maximize their dynamic range. Since colorants can only add positive density, this requirement translates to minimizing the terms q^R and q^G subject to meeting the required equations and the physical constraint that colorants can only add positive density. We would therefore like to determine the smallest feasible values of q^R and q^G for which the above equations are feasible.

For the purpose of further illustration we use a first order approximation, that the amount of colorant added to compensate for unwanted absorption of the other colorant, itself only contributes a negligible amount of unwanted absorption (because of its small value). This assumption implies that the component of Magenta used to offset unwanted absorption of Cyan contributes negligibly to unwanted absorption under green and the component of Cyan used to offset unwanted absorption of Magenta contributes negligibly to unwanted absorption under blue. This assumption is used for illustration only, in practice, one can iteratively determine the appropriate amounts to account for higher-order effects or use an appropriate model/LUT as outlined subsequently in this disclosure. With this simplifying assumption, the range achievable for the desired spatially varying pattern d2_C^R(x,y) under red illumination is between q^R and d_C^R(x,y) with a total density variation or dynamic range of d_C^R(x,y) - q^R. Likewise the total density range available under green illumination is d_M^G(x,y) - q^G.

One set of feasible values for the terms q^R and q^G can be determined as:

• q^R = max( d_M^R(x,y)) = d_M^R(255) = max density for Magenta under red illuminant
• q^G = max(d_C^G(x,y)) = d_C^G(255) = max density for Cyan under green illuminant

This can be thought of as follows: the background density under red light q^R is equal to the maximum unwanted density that one can have from Magenta. The Cyan density component d1_C^R(x,y) is designed carefully so that the combination of Cyan and Magenta at each pixel has a density q^R, this can be achieved by putting no Cyan where Magenta is 100% (255 digital count) and appropriate amounts of Cyan to make up the density to q^R at pixels which have less than 100% Magenta. A similar argument applies to the Magenta density component d1_M^G(x,y) that compensates for the unwanted absorption of Cyan under red illumination.

As the present invention is directed to the multiplexing or demultiplexing of at least one source image encoded in a composite image, the composite image may be defined in a spectrally multiplexed (SM) image plane. This plane may have any number of different patterns of pixels, with a primary characteristic being that the plane is spectrally multiplexed. In general, at each location in the SM plane, a pixel value one or more spectral components may be present, and which spectral component is present depends on the gray level of the corresponding pixel in one of the source image planes. (The invention may also have applications to SM planes in which each pixel includes color values representative of color separation image data from more than one source image plane.)

The general theory of the invention may also be understood with reference to a composite image rendering device in the form of a color hardcopy output device, such as a color printer, calibrated with respect to an image capture device, such as a color scanner; and to mathematical determinations that will now be described.

Consider a color hardcopy output device that employs M colorants. The output spectral reflectance produced when the control values

In the following description, we assume that a control value of 0 for a given colorant represents no printing of that colorant. This convention is not a requirement for the invention and is only adopted for notational simplicity.

If a rendered composite image from this device is captured with a N different image capture sensor devices with sensitivities as a function of wavelength λ given by

This set of N functions characterizes the relation between the control values

With the notation and terminology defined above, the general multiplexed imaging problem reduces to the following mathematical problem.

Given N values

Typically, for N>M (number of image specifications > number of colorants) the system is over-determined and has a solution only under severe constraints on the

It is generally useful to include a gamut mapping step in the spectral multiplexing described herein to ensure that the source images are limited to the gamut of the system before attempting to reproduce them. The gamut mapping may be image independent or image dependent, where the term image is used to imply the set of desired source images recoverable under the different illuminants. In addition, the set of images to be multiplexed may be designed to take into account the gamut limitations and produce the best results with those gamut limitations.

Once the source images to be multiplexed have been mapped to the achievable gamut G, the problem of composite image rendering reduces to the determination of the control values for each of the M colorants for each pixel in the rendered composite image. This corresponds to an inversion of the system of equations in (1); in performing a color calibration step, the inverse may be pre-computed and stored in N-dimensional look-up tables (LUTs), with one LUT one per colorant (or alternately, a single N-dimensional LUT with M outputs).

In practice, the function in (1) itself needs to be determined through measurements of the device response by printing a number of patches with different M-tuples of control values and measuring them suitably to obtain the luminance under the different illuminants. The full spectrum of the patches may be measured for instance on a spectrophotometer from which the spectral responses may be computed using the spectral sensitivities of the different image capture sensor devices. Alternately, a characterization target may be captured by the different image capture sensor devices to directly determine the function in (1) without a knowledge of the spectral sensitivities of the individual image capture sensor devices.

Assume that the "spectral sensitivity band" of an image capture sensor device i refers to the region over which its spectral sensitivity is not negligible. Several simplifications can be incorporated into the general framework above. Suppose that N=M and the colorants and devices are such that colorant i absorbs only within the spectral sensitivity band of image capture sensor device i and is completely transparent in the spectral sensitivity bands of all other image capture sensor devices. This, for instance, represents a situation wherein each colorant has a narrow absorption band, outside of which the colorant is transparent, and the spectral sensitivity bands of the image capture sensor devices correspond to the same narrow absorption bands. In this case, f_i(A₁, A₂ ,...A_M ) = function of A_i alone = f_i(0,0,...,0, A_i,0,...0)≡ g_i (A_i) i=1,2,...N

The system of equations in Eq. (1) then reduces to M independent nonlinear equations, one for each colorant operating with the corresponding image capture sensor device: g_i(B_i) = Y_i   where i=1,2,...N

The achievable gamut can be defined as follows. As i=1,2,...N and let:

In other words, the achievable gamut is the product set of these individual response intervals. Note that an assumption in Eq. (6) is that the complete interval between the max and min limits can be realized without any "gaps" which would typically be expected with physical colorants. (For a definition of a product set, see for instance, A. Friedman,The Foundations of Modern Analysis, Dover, 1982, New York, NY.)

Under the assumption of a single illuminant interacting with a single colorant, the multi-illuminant imaging characterization problem reduces significantly. Instead of requiring N-dimensional LUTs, only an array of one-dimensional LUTs (one per colorant) are needed. The value of each colorant may be determined by the luminance under the corresponding illumination alone, or the response for the corresponding image capture sensor device alone.

In practice, the assumption of one illuminant interacting with only one colorant does not hold for typical colorants. However, if the strongest interactions are between the ith illuminant and the ith colorant with other interactions having a smaller magnitude, the achievable gamut is a reduced N-dimensional region that is contained in G₁. Note that the situation of using cyan, magenta, and yellow colorants with red, green, and blue lights for illumination corresponds to this case, where the cyan interacts most with red, magenta with green, and yellow with blue. Note also that the use of a black colorant that (typically) absorbs all illuminants almost equally, does not satisfy the requirement of strong interaction with only one illuminant. In practice this implies that a black colorant should be viewed as an additional colorant, i.e., if one colorant is black we should have:N = number of illuminants = number of images ≤ number of colorants -1 = M-1

Black may, however, be used with other colorants in special situations (as is described in the examples below) and can help improve achievable gamut (i.e., improve dynamic range), simplify computation, and reduce cost.

The general technique described earlier requires a measurement of the device response in the M-dimensional input space of device control values, and the final characterization may be embodied in the form of multi-dimensional LUTs with N-dimensional inputs. In several cases, the measurement and storage/computation requirements for multi-illuminant color imaging can be significantly reduced by using simple models of the output processes. One useful model is to assume that the visual densities follow an additive model, i.e.,

The terms

Using the above model, the system of equations in (1) reduces to:

The equations in (9) represent a system of K nonlinear equations in M variables (B₁,B₂,...B_M). The functions d_i(A_j) are available from the measurements of the "step-wedges" and the above equations can be solved for the control values B_j for luminance values within the gamut G, which was defined earlier. For points outside the gamut, the equations may be solved in an approximate sense providing a (less-controlled) form of gamut mapping.

Further simplification of these equations is possible by assuming that the densities in different spectral bands are linearly related, i.e., d_i(C)=α^j_i d_j(C)   i=1,2,...N where α^j_i = d_i (C) l d_j (C) is the proportionality factor relating the visual density for the jth colorant under the ith illuminant to the visual density for the jth colorant under the jth illuminant and is assumed to be independent of the colorant value C, and α^j_j = 1, Thus the convention adopted in Eq. (10) is that the density of the jth colorant under all other illuminants is referenced to its density under the jth illuminant itself, which is not strictly a requirement of our model but is chosen because it results in a simplification of the notation alternate conventions could also be equivalently used. Equation (10) is also motivated by the Beer-Bouguer law for transparent colorant materials and the assumption of relatively narrow band illuminants. (For a more detailed background, refer to: F. Grum and C. J. Bartleson, Ed., Optical Radiation Measurements: Color Measurement, vol. 2, 1983, Academic Press, New York, NY or G. Sharma and H.J. Trussell, "Digital Color Imaging", IEEE Transactions on Image Processing, vol. 6, No. 7, pp. 901-932, July 1997.) Even though a number of colorants and marking processes do not follow the Beer-Bouguer law exactly, in practice, Eq. (10) often provides a reasonably accurate empirical model for measured data and may be used for the purposes of the current invention. With the simplification of (10) the system of equations in (9) reduces to a linear system of equations:

The system of linear equations can be solved to determine a value of d , which provides the desired luminance values under the different illuminants (corresponding to the multiplexed images). The individual components of d , i.e., the d_j(B_j) values can then be used with the visual density response for the jth colorant under the jth illuminant to determine the control value corresponding to the jth colorant, i.e., B_j. This process is analogous to inverting a 1-D TRC. Repeating the process for each colorant provides the complete set of colorant control values required by

Note that if N=M, the above set of equations has a unique solution provided A is invertable, which is normally the case for typical colorants and illuminants. The solution in this case is obtained simply by inverting the matrix A . Furthermore, if the colorants and illuminants can be ordered in correspondence, i.e., colorant i absorbs illuminant i most and the other illuminants to a lesser extent, then αji ≤ α jj =1, for all i=1,2...N, i.e., the matrix A is square with the elements along the diagonal as the largest along each row, which is often desirable from a numerical stability standpoint. If M>N the system of equations will have multiple mathematical solutions, and the choice of a particular solution may be governed by additional criteria. One example of a criterion for choosing among the multiple mathematical solutions is feasibility, a feasible solution being a set of density values that can be realized with the range of colorant control values exercisable.

The model inherent in Eq. (12) can also be used to determine suitable approximations to the achievable gamut G and can be of assistance in performing gamut mapping. Typically, the density curves d_j (C) are monotonically increasing functions of the colorant control value C and the achievable range of densities for the jth colorant under the jth illuminant is between dminj= d_j (0) = 0 and dmaxj= d_j (Cmaxj) where Cmaxj is the maximum control value for the jth colorant. The achievable gamut assuming the model of Eq. (12) is valid is

The N images to be produced under the N illuminants provide a N-tuple for each pixel location corresponding to the desired luminance values at that pixel location under the N illuminants. The N-tuples corresponding to all the pixel locations must lie within the gamut G defined earlier in order for the image to be producible using the given colorants and illuminants. If images specified for multiplexing do not satisfy this constraint some form of gamut mapping is necessary.

A simple image-independent gamut mapping scheme may be defined as follows. First, ranges of luminance values under the different illuminants are determined such that all possible values within these ranges lie within the gamut G. This is mathematically equivalent to stating that we determine a set of N-intervals S_i = [Ymini, Ymaxi], i=1,2,...N such that the product set of these intervals is contained within the gamut G, i.e., S₁ × S₂ × S₃ × ... × S_N ⊆ G

The gamut mapping may then be performed on an image independent basis by mapping the set of requested luminance values under the ith illuminant to the intervalS_i = [Ymini, Ymaxi] by some (typically monotonous) function. The interval S_i determines the luminance dynamic range achieved under the ith illuminant. Since there are typically multiple choices of the sets

Figure 3 illustrates a system 100 operable in a first mode for spectrally multiplexing a plurality of source images to form a composite image, in a second mode for rendering the composite image, or in a third mode for demultiplexing the spectrally multiplexed composite image so as to recover at least one of the plurality of source images.

As shown in Figure 3, a plurality of disparate source image arrays 11-1, 11-2, ... 11-N are presented to an image input device 20 in another embodiment of spectral multiplexing system 101. Image input device 20 may be equipped to receive plural monochromatic source images or a combination of monochromatic and multichromatic source images. Image input device 20 may include an image capture device such as a digital scanner coupled to a random access memory, or any type of analog or digital camera coupled to a storage means such as a computer memory or a magnetic or optical recording medium. Image input device 20 may also include means for receiving a source image that had previously been stored in a random access memory, on video tape, or a laser-encoded disk, etc., or for receiving a source image created by a computer image generator, or a source image encoded in an appropriate format and transmitted on a network.

The illustrative representation of the plural source images in respective image arrays received by the image input device 20 in this example includes a first source image 12-1 represented in a first source image array 11-1 and a second source image 12-2 represented in a second source image array 11-2. The system 101 can optionally receive N source images which are represented in a respective number of image arrays. In this exemplary embodiment of the invention, disparate pictorial source images are employed and at least one of the plural source images is intended for ultimate recovery (via spectral demultiplexing) from a rendered composite image, the latter which will be described shortly.

Once the source image data is received in the input image device 20, it is presented to a spectral multiplexer 30, which encodes a data representation of a composite of at least the first and second source images, so as to provide a composite image 32 on an spectrally multiplexed (SM) image plane. Such encoding may proceed in one embodiment with mapping for every pixel location, or by mapping in localized areas rather than specific pixels, to the composite image 32, so as to multiplex the information necessary for encoding of each corresponding pixel located in each source image.

Next, according to operation of a composite image rendering system 102, data representative of the composite image is provided to a rendering device 40, which can be connected to the spectral multiplexer 30 by any one of a variety of suitable means for transmitting or storing electronic information. The rendering device 40 records the composite image 32 on a substrate 44 with use of a predetermined array of colorants, so as to form a rendered composite image 42. The rendered composite image 42 is thereby fixed on the substrate 44.

The rendered composite image 42 is available for scanning by a demultiplexer 50. Although the rendered composite image 42 is representative of data encoded in the spectrally multiplexed plane using the method of the invention, the rendered composite image 42 may exhibit a confused appearance under conventional ambient lighting conditions. In particular, at least one of the source images 12-1, 12-2, etc. is made difficult or impossible to distinguish under conventional ambient lighting conditions. A particular source image may thereby be embedded and made difficult or impossible to distinguish until a demultiplexer 50 is operated to selectively process the composite image 42 in a manner sufficient to recover the embedded source image.

According to operation of a spectral demultiplexing system 103, the particular source image (as shown in Figure 3, source image 12-1) may be recovered from the composite image 42. In the embodiment illustrated in Figure 3, the output of the demultiplexer 50 is a recovered source image 12-1 using the method of the invention. The recovered image source may thus be made perceptible to an observer (as will be described with respect to Figure 6A) or may be captured by an image capture device which is then represented in an image data signal provided to conventional apparatus (as will be described with respect to Figure 6B) for a variety of uses including, for example, simple detection of the presence or absence of the recovered source image, with subsequent interpretation of such significance; or for comparison to a standard for authentication, wherein the recovered image data is compared, for example, to a stored version of the original source image 12-1 that was initially provided to the image input device 20.

Recovery of a particular source image by use of controlled illumination will be understood to generally proceed according to an exemplary embodiment of the spectral demultiplexing system 103 as follows. The substrate 44 is positioned with respect to an illuminant source operable within the demultiplexer 50, such that one or more illuminants generated by the demultiplexer 50 illuminate the composite image 42 so as to subject the array of colorants in the rendered composite image 42 to the selected illuminant. As a result of the rendered composite image 42 thus being controllably and selectively illuminated by at least one illuminant, a desired source image is then detected by the demultiplexer 50. Output of the sensor data representative of the desired source image 12-1, now recovered, is thereby available for further uses.

Figure 4 is a simplified schematic diagram of exemplary embodiments of spectral multiplexing, rendering, and spectral demultiplexing methods 61, 62, 63, respectively. In step 61 for multiplexing plural source images, a first source image 71 and a second source image 72 are provided to the multiplexer 30, which outputs a composite image data file to a rendering device 40. The output of the rendering device 40 is substrate 90 which has incorporated therein a composite image 92. The original source image 71 is rendered as a pattern using a first colorant; in the illustrated embodiment, a cyan ink or toner is chosen. The second source image 72 is rendered as a pattern using a second colorant; in the illustrated embodiment, a magenta ink or toner is chosen. (As there is typically some overlap in absorption bands between practical narrow band colorants, the two source images are preferably encoded in step 61 to account for the absorption that will occur when plural colorants are utilized to produce the composite image.)

In a rendering step 62, the composite image specifies patterns in cyan and magenta colorants that are accordingly rendered on a substrate 90 to form the rendered composite image 92. Those skilled in the art will appreciate that certain portions of the two patterns may be co-located and other portions are relatively spatially distinct. Nonetheless, in certain embodiments of the present invention, visual recognition by an observer of at least one of the source images in the composite image may be made difficult or impossible due to the confusion between source images that are encoded in the composite image.

In step 63 for demultiplexing the rendered composite image 92, the substrate 90 having the rendered composite image 92 fixed thereon is illuminated by the demultiplexer 50. Controlled illumination of the substrate 90 according to a first mode 51 of illumination causes the first source image 71 to achieve a particular level of density with respect to the remainder of the composite image and thus the first source image 71 becomes detectable on the substrate 90. Alternatively, controlled illumination of the substrate 90 according to a second mode 52 of illumination causes the second source image 72 to be similarly detectable on the substrate 90. In the illustrated embodiments, the first source image 71 and the second source image 72 are therefore selectably distinguishable on the substrate 90.

Figure 5 illustrates a schematic simplified representation of the spectral multiplexing system 101 of Figure 3, in which an image processing unit 130 and associated peripheral devices and subsystems are employed. An image input terminal 120 may include an image capture device 122 such as a scanner, digital camera, or image sensor array; a computer image generator 124 or similar device that converts 2-D data to an image; or an image storage device 126 such as a semiconductor memory or a magnetic, optical, or magneto-optical data storage device. The image input terminal 120 derives or delivers digital image data in the form of, for example, plural monochromatic image files, wherein the picture elements or "pixels" of each image are defined at some gray value. For example, the input terminal 120 may be employed to derive an electronic representation of, for example a document or photograph from image capture device 122, in a format related to the physical characteristics of the device, and commonly with pixels defined at m bits per pixel. If a color document, the image is defined with two or more separation bitmaps, usually with identical resolution and pixel depth. Image data from the input terminal 120 is directed to an image processing unit (IPU) 130 for processing so as to be encoded to create a composite image. It will be recognized that the data representing one or more source images is spectrally encoded by the image processing unit 130 to provide secondary image data representative of a composite image suitable for subsequent rendering.

The image processing unit 130 may include image memory 132 which receives input image data from image input terminal 120 or from another suitable image data source, such as an appropriately programmed general purpose computer (not shown) and stores the input image data in suitable devices such as random access memory (RAM). Image processing unit 130 commonly includes processor 134. The input image data may be processed via a processor 134 to provide image data representative of plural source images defined on respective source image planes in accordance with the present invention. For example, image data signals in the form of RGB or black and white (B/W) images may be processed, and the luminance information derived therefrom may be used to provide data representative of a source image. Image data signals presented in other formats are similarly processed: image data signals in, for example the L*a*b format, may be processed to obtain data representing a source image from the lightness channel. Image data signals that are already formatted in grayscale are generally usable without further processing.

Operation of the image processing unit 130 may proceed according to one or more image processing functions 138, 139 so as to encode the source image data into the composite image file as described hereinabove. Processing may include a color conversion which, if necessary, may be implemented to convert a three component color description to the printer-specific four or more component color description, and may include a halftoner which converts a c bit digital image signals to d bit digital image signals, suitable for driving a particular printer, where c and d are integer values. In certain embodiments, additional functions may include one or more of color space transformation, color correction, gamut mapping, and under color removal (UCR)/gray component replacement (GCR) functions. Control signals and composite image output data are provided to an interface 136 for output from the image processing unit 130.

The image processing unit 130 may be embodied as an embedded processor, or as part of a general purpose computer. It may include special purpose hardware such as for accomplishing digital signal processing, or merely represent appropriate programs running on a general purpose computer. It may also represent one or more special purpose programs running on a remote computer.

Figures 6A and 6B are simplified schematic representations of the spectral demultiplexing system 103 of Figure 3, in which a controller and associated peripheral devices and subsystems are employed to recover one or more recovered source images 171, 172. Figure 6A shows a controller 150 connected to a illuminant source 160 that is operable for subjecting the composite image 42 on substrate 44 to first and second predefined illuminants 161, 162. Firstly, as illustrated with reference to the rendered composite image 42 on substrate 44, while subject to conventional ambient lighting and in the absence of an illuminant 161, 162, only the composite image 42 is distinguishable and no source image is detected by an observer 170. However, upon activation of the source 160 so as to provide the first predefined illuminant 161, a first recovered source image 171 becomes detectable to an observer 170. Alternatively, the mode of operation of the source 160 may be switched so as to provide a second predefined illuminant 162, whereupon the composite image 42 is instead subjected to the second illuminant 162, and a second recovered source image 172 becomes detectable.

In Figure 6B, upon activation of the source 160 so as to provide the first predefined illuminant 161, the first recovered source image 171 is captured by an image capture device having an array 180 of sensor devices. Alternatively, the mode of operation of the source 160 may be switched so as to provide a second predefined illuminant 162, whereupon the rendered composite image 42 is instead subjected to the second illuminant 162, and the second recovered source image 172 is captured.

In its simplest form, the controller 150 may be constructed as a manually-operable illuminant selector switch. Alternatively, as illustrated, the controller 150 may be provided in the form of a computer-based control device having an interface 156 connected to an image capture device such as a color scanner, having source 160 and array 180 integrated therein, which offers programmable control of the operation of the illuminant source 160 and array 180. The controller 150 may thus be operated to cause selective activation and deactivation of the illuminant source 160 so as to provide one or more selected fields of illumination 162.

The controller 150 is operable for accomplishing tasks such as activation, deactivation, or sequencing of the illuminant source 160, setting illuminant intensity, illuminant frequency, etc. Embodiments of the controller 150 benefit from operation of a programmable control system comprising standard memory 152 and processor 154. The controller 150 may be employed, for example, for supplying uniform R or G or B screen images to the interface 156 for subsequent display on the illuminant source 160 when the latter is constructed in the form of a CRT monitor.

Operation of the illuminant source 160 by the controller 150 may proceed according to certain sequenced control functions so as to provide, for example, controlled operation of the illuminant source 160 to afford a field of illumination that varies according to selective characteristics such as a sequential activation and deactivation of selected narrow band illuminants, or of controlled operation of the intensity of same; or with interactive control according to intervention by an operator of the particular sequence, intensity, or duration of the illuminants. As noted above, the rendered composite image may be constructed to have a plurality of source images encoded therein; for example, of at least first and second patterns of respective first and second colorants. The rendered composite image may be subjected to a temporal sequencing of illumination by respective first and second narrow band illuminants, thus allowing a respective one of the first and second recovered source images 171, 172 to be sequentially distinguishable.

In consideration of the problem of interaction between commonly-available colorants and commonly-available narrow band illuminants, a characterization of the printing process and of the illuminants is performed according to the present invention so that the interaction effects can be characterized and used to an advantage. With proper characterization, one can produce rendered composite images that, when subjected to a particular illuminant, reveal substantially only the correspondingly desired source image.

The model of Eq. (12) was applied to render a composite image on a xerographic printer, such as the Model DC12 printer commercially available from Xerox Corporation. Two images were printed with cyan and magenta colorants for the purpose of viewing under CRT red and green illumination. The two images were mapped to a subset of the gamut for the two illuminant, two-image problem and the corresponding colorant values were obtained by the procedure described above. The resulting rendered composite image is shown in Figure 7.

Figure 7 is a rendered composite image generated using cyan and magenta colorants, having encoded therein first and second source images are suited for recovery under typical CRT red and green illuminants. CRT red illuminant refers to a field of illumination supplied by the CRT when the red channel alone is driven and the other channels are off. Typically this would correspond to a maximum value set for the red signal (usually 255) and a 0 value set for the other channels, although other intermediate values could also be used, provided the other channels are off. CRT green illuminant similarly refers to a field of illumination supplied by the CRT when the green channel alone is driven. According to the characterization process described herein, the amounts of cyan and magenta colorants at each image location in the rendered composite image were adjusted such that when the rendered composite image is subjected to CRT green illuminant only the first source image (Chapel image) is recovered whereas the second source image (Balboa image) is largely undetected; and, likewise, under CRT red illuminant, only the second source image (Balboa image) is recovered and the first source image (Chapel image) is largely undetected.

The calibration process can be further simplified if some of the colorants are non-interacting. For instance, a typical yellow colorant often has a very sharp and clean cut-off wavelength above which it does not absorb any light energy. Thus the yellow colorant has almost no impact on the luminance seen under red and green illuminants. Thus, if a combination of black and cyan is used to produce an image under red illuminant, and a region of uniform luminance under blue light, the inclusion of a source image in yellow in the composite image will not interfere with the source image seen under red illuminant while being clearly visible over the uniform background when subjected to the blue illuminant. These considerations were used to generate the rendered composite image shown in Figure 8.

Figure 8 is a rendered composite image generated using cyan and black colorants, having encoded therein first and second source images which are suited for recovery under typical CRT red and blue illuminants, wherein a spatially varying amount of black is provided so as to result in a uniform light gray background in the recovered image when subjected to blue illumination. The composite image includes encoding the first source image (Balboa image) as a cyan separation. A second source image encoded in the form of a spatially-varying amount of black is multiplexed in the composite image so as to cause the rendered composite image to appear as a uniform light gray background when viewed under blue illumination. Under CRT red illuminant, due to the strong absorption of the red light by the cyan colorant, the first source image (Balboa image) can be clearly seen. Under CRT blue illuminant, the rendered composite image appears to have a uniform light gray region. Note that for Figure 8, the first source image (Balboa image) can also be seen under CRT green illuminant due to the unwanted absorption of the cyan colorant in the green band.

Figure 9 is a rendered composite image generated using cyan, yellow, and black colorants, having encoded therein first, second, and third source images which are suited for recovery under typical CRT red, blue, and green illuminants, respectively. The first and second source images include those used in generating the rendered composite image of Figure 8, with a third source image (the portrait image "Erica") additionally encoded as a yellow separation. Since yellow has almost no unwanted absorption in the red and green regions of the spectrum, the appearance of the rendered composite image under CRT-based red and green illuminant is largely unchanged. However, the yellow colorant strongly absorbs the CRT blue illuminant and the third source image (the portrait image Erica) can therefore be detected over the uniform light gray background that is obtained with Figure 8 under CRT blue illuminant.

Note that one skilled in the art may realize a variety of alternatives are within the scope of this invention for implementing the above described embodiments. Its advantageous use is expected in color printing by various processes including offset lithography, letterpress, gravure, xerography, photography, and any other color reproduction process which uses a defined number of colorants, usually three or four, in various mixtures. Embodiments of the rendering system 102 include apparatus capable of depositing or integrating a defined array of colorants in a substrate, according to the composite image, such that the array of colorants is susceptible to selective reflection or transmission of a selected narrow band illuminant incident thereon. For example, the composite image may be rendered on a transparent film and a desired source image may be recovered when the substrate is backlit by a suitable narrow band illuminant. Examples include hardcopy reprographic devices such as inkjet, dye sublimation, and xerographic printers, lithographic printing systems, silk-screening systems, and photographic printing apparatus; systems for imagewise deposition of discrete quantities of a color on a substrate surface, such as paint, chemical, and film deposition systems; and systems for integration of colorant materials in an exposed surface of a substrate, such as textile printing systems.