Basic knowledge of image analysis

1. Lumination technique

1.1 Luminous flux

1.2 Luminosity

1.3 Relation between luminosity and luminous flux

1.4 Illuminance

1.5 Luminous density

1.6 Lambert beams

1.7 Luminous flow for lens representation


2. Lumen-technical characteristics of components

2.1 Spectral characteristics

2.2 Light sensitivity of photodiodes

2.3 Luminous sensitivity of CCD sensors and cameras


3. Conversion rules for common measurements

4. Video technique

4.1 Monochrome video signal after CCIR 624

Electric specification

Image structure

Image synchronization signals

Line synchronization impulse

4.2 Color transmission

RGB video signal

YC video signal (SVHS-Signal)

PAL video signal

4.3 Full-frame cameras

4.4 Camera operation modes

Integration time

Field integration operation

Frame integration operation

Gain / AGC

g - correction


5. Optics

5.1 Sharp edge requirement

5.2 Representation scale

5.3 Aperture value

5.4 Depth of field

5.5 Telecentric lenses





All image analysis is based on the highest possible image processing quality. In order to maintain a high standard, the user will find it beneficial to have some basic knowledge on the following areas:

  • Lumination- and illumination technique
  • Lumen-technical characteristics of components
  • Video technique
  • Optics

The following chapters are an attempt to outline the terminology mostly met in practice in a concise and well-structured way. Extensive mathematical derivations and physical details are omitted on purpose.



1. Lumination technique

1.1 Luminous flux

By the luminous flux Fv , we address the spectral radiation flux and the measure for the entire luminous capacity.


The spectral flux of the green illuminator (l = 555 nm) with 1 W capacity equals a luminous flux of 683 lumen (lm).

1 W = 683 lm



1.2 Luminosity

The Luminosity Iv is the luminous flux per spatial angle sr (standard globe: spatial angle = 4p [sr]) and is given in Candela (cd).

1 cd = 1 lm / sr



1.3 Relation between Luminosity and luminous flux

For a conical reflector with a = half of the conic angle, the entire luminous flux (=spectral capacity) is derived from the Luminosity Iv as follows:

Punctual reflector: Fv = 4p * Iv

Reflector with angle a < 90° : F v = 2p * Iv * (1 - cos a)



1.4 Illuminance

The Illuminance Ev is the illuminative flow per square area and is given in lux (lx).

1 lx = 1 lm / m²



1.5 Luminous density

The luminous density Lv represents the luminous capacity Fv per area angle and per square area element of a luminous or illuminated and reflecting area. It equals the impression of lightness on an area and is given in Nit (nt).

1 nt = 1 cd / m² = 1 lx / sr



1.6 Lambert beams

Lambert beams are totally undiffused and waste-free reflecting areas. If a Lambert beam is shone on with a illumination intensity of Ev, then the luminous density of the Lambert beam is:

Image


In practice, dull surfaces come pretty close to the ideal Lambert beam, with reflection degrees varying considerably, dependent on the spectral characteristics (of colored areas). For a white piece of paper, the reflection degree is around 66% and can be used for orientation.



1.7 Luminous flow for lens representation

In order to determine the necessary illumination intensity for camera photographs, the relation between luminous density of the surfaces and the resulting Illuminance on image level (sensor level) is important. The Illuminance on image level, in the following referred to as Evb, can be approximated fairly well from the illumination density Lv of objects to be photographed as follows:

Image


Z = f/D is the filter number (= relation between focal length and objective opening) of the objective used.


If the area to be photographed is a Lambert beam, then the Illuminance Evb on image level (=sensor level) can be approximated by using the Illuminance Evb of the Lambert beam as follows:

Evb » Ev / 4Z²

Ev » 4Evb

Using the above formulas, consideration should also be given in practice to the reflexion factor of the illuminated surfaces and the spectral characteristics of the illustration, surfaces and camera sensor.



2. Lumen-technical characteristics of components

2.1 Spectral characteristics

The following table gives an overview of the spectral sensitivities of the human eye, a typical CCD sensor, a Si photodiode, and an emission specter in an electric bulb. As opposed to electric bulbs, LED´s have a very small spectral bandwidth. Typically, their spectral bandwidth lies around 50 nm. In order to adapt their relative spectral sensitivity to that of the human eye, CCD cameras are often equipped with IR lock filters to reduce the CCD sensor's IR sensitivity.
Extremely high short distance IR sensitivity values of CCD sensors make CCD cameras a useful tool for illustration through IR-LEDs. Having a higher activity range then LED´s in the visible areas, they can realize photographic systems with suitable irritation suppression (in particular against fluorescent tubes) if coupled with lock filters in the visible area.


Image


Extremely high short distance IR sensitivity values of CCD sensors make CCD cameras a useful tool for illustration through IR-LEDs. Having a higher activity range then LEDs in the visible areas, they can realize photographic systems with suitable irritation suppression (in particular against fluorescent tubes) if coupled with lock filters in the visible area.



2.2 Light sensitivity of photodiodes

A photodiode generates a photographic flow proportionate to the illuminative intensity independent of the illumination duration.
In the following, the sensitivity of silicium photodiodes is referred to as SD. Typically, it values:

SD» 0,5 A / W

The photographic flow I of a silicium photodiode covering the area A can be derived from the illumination intensity Ev as:

I = Ev * SD * A

Typical photographic stream of silicium photodiodes: I » 0,8 nA * Ev [lx] * A [mm²]

BPX 65: I » 1 nA / lx
BPW 21R: I » 7 nA / lx

Besides a high illumination sensitivity, photodiodes also have a very high differentiation between signal and noise. Dynamics valuing over 120 dB can be realized easily with common photodiodes.



2.3 Luminous sensitivity of CCD sensors and cameras

A CCD sensor is based on the tension principle. The luminous stream generates tension wish is read serially by the sensor and measured in voltage. There is a proportional relation between the tension intensity and the product of illumination intensity and illumination duration.
We give the sensitivity RCCD of a CCD-Sensors in V / mJcm². Another important value is the saturation tolerance limit SECCD [mJ / cm²]. In order to achieve good values for the signal-noise relation in an active sensor CCD, the operating conditions should be chosen to have the lightest image points near the saturation tolerance limit of the sensor.
Typical values for saturation tolerance limit in common CCD sensors (line and matrix sensors) are:

SECCDtyp » 0,005 - 0,5 mJ / cm² » 0,034 - 3,4 lx s

For line sensors, the utilizable range of dynamics (difference between signal and noise) is up to 80 dB, whereas CCD cameras generally have a range between 50 - 60 dB.
Note that the camera sensitivity given in lx by most camera producers is inaccurate even in terms of nomenclature and often causes misconceptions. (It must be added, however, that the illumination duration is mostly taken to be 20 ms). Since often the control conditions, such as spectral area and degree of control (visible light or complete control) are not mentioned, the data provided by producers are not too useful in practice. A comparison between different camera types is hard with only these data. It would be far more advantageous if camera producers gave the values for saturation tolerance limit SECCD, signal-noise difference and spectral sensitivity. Instead, these data have to be requested separately, or else deduced from the data sheets of the CCD sensors.



3. Conversion rules for common measurements

1 J = 1 W s

1 W = 683 lm

1 cd = 1 lm / sr = 1,464 mW / sr

1 lx = 1 lm / m² = 1,464 mW / m² = 0,1464 mW / cm²

1 W / m² = 683 lx 1 mW / cm² = 6830 lx 1 mW / cm² = 6,83 lx

1 mJ / cm² = 1 mW s / cm² = 6,83 lx s



4. Video technique

With the exception of high solution cameras and other kinds of special image providers (such as scanners or slow scan systems), image information is mostly transferred by cameras as video signals. In Germany, the norm for monochrome signals is CCIR 624 and PAL for colored signals. In other countries, the norms are, for instance, NTSC (USA) and SECAM (France, eastern European countries), all of which differ from the CCIR 624 norm in line number and temporal behavior, but not in their basic structure. In the following, we will treat the introduction of the CCIR 624 norm for image information coding as an example. The transfer of color images is based on that for monochrome signals. The necessary extensions are dealt with in section 4.2.



4.1 Monochrome video signal after CCIR 624

Electric specification

Video signals are transmitted as analog signals with a signal capacity of 1 VSS. The connection impedance for signal transmission is set at 75 W. In industrial settings, the connection is mostly realized by BNC linkages and 10 pole video connectors, whereas cinch connectors are common for private consumers
A video signal is made up by the intensity signal (the actual image information) and the signals for image synchronization. The intensity signal with a signal capacity of approximately 0.75 VSS and the synchronization signals with around 0.25 VSS are made to overlap the proper video signal by summation. Since the video signal annexation for cameras is usually carried out through AC coupling (capacity dependent, without fixed tension potential), the intensity measurements for reference tension values of black (in the following referred to as black value) has to be deduced from the signal. For this reason, the black value is transmitted before and after each image information line after line. An absolute intensity measurement is enabled by computing the difference between the interval measurement values within a line and the respective black value.


Image structure

Video signal coding has been determined along with the development of television technique. In order to prevent flickering monitors as far as possible, an image frequency of at least 50 Hz is required. Since this image frequency results in signal frequencies that at the time of television invention were hard to manage, the frame is composed of two fields. One of them contains all information on even image lines, the other that on uneven ones. The sub-frames are transmitted alternatively with a frequency of 50 Hz, by which technique flickering could be reduced tolerably, regardless of the low frame frequency of 25 Hz. Images are scanned line by line, beginning on the upper left corner and finishing at the lower right corner.
Generally, the image synchronization signals as described in the following have to be generated by a video signal through synchronized connection. Industrial cameras used in image analysis have direct access to some of these signals, which supports a simpler construction and higher precision in image processing.


Image synchronization signals

We will now discuss the most important synchronization signals used in image analysis.
A full image consists of 625 lines divided into two semi-images of 312.5 each. In the semi-images, transmission of either the even or uneven image lines is activated. The semi-image is initiated by a V-Sync-impulse, and the lines of each semi-image are in discord with the impulse by half of the line duration. Within a semi-image, the lines are separated by H-Sync impulses. The overlapping of H-Sync and V-Sync impulses results in the synchronization signal S, which is again made to cover the intensity data.
Video synchronization signals in the first and second semi-images


Image


· S: composite synchronization signal
· S*: S-Signal devoid of satellite impulses
· H-Imp: horizontal synchronization impulse
· V-Imp: vertical synchronization impulse
· TR-Imp: satellite impulse
· U-Imp: interruption impulse

In practice, there is often the S* signal in addition to the S signal serving as a synchronization signal, especially for monitors with special image providers. When designing an image synchronization, we therefore have to take into consideration that both types of synchronization signals should be supported.



Line synchronization impulse

Besides the H impulse, several other impulses are important for decoding an image line. In the table below, some essential impulses are listed. The following values must be assigned to the individual lines:

Line duration Z = 64 µs Delay d = 1,55 µs

H-impulse h = 4,65 µs TR-impulse tr = 2,25 µs

U-impulse u = 4,65 µs scanning break A(H) = 12 µs

Burst delay db = 7 µs Burst impulse b = 2,25 µs

Image


Each video image line can be subdivided into two areas for synchronization and image information. The synchronization information is transmitted in the time interval of the horizontal scan break A(H). During the remaining 52 µs the line gray-tones (or intensity and color information in case of color images) is transmitted.
First and foremost during the scan break, the H synchronization impulse is transferred. Then comes the black value for monochrome images or a burst impulse for color decode phase synchronization that overlaps the black value if the image is colored (PAL). The signal level within the area is the reference value for the intensity information along the line.
In image synchronization, TR and U impulses are transmitted as substitutes for H impulses in the respective lines. No image information transfer is carried out in these lines.
Approximately 580 lines out of the 625 available lines are commonly used for image information transmission. The intensity values of the remaining lines are set to 0 or utilized for special information like VITC time code or video text.



4.2 Color transmission

For color image coding, three basic signal types are mainly used in video technique. In the following section, we will introduce them and explain their specific characteristics. For the impulse terminology, the reader is referred to the overview on line synchronization impulses.



RGB video signal

This representation is mostly used with monitors and 3 chip CCD cameras. Color information is transmitted through separate video signals for the red, green and blue light spectrum. They are structured like the monochrome signal, with the synchronization information either transmitted through separate signals or eclipsing the green signal.
Since the bandwidth for signals is unlimited for RGB signals, it can be regarded the highest-quality solution for color coding.



YC video signal (SVHS signal)

Using a YC signal means that the RGB information is transformed into a representation in the HSI coordinate sphere. The acronym is for hue, saturation and intensity of an image point. Intensity information is transmitted via the intensity signal Y (luminous signal), which equals the video signal in monochrome settings. The H and S information is coded by the chrome signal C through modulation of the color carrier with a carrier frequency of 4.43 MHz. This color carrier is amplitude modulated by its saturation. An additional phase modulation is carried out along with the coloring. For each line, a color burst signal with phase position 0° is generated as reference signal. Since the color information is modulated, it is limited to a bandwidth of 4.43 MHz, whereas the lumination information is transmitted without bandwidth limitations.



PAL video signal

Using a PAL video signal the Y and C signals are added up to a single signal. In order to enable subsequent division into component signals, the bandwidth of the Y signal must be restricted to under 4.43 MHz. Both chrominance and luminance information are bandwidth restricted, which means that this type of signal provides the poorest possible image quality.



4.3 Full-frame cameras

There is one disadvantage to interlaced photography with video cameras, that is, the asynchronous illumination of both semi-images. If moving objects are photographed, the result of this is a line discrepancy in both semi-images of the object, which means that only one of the two semi-images can be used for processing. In other words, the utilizable solution is reduced by half.
In recent years, a way has been found to illuminate the entire image at one time by the full frame cameras especially designed for image analysis. In principle, the outgoing signal is the same as the video signal: the full image is shown either undivided or, for reasons of compatibility divided into two semi-images, dependent on the type of camera. What makes the difference is not the initial signal coding but the camera technique.



4.4 Camera operation modes

Integration time

The integration time equals the camera illumination time, during which the incoming light is integrated into the sensor. If the setting is less than the semi-image duration of 20 ms, then the illumination is carried out after each semi-image. During this time, the sensor is usually emptied on an electronic shutter, which enhances the removal of tension created by noise during execution. Even if a flash is used, it is advisable to adjust the electronic illumination time.


Field integration operation

This operation type provides illumination of a semi-image during the processing of the preceding semi-image, which is the most common operation type for CCD cameras. It allows a maximum for illumination duration of 20 ms (50 Hz PAL signal).

Function diagram for field integration operation with 10 ms illumination time.

Image



Frame integration operation

With this operation mode additionally available for some cameras, the user can choose longer illumination times stretching over two preceding semi-images (40 ms). This results in higher light sensitivity. However, a technically caused solution minimization must be accepted.

Function diagram for frame integration operation

Image



An interesting issue connected with this operation type is the option of fully illuminating images with flashes, since the illumination intervals for two subsequent semi-images overlap. Thus, full images can also be taken with interlaced cameras. The disadvantage lies in the high alien-illumination sensitivity, because it overlaps with the flash while the illumination duration has to remain on 40 ms. Moreover, a long integration time also causes a higher signal noise than would be the case with a real full frame camera. In exceptional cases, however, this photographic technique can provide an economical alternative to still overpriced full frame cameras.



Gain / AGC

Gain adjustment is responsible for the post-enhancement of video signals produced by the CCD sensor. Even though a high post-enhancement can make a dark image lighter, the quality of the image will inevitably suffer from such a procedure, since the gain regulation does not influence the illuminative sensitivity of the CCD sensor. Consequently, a high enhancement heightens the noise in the video signal.
AGC is the automatic enhancement control keeping the signal amplitude constant regardless of varying light conditions. Besides the AGC control, some cameras offer automatic integration time control and object filter control. This results in a very high illumination flexibility which can be achieved without manual readjustment. In particular for control applications this has been beneficial.



g - correction

Mit g-correction means an adjustable non-linearity of camera enhancement recognition lines. For a gamma value g = 1 , the resulting enhancement is linear. The recognition line patterns of all other values can be seen in the diagram below.


Image



g- correction is applied for optimization of a utilizable image contrast through non-linear enhancement under various conditions.



5. Optics

In this chapter, we will introduce the basic objective computing rules. For reasons of simplification, an objective will be regarded as an individual, thin lens. We name:

Z: aperture value

D: Objective aperture [mm]

f: Focal length [mm]

b: Image length [mm]

g: Object length [mm]

Dg: Depth precision = interval length of object length for accurate representation [mm]


Image



5.1 Sharp edge requirement

A sharp-edged object representation is one that meets the following requirement. For further considerations, we will assume that these requirements are met.

1/g + 1/b = 1/f or f² = (g - f) ( b - f)

For real images, we must have g > f. For g < f we get virtual images (lens). g = f fails to render a representation.



5.2 Representation scale

The representation scale m, i.e. the relation between image dimension and object dimension is computed as follows:

m = b / g = (b - f) / f = f / (g - f)

For b = g = 2f we get a representation factor of m = 1.



5.3 Aperture value

The aperture value Z, that is the measurement for object illumination intensity, is defined as the focal length relative to the objective aperture (double lens radius).

Z = f / D

A doubling of the aperture value results in an illumination intensity reduction on image level to ¼. See also section 1.7 on this issue.


5.4 Depth of field

The depth field Dg is defined as the tolerance field with the object width g in which the object is still represented "sharp". In this context, "sharp" means that you cannot detect blurred contours when regarding or processing the image. If the object is point shaped and located in the object plane, then a variation of the object width g renders the object as a more or less big disc (blurred circle the double radius of which will be called d) in the image plane. As soon as the double radius of this disc is smaller than the image plane solution (pixel distance of CCD cameras, film kernel in photography), the representation is sharp.

For CCD cameras, d is between 0.01 and 0.005 mm.


Image


When computing the depth of field, we assume that g > f (realistic representation).
The terminology is:

G: Object level B: Image level

g: Object length b: Image length

A point in object level G is represented as point B in image level if the accuracy is right (see requirements under section 5.1). Points on the positions G´ and G" are represented to B´ and B", respectively. As can be seen in the figure below, the image of these points on image level B is a circle with the radius d.

It is our goal to compute the depths of field Dg-:= g-g´ and Dg+:=g"-g relative to the objective aperture D and the double radius of the blurred circle d .
As the radiation axiom says:


Image

Image



By remodeling and application of the sharpness relation 1/f = 1/b + 1/g = 1/b´ + 1/g´ = 1/b" + 1/g" we get:


Image

Image



Solving the equation for g´ and g", we get the minimum and maximum distance for the representation of a point on an blurred circle <<FONT FACE="Symbol">d as:


Image

Image



Computing the difference to the object length g and using the relation Z = f / D, we can then deduce the depth sharpness:


Image

Image



The above formula for Dg+ is only valid for object lengths g < (f² / d Z). For g >= (f² / d Z) the depth sharpness Dg+ = ¥ and the given computation formula gets invalid since the sharpness requirements introduced in section 5.1. have been used both for the deduction of g and g".

Thus, the entire depth sharpness Dg is the summation of Dg+ and Dg- under consideration of all margin requirements for Dg+ as:


Image



For extremely low values of d and g << f² / d Z, the expression (dZg / f)² can be neglected, which makes it possible to approach the depth sharpness Dg as follows:


Image



5.5 Telecentric lenses

Especially for object measurement, the limited depth sharpness and interdependency between representation scale and object width (distance between object and objective) have an undesired effect. In order to suppress these errors, more and more users replace the aforementioned objectives with telecentric optical devices. They are structured as follows:

Image


A small aperture allowing only those rays to penetrate which run through the focal point of the objective is inserted into the focal level of the objective. Only those spectral rays running parallel to the objective meet this requirement. If the object width g is altered, neither the representation scale nor the object sharpness of an ideal, point-shaped aperture are affected.

In practice, however, the ideal state is restricted by the real radius of the telecentric aperture. However, compared with conventional optical devices, the achievable improvement of depth sharpness and stability of representation scale are clearly an improvement.

The disadvantages of telecentric optics are the large objective aperture D, which has to fit the object width, and the small illumination intensity caused by the telecentric aperture.



...back