The present invention relates to a semiconductor operational circuit,
and in particular, relates to an operational circuit which is applied to high speed
image processing or the like.
In recent years, in concert with the development in computer technology,
the progress in the field of data processing technology has been truly remarkable.
However, when attempts were made to realize the flexible type of data processing
conducted by human beings, it was almost impossible to obtain the results of such
calculations in real time using present computers. The reasons advanced for this
are that the data which human beings process in the course of their daily lives
are analog data, and these data are vague. It is thus a problem in present data
processing systems that the extremely redundant analog data are all converted into
digital values, and rigorous digital operations are conducted one by one.
An example of this is image processing. For example, if one screen
is incorporated into a 500 x 500 two dimensional array, then the total number of
pixels is 250, 000, and when the strength of the three colors red, green, and blue
for each pixel is expressed in terms of 8 bits, then the amount of data in one
stationary image reaches 750,000 bits. In moving images, the amount of image data
increases with time.
Present day computers conduct data processing with respect to enormous
amounts of digital signals by repetitive operations, so that an enormous amount
of time is required, and real time processing is impossible.
An example of this is the detection of movement vectors, which is
one the important operations in the processing of moving images. That is to say,
in this operation, with respect to the images in two frames which are continuous
in time, the amount of motion in the image of the object photographed is determined.
In this operation, the image may be moved by ± 8 pixels vertically or horizontally,
and the amount of dislocation may be determined by overlaying the images until
they line up. In other words, the amount of dislocation in the image between the
two frames is calculated with respect to a total of 64 combinations, and the combination
having the smallest amount of dislocation is found. Total calculations of a few
tens of GOPS are required, and even if extremely high speed processors are employed
on a number of chips in parallel, a period of approximately 30 msec is required.
In order to control robots in real time, it is important to conduct the image
data processing in 1 msec or less; however, this is completely impossible with
The present invention was designed in light of the above circumstances;
it has as an object thereof to provide a semiconductor operational circuit which
is capable of instantaneously processing in parallel a large quantity of information.
Disclosure of the Invention
The semiconductor operational circuit of the present invention which
executes a predetermined operation with respect to a first signal train of signals
A1, A2, ..., AN-1, AN (where N is a
positive integer) of N signals numbered from 1 to N, and a second signal train
of signals B1, B2, ..., BM-1, BM (where
M is a positive integer) of M signals numbered from 1 to M, comprising a plurality
of first operational circuits for executing a predetermined operation with respect
to Ai and Bi+n
(where i is a positive integer and n is a positive
or negative integer and 1 ≤ i ≤ n and 1 ≤ i + n ≤ M) and generating
an output signal Ci,n, at least one second operational circuit for
generating the sum Sn of a part or the whole of output signals of the
first operational circuits with respect to a predetermined value of n, where i
has differing values, or for generating a predetermined signal Tn determined
by the sum Sn, and a third operational circuit for finding the value
of Sn or Tn with respect to a plurality of different n values
and for determining the n value for which the maximum or minimum value of Sn
or Tn is given.
By means of the present invention, it becomes possible to conduct
operations with respect to an enormous amount of analog data gathered from the
outside world without altering these data, and by means of this, to determine logic,
and as a result, it is possible to realize extremely high speed data processing
employing simple operational circuitry.
Brief Description of the Drawings
Figure 1(a) and Figure 1(b) are schematic diagrams explaining a first
embodiment of the present invention.
Figure 2(a) and Figure 2(b) are schematic diagrams explaining the
function of the ΔX detection circuitry of Figure 1.
Figure 3 is a schematic diagram showing the circuitry which calculates
the sum of Ci,n.
Figure 4 is a schematic diagram showing an amount of motion detector.
Figure 5 is a circuit diagram showing an example of a winner-take-all
Figure 6 (a) shows an example of an absolute value operational circuit,
while Figure 6(b) shows the relationship between the input and output thereof.
Figure 7(a), Figure 7(b), Figure 7(c), and Figure 7(d) illustrate
the operation of the absolute value operational circuit.
Figure 8 is a schematic diagram illustrating a second embodiment
of the present invention.
Figure 9(a) shows another example of an absolute value operational
circuit, while Figure 9(b) shows the relationship between the input and output
Figure 10 is a graph showing the results of the operation of the
absolute value operational circuit of Figure 9.
Figure 11 is a circuit diagram showing another example of a winner-take-all
Figure 12 is a graph showing the results of the operation of the
circuitry of Figure 8.
Figure 13 is a graph showing the time t signal and the time t + Δt
Figure 14 is a schematic diagram illustrating a third embodiment
of the present invention.
Figure 15(a), Figure 15(b), Figure 15(c), Figure 15(d), Figure 15(e),
and Figure 15(f) are schematic diagrams illustrating the detection principle of
the third embodiment.
Figure 16 is a schematic diagram illustrating a fourth embodiment
of the present invention.
Figure 17 is a schematic diagram showing an example of a method for
setting the gain of each source follower to the same value.
Figure 18 is a schematic diagram illustrating a method for detaching
Figure 19(a) and Figure 19(b) illustrate the operation of the circuitry
when SEARCH (N,M) is executed.
Figure 20 is a schematic diagram illustrating a fifth embodiment
of the present invention.
Description of the References
Best Mode for Carrying out the Invention
- image sensor array,
- 102, 103
- images of an aircraft captured as time t and t + Δt,
- data projected along the x axis relating to image 102,
- data projected along the x axis relating to image 103,
- data projected along the y axis relating to image 102,
- data projected along the y axis relating to image 103,
- Δx detection circuit,
- projection data memory,
- correlation operational unit,
- amount of movement detector,
- Δy detection circuit,
- 201, 202
- correlation operational unit,
- 203a, 203b
- correlation operational circuit cell,
- 303, 304
- floating electrode,
- 306, 307
- depletion type NMOS and PMOS,
- source follower circuit having a CMOS structure,
- winner-take-all circuit (WTA),
- input signal,
- address encoder,
- CMOS inverter,
- common floating gate,
- 9-input NAND circuit,
- 601, 602
- NMOS switches,
- 603, 604
- NMOS transistors,
- 603a, 604a
- floating gate electrodes,
- switches of the input part,
- 606, 607
- A data series memory storing t + Δt data,
- B data series memory storing t data,
- 803a, 803b
- correlation operational circuit cells,
- 804, 805
- source follower circuit,
- floating gate,
- 903, 904
- 903a, 904a
- floating gates,
- 905, 906
- image sensor array,
- Δx detection circuit,
- circuit block,
- memory storing t - Δt data,
- t data memory,
- t + Δt data memory,
- absolute value operational circuit,
- memory storing A data,
- memory storing B data,
- circuit group similar to 203,
- 1701, 1702
- dummy capacitors,
- floating gate,
- 1704, 1705
- source follower,
- floating gate,
- 1811, 1812
- A data series memory,
- B data series memory,
- analog data shift register,
- operational circuit,
- source follower circuit,
- floating gate.
Hereinbelow, embodiments of the present invention will be explained
using the figures.
Figure 1(a) shows a first embodiment of the present invention as
a block diagram; this is an operational circuit which detects motion vectors of
images captured by an image sensor array 101. First, the function of this circuit
will be explained briefly using Figure 1(b). References 102 and 103 indicate images
of an airplane captured at times t and t + Δt, respectively. Reference 102'
indicates data in which data of each pixel in image 102 (indicating the brightness
of each pixel) is totaled with respect to vertical columns and this is plotted
along the x axis; these data thus represent a so-called projection of image 102
onto the x axis. Reference 103' indicates the x axis projection data of image
103, and 102" and 103" indicate y axis projection data relating to images 102 and
103. The circuit of Figure 1(a) determines the amount of movement Δx in the
x direction of the aircraft by detecting the displacement in the x axis projection
data, and determines the amount of movement Δy in the y direction from the
amount of displacement in the y axis projection data, and by means of this detects
a movement vector (Δx, Δy).
In Figure 1(a), reference 101 indicates a sensor array; here, in
order to keep the explanation simple, an 16 x 16 sensor array (a total of 256 cells)
was used as an example, but it is of course the case that any number of cells may
be employed. First, the explanation will center on Δx detection circuit 104.
Reference 105 indicates a analog memory which stores 16 data projected on the
x axis, for each column of the sensor array, and two groups of values are maintained;
the values for time t, and the values for time t + Δt. Reference 106 indicates
a circuit which obtains the correlation between the two groups of data series.
That is to say, the two groups of data series described above are shifted one
pixel at a time in the horizontal direction, and the amount of displacement is
calculated; in this embodiment, the amount of displacement when a shift of a maximum
of 4 pixels is carried out is determined by means of simultaneous parallel operations.
This output is inputted into amount of movement amount detector 107, and the amount
of shift which results in the minimum evaluated amount of displacement is determined,
and by means of this, the circuit specifies the amount of movement Δx. Δy
detection circuit 108 is similar to the Δx detection circuit, and this circuit
specifies the amount of movement Δy.
Next, the structures of the portions marked 105 and 106 are shown
in greater detail in Figure 2(a). First, the X axis projection data trains obtained
at time t is A1, A2, ..., A16, and memory 201
stores this temporarily. For example, A1 is stored as a voltage value
proportional to the sum of all the sensor outputs of the first column of image
sensor array 101.
B5, B6, ..., B12 represents a x axis
projection data trains obtained at time t + Δt; the fifth to the twelfth
data is stored in memory 202.
Reference 203 represents a correlation operational unit; this comprises
72 correlation operational circuit cells 203a, 203b, ... having the same function,
which are arranged in two dimensions as shown in the figure. As is shown in Figure
2(b), the function of each cell is such that the circuit calculates the absolute
value Ci,n = |Ai - Bi+n| with respect to the data
Ai supplied from memory 201 and the data Bi+n supplied from
memory 202, and outputs the results.
The same data are supplied from memory 201 in the direction shown
by the arrows in the figure. That is to say, the data A1, A2,
..., A8 are supplied to row 204a, the data A2, A3,...,
are supplied to row 204b, and the data A5, A6,...,
A12 are supplied to row 204c.
The same data are supplied from memory 202 in the direction shown
in the figure by the arrows. The eight data B5, B6, B7
,..., B12 are supplied to each row, in other words.
Accordingly, the data A2 - A9 from memory 201,
and the data B5 - B12 from memory 202, are supplied to the
cells of row 204b, so that |A2 - B5| is calculated in cell
203a, and lA3 - B6| is calculated in cell 203b, for example.
In other words, in this row, calculations are conducted in which n = 3 and the
B data are shifted to the left by 3 pixels with respect to the A data, and the
absolute value of the difference therebetween is obtained. In row 204c, the calculations
|A5 - B5|, |A6 - B6|, ..., |A12
B12|, respectively, are carried out, so that n = 0, and the amount of
displacement when a comparison is carried out without shifting the data is determined.
A circuit which sums all these absolute values or which finds, for example, S0
= |A5 - B5| + |A6 - B6| + ... + |A12
- B12|, is provided with respect to each of the rows in correlation
operational unit 203. This structure is formed by, for example, the circuitry shown
in Figure 3. This example concerns the case of row 204c, and each cell 301, 302,
and the like are circuits identical to those in Figure 2(b); the respective output
voltages C5,0, C6,0 are coupled with an electrically floating
electrode 305 via capacitors (having a capacitance Cs) 303 and 304.
References 306 and 307 indicate, respectively, depletion type NMOS and PMOS transistors,
and these form a source follower circuit 309 having a CMOS structure. This is
an amp in which the voltage gain when VOUT = VFG (VFG:
potential of floating gate 305) is approximately 1. Reference 308 shows the floating
Since VFG = Cs (C5,0 + C6,0
+ C7,0, + ... + C12,0)/(8Cs + Co),
a voltage proportional to the sum So of the amount of displacement determined in
each cell is outputted to VOUT.
Similar calculations are carried out in each row 204a, 204b, and
the like, and output voltages proportional to the sums S4, S3,
and the like of the amount of displacement from each row are outputted simultaneously.
These 9 data are inputted into motion amount detector 107. The details of this
portion are shown in Figure 4.
In Figure 4, Sn (where n = 4, 3, 2, 1, 0, -1, -2, -3, and -4) shows
the output voltages from each row of correlation operational unit 203; these form
the 9 input voltages 402 into winner-take-all circuit (WTA) 401. In this embodiment,
the function of the WTA is such that the circuit outputs a value of 1 only to
the output terminal corresponding to the smallest input, while the value is outputted
to all other output terminals; Figure 4 shows the case in which S3 has
the smallest value among the 9 input signals. That is to say, only the output of
the WTA corresponding to S3 has a value of 1. This is inputted into
address encoder 403, and a binary code corresponding to +3, for example, 0011,
is outputted as the value of Δx.
Here, the lead 0 indicates "+", while the following 3 bits indicate
A ROM with prescribed codes written thereinto may be employed as
this address encoder, or alternatively, combined logical circuitry may be employed.
The fact S3 has the smallest value indicates that the sum
of the amounts of displacement in row 204b in Figure 2(a) is the smallest. That
is to say, whet the t + Δt data are shifted 3 pixels to the left, this row
has the best agreement with the t data; this corresponds to the case in which the
movement of the image during the period of time Δt was 3 pixels to the right,
so that in other words, Δx = 3. In this way, it is possible to rapidly find
the x component of the movement vector. In Figure 1, the structure of the Δy
detection circuitry is identical, and it is possible to rapidly find the value
of Δy in the same manner.
Next, a concrete example of the circuitry of WTA 401 is shown in
Figure 5. Sn and Vn are, respectively, the input terminal
and output terminal corresponding to the number n of WTA 401; circuitry identical
to that of reference 501 is used with respect to each input. Reference 502 indicates
a CMOS inverter; the common gate 503 thereof is placed in an electrically floating
state by placing switches SW1 and SW2 in an OFF state. The 2 inputs Sn
and VR are capacitively coupled with floating gate 503 via capacitors
having the same size.
Next, the operation of the circuit will be explained. First, in the
state Sn = 0 and VR = VDD, switch SW1 is closed.
When this is done, CMOS inverter 502 is biased at the point at which the input
and output characteristics change the most rapidly, and VFG becomes
equal to VDD/2. At this point in time, switch SW1 is placed in an OFF
state, and the common gate 503 is placed in a floating state. Subsequently, when
VFG becomes VDD/2, CMOS inverter 502 enters an ON state,
the output thereof drops to 0 V, and if VFG is less than VDD/2,
the inverter enters an OFF state, and the output thereof rises to VDD.
Next, the respective input voltages are inputted into Sn. Now, if Sn
is greater than 0, then VFG = (VR + Sn)/2 is given,
and if VR = VDD, then VFG > VDD/2.
That is to say, CMOS inverter 502 is in an ON state with respect
to all values of n, and Vn = 0 and Va = VDD results.
Reference 504 indicates a 9 input NAND circuit; since the inputs thereof are all
values of 1, the output 505 has a value of 0. By means of this, switch SW2 is placed
in an OFF state.
Next, when Vr is ramped down from VDD to 0 V in the space,
for example, of 200 nsec, when VFG, VDD/2, inverter 502 enters
an OFF state, and Vn = VDD and Va = 0 results.
There are a total of 9 circuits 501 provided for WTA 401; however, the first circuit
to enter an OFF state is that circuit having the smallest value of Sn.
NAND circuit 504 outputs a value of 1 if even one of the inputs thereof drops to
a value of 0, and switch SW2 is in an ON state in all the circuits, so that the
output voltage Va is fed back in each circuit to the floating gate,
and thereby values are latched in an unchanged manner. That is to say, in the circuit
having the smallest input, Vn = 1 (Va = 0) results, and in
the other circuitry, Vn = 0 (Va = 1) results.
The WTA function is realized in the above manner. The circuitry of
Figure 5 represents only 1 example of a WTA, and it is of course the case that
circuits having other forms may be employed.
In Figure 3, the value of Sn was outputted by a source
follower circuit 309; however, this source follower 309 may be omitted. That is
to say, floating gate 305 may be made identical to the floating gate 503 in Figure
5. At this time, the Sn input of Figure 5 becomes unnecessary, and it
is necessary that the size of the capacitor of the VR input be set equal
Next, a concrete example of the absolute value operational circuit
of Figure 2(b) is shown in Figure 6. Figure 6(a) is a circuit diagram, and V1
and V2 represent the two inputs thereof; these correspond to the input
terminals for the A data and the B data.
Figure 6(b) shows the relationship between Vout and V1
- V2; it can be seen that Vout = |V1-V2|.
The operational principle of the present circuit will be explained using Figure
7. First, the state of each switch in the state in which a prescribed input voltage
is applied to V1 and V2 is as shown in Figure 7(a). Next,
NMOS switches 601 and 602 are placed in an OFF state, and the gate electrodes
603a and 604a of NMOS transistors 603 and 604 are placed in a floating state (Figure
7(b)). Next, when the switches 605a-d of the input portion are switched and the
input voltages are switched, the potential of floating gates 603 and 604 become
equal to V2 - V1 and V1 - V2. If V1
> V2, then V2 - V1 < 0, and the potential
thereof is fixed at the diffusion potential (approximately -0.7 V) by means of
the drain PN junction of the NMOS transistor 601 (Figure 7(c)).
Next, when switches 606 and 607 are switched, the output terminal
VOUT is raised as shown in Figure 7(d) by the supply of current from
VDD. If the threshold voltage of NMOS 603 and 604 is set to, for example,
0 V, then Vout rises to a potential equal to the higher of the potential
of floating gates 603a and 604a. That is to say, the circuit becomes one which
outputs the maximum value. That is to say, Vout becomes equal to |V1
The circuit of Figure 6 represents only one example; it is of course
the case that any circuit may be employed insofar as it is a circuit which outputs
a value proportional to |V1 - V2| or a value which increases
monotonically with the value of |V1
In the circuit of Figure 2, a time t data series was inputted into
201, while a time t + Δt data series was inputted into 202; however, it is
clear that no problems will be caused even if this is reversed. A second embodiment
of the present invention, which has such a structure, is shown in Figure 8.
Reference 801 indicates an A data series storing the t + Δt
data, while reference 802 indicates a B data series memory storing the t data.
803a, 803b, and the like are correlation operational circuit cells; in the present
embodiment, these output a value of Ci,n = VDD - |Ai
- Bi+n|. A concrete circuit diagram is shown in Figure 9. In Figure
8, the wiring supplying the A data 801 and B data 802 to each cell is shown by,
respectively, references 804 and 805 (the lines running in a diagonal direction)
. The basic structure is identical to that of Figure 2, so that a detailed explanation
thereof will be omitted here. Reference 806 corresponds to the source follower
circuit 309, while reference 807 corresponds to the floating gate 305 thereof.
Reference 808 indicates a WTA. In this case, the WTA circuit outputs a value of
1 only at the position of the input having the maximum value; concretely, a circuit
such as that shown in, for example, Figure 11, may be employed.
The circuit shown in Figure 9(a) is a circuit which is almost identical
in principle to that of Figure 6(a); the chief differences thereof are that PMOS
903 and 904 are employed in place of NMOS 603 and 604, and the voltage becomes
VDD when the gates 903a and 904a thereof are reset. Figure 9(b), which
shows the characteristics of the Vout thereof, shows characteristics
in which 0 and VDD are inputted in a reversed manner (characteristics
such that the graph appears to be turned upside down) and the circuit outputs the
largest value (VDD) when V1
and V2 are in agreement, while
when V1 and V2 are separated by the furthest amount, then
the minimum value (0 V) is outputted. That is to say, as the data in each cell
in Figure 8 become closer, the score becomes higher, and the value of Sn
Figure 10 shows the results of a simulation of the operation of the
circuit of Figure 9 using a circuit simulator (HSPICE). In the figure, RST indicates
the control signal applied to terminal 905, while SFact indicates the control signal
applied to terminal 906; output is obtainable when both of these are at the low
(0 V) value. It can be seen that the circuit operated as expected.
The circuit of Figure 11 is almost identical to that of Figure 5.
There are 3 differences: the output inverter 506 of Figure 5 is removed, the NAND
circuit is replaced by OR circuit 1101, and VR is initially set to 0
V, and is then ramped up from 0 to VDD. Vn has a value of
'1' only when Sn has the largest value in the circuit.
Figure 12 shows the results of a simulation of the circuit of Figure
8 using a circuit simulator (HSPICE). Here, as time t and t + Δt data, the
Gaussian distribution signal shown in Figure 13, in which a shift of +3 pixels
to the right was carried out after Δt, was applied, and calculations were
conducted. In Figure 12, SFinp indicates the input voltage into source follower
circuit 806 and the like, or in other words, the potential wave form of floating
gate 807 and the like, while SFout indicates the output wave form of the source
follower. Vout indicates the output of WTA circuit 808; only the output of the
terminal corresponding to n = +3 has a value of 5V, while the other terminals
all have a value of 0 V, and this indicates that Δx = +3 pixels.
As is clear from this figure, operations are concluded after 300
nsec no matter how long, and the movement vector can be selected. In comparison
with the conventional digital method, in which it was extremely difficult to end
operations within 30 msec, it can be seen that this represents an 105-fold
(a 100,000-fold) increase in speed. Accordingly, the present invention is extremely
effective in the real time processing of image data. Furthermore, because this
invention can realized using simple circuitry such as that shown in Figure 8, it
can be integrated on the same chip as the image sensor, and applications such as
the direct provision of intelligent functions in robot eyes and the like can be
accomplished in an extremely simple manner.
In the first and second embodiments described above, movement vectors
were found using data representing the direct addition, by row or by column, of
2 dimensional image sensor data; however, image processing such as edge detection
or the like may conducted in advance with respect to the 2 dimensional image data,
and after that, the data may be added by row or by column. There are cases in
which this method improved the accuracy of detection. Furthermore, a method may
be adopted in which one or other of these methods are appropriately selected, operations
are conducted successively in which results are determined using both cases with
the same hardware, and thus movement detection is conducted with higher accuracy.
Furthermore, the result of the addition of 1 row or 1 column of pixel data corresponded
to 1 datum; however, 2 or more rows, or two or more column of data may be added,
and this may be made to correspond to 1 datum. This is effective in the case of
image sensors having a large number of pixels. Furthermore, a correlation operational
circuit was described which only conducted operations which obtained the absolute
value; however, other operations may be employed. For example, an operation which
determines the largest value of Ai
and Bi+n may be conducted,
and the amount of movement may be determined by the minimum value of the total
of these maximum values for each cell. Furthermore, this may be reversed, and the
minimum values of Ai and Bi+n may determined, and the amount
of movement found by finding the maximum total of the minimum values for each
cells. Furthermore, a so-called matching operation may be conducted in which the
output is VDD only when |Ai - Bi+n| < w, and the output
is 0 at other times. Furthermore, a matching operation may of course be conducted
in which the output is 0 only when |Ai - Bi+n| < w and
the output is VDD at other times.
A further important point in the present invention is that it is
not necessary that the t data be in perfect agreement with the t + Δt data
with respect to the pixel shift. The shift resulting in the relatively closest
fit is found, so that even if an object moves while the form thereof is changed
slightly, the amount of movement can be found without problem.
Next, a third embodiment of the present invention will be explained
using Figure 14. This is an operational circuit which accurately finds only the
amount of movement of a moving object when the specified object is moving against
a still background. Reference 1401 indicates an image sensor array, while reference
1402 indicates a Δx detection circuit; these are identical to those described
in Figure 1. In the present embodiment, a new circuit block 1403 is added.
References 1404 - 1406 indicate memories which store sum signals
in the columnar direction of the image sensor, that is to say, the x axis projection
data; these memories are used for the data of the 3 time frames such that memory
1404 stores the t - Δt data, memory 1405 stores the t data, and memory 1406
stores the t + Δt data. Reference 1407 indicates an absolute value operational
circuit; this calculates the absolute value of the difference of the t - Δt
data and the t data, and the absolute value of the difference between the t data
and the t + Δt data, and provides these as the A data series and B data series
to the Δx detection circuit. The circuit shown in Figure 6 may be employed
as this circuit.
By means of this, it is possible to specify the movement of the moving
object against a still background with a high degree of accuracy. The principle
thereof will be explained using Figure 15.
Figure 15(a) depicts a balloon moving over a building. Only the balloon
moves, and it moves to the right. The x axis projection data of the data of (a)
are as shown in Figure 15(b), and the data after Δt are as shown in Figure
15(c), and as the background is incorporated in these data, it is extremely difficult
to determine the amount of movement using a pixel shift. When the absolute of the
difference of both is obtained as in Figure 15(e), the background is made stationary,
so that this is canceled out and disappears. Next, when the difference between
the t + Δt data (Figure 15(d)) and the t data is obtained, this results
in Figure 15(f). If the data of Figure 15(e) and Figure 15(f) are then used as
the time t' data and the t' + Δt data, it is possible to find the amount
of movement Δx using a circuit 1402 identical to that of the first and second
A fourth embodiment of the present invention is shown in Figure 16.
This circuit executes a SEARCH (N, M) command with respect to data series A1
- A8 and B1 - B8 comprising 2 groups of 8 data.
That is to say, the circuit takes a number of continuous data M from the position
having the ordinal number N in the A data series, and determines at what position
in the B data the best agreement is found. For example, in this embodiment, if
M is the values from 4 - 6, then N takes a value within a range of 1 - (8 - M).
References 1601 and 1602 indicate memories storing, respectively,
the A data and the B data, and reference 1603 indicates a circuit group identical
to 203 in Figure 2; each correlation operational circuit cell is identical to those
in the first and second embodiment, and any type of circuit may be employed. The
A data and the B data are supplied to each cell along the lines shown, respectively,
by the arrows and the dotted lines.
To use an example, the sixth data on the right hand side A3,
A4,- A8 and the sixth data B1, B2, -
B6 are supplied to the row 1604.
The major point of difference with the case shown in Figure 2 is
that length of each row is different. As a result, in the circuitry shown in Figure
3, the number of cells 301, 302, and the like connected to floating gate 305 differs
for each row, and this interferes when a comparison is conducted between the sizes
of the sums of the absolute values of the difference between differing rows. That
is to say, it is necessary to provide a mechanism for setting the gain of each
source follower to the same value. An example thereof is shown in Figure 17.
This figure shows the structure related to the cells of row 1604
in Figure 16. Since only 6 cells are incorporated in 1604, this row has 2 fewer
cells than the largest row 1605, which has 8 cells. Accordingly, dummy capacitors
1701 and 1702 are added so that the total reaches 8, and the input terminal thereof
fall to the ground potential. By means of this, it is possible to set the total
capacity value as seen from the floating gate 1703 to the same value in all rows.
Furthermore, the minimum value of M of the search (N, M) is 4, so that there are
cases in which only 4 cells are employed. In this case, switches 1704 and 1705,
for example, may be set to the ground side, and cells 1706 and 1707 may be cut
off. The cells 1708 - 1711 which are necessary for the operation may have the cell
outputs thereof connected to capacitors via switches, and the output thereof may
be transmitted to floating gate 1703. By doing this, it is possible to conduct
size comparisons with a constant value for the gain of the source follower circuit
Figure 18 illustrates a different invention for cutting off unnecessary
cells. In this method, no dummy capacitors are employed. In order to cut off the
2 cells 1801 and 1802 from the six cells 1801 - 1806, the switches 1807 and 1808
are thrown to the left, and connected to the output of source follower 1809. The
value of Vout is essentially equal to the voltage VFG of
floating gate 1810, so that no voltage is applied to the two sides of capacitors
1811 and 1812 and no charge builds up. This is the same as if capacitors 1811 and
1812 were not present, and this is essentially equivalent to cutting these capacitors
off completely from the floating gate 1810. By the use of this method, the source
follower operates constantly with the largest gain even if the number of cells
is small, so that detection of the degree of movement can be conducted with a high
degree of accuracy.
Figure 19 shows an example of the operation of the circuitry when
the SEARCH (N, M) command is actually executed. Figure 19(a) shows, for example,
SEARCH (3,4); the data A3, A4, A5, and A6
within the A data memory 1901 are compared with the B data 1903. At this time,
only those correlation operational cells within the box indicated by the heavy
line 1904 are employed, so that control must be conducted which ignores the output
of the other cells.
Accordingly, in rows 1904a, 1904b, and the like, it is necessary
to cut off unnecessary cells using the method shown in Figure 17 and 18. Furthermore,
in other rows, for example, in rows 1905a, 1905b, and the like, there is no need
to input the operational results S-4, S-3, and the like into
the WTA (for example, 401, 808, or the like), so that the input into the WTA may
be fixed at a standard value of 0 V, VDD, or the like. If the WTA determines
the input having the smallest value as shown in Figure 5, then the inputs may be
set to VDD, while if the WTA determines the input having the largest
value as in Figure 11, these inputs may be set to 0 V. Figure 19(b) shows the case
of the command SEARCH (1, 6); the cells outside the box marked with the heavy
line may be cut off using control identical to that described above.
In the fourth embodiment described above, the case was described
in which operations were conducted in which the agreement between two groups of
data series having 8 data was determined, in order to keep the explanation simple;
however, this may be conducted with respect to data series containing a larger
number of data. Furthermore, data series may be used as the two groups of data
which are obtained from, for example, a 1 dimensional image sensor (an image sensor
in which a plurality of pixels are arranged in a series).
Using a camera or the like, incident rays may be split in two directions
using a micro lens, and these may be captured by different 1 dimensional image
sensors, and operations may be conducted using the data thereof as the two groups
of data series. By means of this, the dislocation and the focal point may be detected
and tight adjustment of the photo lens may be conducted, and thereby, an autofocus
function may be realized.
If autofocusing is conducted using the circuit of the present invention,
high speed control is realizable using extremely simple circuitry.
Figure 20 shows a fifth embodiment of the present invention.
The function of this embodiment is identical to that of the fourth
embodiment; a SEARCH (N, M) command is executed with respect to two types of data
series A and B. Reference 2001 indicates an A data series memory, while reference
2002 indicates a B data series memory. Reference 2003 indicates an analog data
shift register; here, this has the function of shifting the data to the left one
datum at a time.
Reference 2004 indicates 6 operational circuits arranged in a series,
which have the same function as those of Figure 2(b). The process of executing
a SEARCH (3, 4) command will be explained hereinbelow.
First, the entire A data series of memory 2001 is transferred to
shift register 2003, and these are shifted 3 places to the left, and stored in
operational circuit 2004. Next, the B data series are entered into shift register
2003, and are then transferred to operational circuits 2004 without being shifted.
In this way, the data A3, A4, A5, A6,
A7, and A8 and the data B1, B2, B3,
B4, B5, and B6 are incorporated into the cells
C1, C2, ..., C6 of the operational circuit, and
if the operation |Ai
-Bi+n| (n=-2) is conducted, the output
is transferred to floating gate 2006 of source follower circuit 2005 via capacitive
coupling. At this time, 4 comparator operations are necessary, so that it is necessary
to cut off the output of C5 and C6. The technology of Figure
17 and 18 is used to do this. The Vout obtained in this manner is S-2.
In the next operation, after the B data series have been moved to
shift register 2003, the data are shifted one place to the left, and transferred
to operational circuit 2004, and the same type of operation is conducted to determine
S-1. Similar operations are repeated to find S0, S1,
and S2 and the value of n having the smallest value is determined.
Sn is obtained in a successive time series; this may be
temporarily stored in an analog memory, and the value of n giving the minimum
value may be specified using a WTA such as that shown, for example, in Figure 5.
Alternatively, size comparisons may be conducted with respect to
the successively appearing values of Sn, and the value of n giving the
smaller value may be constantly followed. If the present embodiment of the present
invention is employed, it is possible to execute operations which find the degree
of agreement using smaller scale circuitry.
Furthermore, a shift register 2003 was employed here; however, a
switch matrix, for example, may be employed, and the prescribed data series may
be selected by means of switches and conducted to operational circuit 2004.
In the first through fifth embodiments described above, the analog
memory elements were not specified; however, it is of course the case that any
technology may be used for these. For example, analog data may be stored as a charge
in capacitors, and these may be read out by a source follower circuit. Alternatively,
the data may be stored in the base capacity of bi-polar transistors, and may be
read out by an emitter follower circuit. If necessary, storage may be conducted
as digital data, or a many-valued memory (for example, that of R. Au, et, al.,
ISSCC' 94 Digest of Technical Papers, pp. 270 - 271) maybe employed. Furthermore,
with respect to the image sensor, it is of course the case that any technology
may be employed, such as, for example, a CCD (charge conduction device), a MOS
image sensor, a bi-polar image sensor, or the like.
Furthermore, the circuit of the present invention can be realized
using extremely small scale circuitry, so that it may be integrated together with
a 1 dimensional image sensor or 2 dimensional image sensor on the same chip, and
is thus optimal for data processing in which image data are captured and operations
are instantly executed with respect to these data. It is especially suited to conducting
such operations. However, if the image sensor is on a separate chip, this does
not depart from the essential features of the present invention. After 2 dimensional
image data have been subjected one by one to A/D conversion, and incorporated in
a frame memory comprising DRAM sets or the like, processing may be conducted using
the circuit of the present invention. At this time, the D/A conversion employs
a ratio having sizes in which the capacity values are multiples of 2, such as 1,
2, 4, 8, ..., and the conversion takes advantage of the capacitive coupling with
the floating gate. That is to say, D/A conversion may be easily conducted at this
floating gate as the output of a controlled source follower circuit. Alternatively,
this floating gate may be the floating gate (603a, 604a) of, for example, an absolute
value operational circuit (Figure 6(a)). If this is done, there is no gain drop
as a result of the source follower, and it is possible to execute operations having
a higher degree of accuracy.
Furthermore, all the circuits of the present invention may be directly
integrated on a DRAM chip comprising frame memory, or may be placed in one portion
of a microprocessor chip. That is to say, the circuitry of the present invention
may be combined as one block within a purely digital circuit, and in the exclusively
digital operations, may be employed as an acceleration engine with respect to special
jobs which aids those operations requiring an enormous amount of time.
By means of the present invention, it becomes possible to conduct
operations with respect to an enormous amount of analog data such as image data
and to determine the logic thereof, and as a result, it is possible to execute
extremely high speed data processing using simple operational circuitry.