Title: Device and system for detecting abnormality
Abstract: An abnormality detection device includes small motion sensors that detect small motions of a person in a house; a data collecting unit that collects and stores sensor signals from the small motion sensors as sensor patterns, and a Markov chain operating unit 33 that transforms the sensor patterns into a cluster sequence by vector-quantizing input patterns which are obtained by averaging and normalizing the sensor patterns, and calculates a transition number matrix and a duration time distribution of a Markov and so on using a Markov chain model. The abnormality detection device also includes a comparing unit that calculates a characteristic amount (Euclid distance and average log likelihood in an appearance frequency of a Markov chain and an average log likelihood to the duration time distribution of a Markov chain) of a sample activity as against a daily activity based on the obtained transition number matrix and the duration time distribution and so on.
Patent Number: 6,989,742 Issued on 01/24/2006 to Ueno, et al.
| Inventors:
|
Ueno; Reiko (Takarazuka, JP);
Kaneda; Noriko (Kobe, JP);
Omori; Takashi (Sapporo, JP);
Hara; Kousuke (Hachioji, JP);
Yamamoto; Hiroshi (Shijonawate, JP);
Inoue; Shigeyuki (Kyotanabe, JP);
Tanaka; Shinji (Ibaraki, JP)
|
| Assignee:
|
Matsushita Electric Industrial Co., Ltd. (Osaka, JP)
|
| Appl. No.:
|
326447 |
| Filed:
|
December 23, 2002 |
Foreign Application Priority Data
| Dec 25, 2001[JP] | 2001-392921 |
| Apr 12, 2002[JP] | 2002-111292 |
| Current U.S. Class: |
340/511; 340/539.14; 340/539.25; 348/153; 348/154; 348/155; 348/159 |
| Current Intern'l Class: |
G08B 29/00 (20060101) |
| Field of Search: |
340/511,539.14,539.25
348/155,153,154,159
|
References Cited [Referenced By]
U.S. Patent Documents
| 4737847 | Apr., 1988 | Araki et al.
| |
| 5289562 | Feb., 1994 | Mizuta et al.
| |
| 5396437 | Mar., 1995 | Takahashi.
| |
| 5737486 | Apr., 1998 | Iso.
| |
| 5789925 | Aug., 1998 | Yokotani et al.
| |
| 5841946 | Nov., 1998 | Naito et al.
| |
| 6002994 | Dec., 1999 | Lane et al.
| |
| 6028626 | Feb., 2000 | Aviv.
| |
| 6396535 | May., 2002 | Waters.
| |
| 6625383 | Sep., 2003 | Wakimoto et al.
| |
| 6816186 | Nov., 2004 | Luke et al.
| |
| 2002/0171551 | Nov., 2002 | Eshelman et al.
| |
| 2003/0058111 | Mar., 2003 | Lee et al.
| |
| 2003/0065409 | Apr., 2003 | Raeth et al.
| |
| Foreign Patent Documents |
| 1 071 055 | Jan., 2001 | EP.
| |
| 1071055 | Jan., 2001 | EP.
| |
| 2001/-067576 | Mar., 2001 | JP.
| |
Primary Examiner: Wu; Daniel
Assistant Examiner: Walk; Samuel J.
Attorney, Agent or Firm: Wenderoth, Lind & Ponack, L.L.P.
Claims
What is claimed is:
1. An abnormality detection device that detects an occurrence of an abnormality
in an event under observation, said abnormality detection device comprising:
an input unit operable to acquire a sequence of an input pattern that is data
depending upon the event;
a transition analyzing unit operable to analyze a characteristic of a transition
in the sequence of the input pattern acquired by said input unit;
a comparing unit operable to compare the characteristic of the transition analyzed
by said transition analyzing unit with a predetermined reference value, and to
judge that an abnormality has occurred in the event when the characteristic and
the reference value are not approximate to each other within a predetermined range;
and
an output unit operable to output an occurrence of the abnormality when said
comparing unit judges that the abnormality has occurred;
wherein said input unit includes a plurality of small motion detection sensors
located in a plurality of places in a house, and said input unit is operable to
acquire, as the input pattern, a sensor pattern that is a combination of data indicating
whether or not there is a small motion outputted from a plurality of said plurality
of small motion detection sensors; and
wherein said transition analyzing unit is operable to calculate a Markov chain
in the sequence of the input pattern and to analyze a characteristic of the calculated
Markov chain as the analysis.
2. The abnormality detection device according to claim 1,
wherein said transition analyzing unit is operable to calculate an appearance
frequency for each type of the transition in the sequence of the input pattern
as the analysis, and
said comparing unit is operable to judge whether or not an abnormality has occurred
by comparing the input pattern acquired by said input unit with a reference input
pattern for the appearance frequency.
3. The abnormality detection device according to claim 2,
wherein said transition analyzing unit is operable to calculate a probability
distribution for each type of the transition as the appearance frequency, and
said comparing unit is operable to calculate a distance between the input pattern
acquired by said input unit and a reference input pattern for the probability distribution,
and to judge that an abnormality has occurred when the calculated distance exceeds
a predetermined value.
4. The abnormality detection device according to claim 2,
wherein said transition analyzing unit is operable to calculate probability distribution
for each type of the transition as the appearance frequency, and
said comparing unit is operable to calculate a likelihood of the input pattern
acquired by said input unit to the reference input pattern for the probability
distribution, and to judge that an abnormality has occurred when the calculated
likelihood is a predetermined value or less.
5. The abnormality detection device according to claim 1,
wherein said transition analyzing unit calculates a duration time distribution
which is a distribution of time that is required for the transition in the sequence
of the input pattern as the analysis, and
said comparing unit is operable to judge whether or not an abnormality has occurred
by comparing the input pattern acquired by said input unit with a reference input
pattern for the duration time distribution.
6. The abnormality detection device according to claim 5, wherein said comparing
unit is operable to calculate a likelihood of the input pattern acquired by said
input unit to the reference input pattern for the duration time distribution, and
to judge that an abnormality has occurred when the calculated likelihood is a predetermined
value or less.
7. The abnormality detection device according to claim 1, further comprising
a clustering unit operable to transform the sequence of the input pattern acquired
by said input unit into a cluster sequence that is a predetermined representative
input pattern,
wherein said transition analyzing unit is operable to analyze the characteristic
in the cluster sequence which is transformed by said clustering unit.
8. The abnormality detection device according to claim 7, wherein said clustering
unit is operable to obtain the cluster sequence after averaging and normalizing
the sequence of the input pattern acquired by said input unit.
9. The abnormality detection device according to claim 8, wherein said clustering
unit is operable to obtain the cluster sequence after specifying all the clusters
by vector-quantizing the sequence of the input pattern obtained by the normalization.
10. The abnormality detection device according to claim 1, wherein the reference
value is a value, for a reference event, obtained by acquiring the sequence of
the input pattern in advance through said input unit and analyzing the characteristic
through said transition analyzing unit.
11. The abnormality detection device according to claim 10, wherein the reference
value is a value determined based on data given by an operator which is learned
with a teacher and the input pattern which is acquired repeatedly by the said input unit.
12. The abnormality detection device according to claim 1, wherein the reference
value is a value specified by an operator.
13. The abnormality detection device according to claim 1, wherein said input
unit is operable to assume that a new sensor pattern has occurred each time the
sensor pattern changes, and to acquire a sequence of an input pattern corresponding
to a sequence of the sensor pattern.
14. The abnormality detection device according to claim 13, wherein said input
unit is operable to acquire the input pattern by averaging and normalizing the
sensor pattern in the time domain.
15. The abnormality detection device according to claim 1, wherein said input
unit is operable to acquire the sequence of the input pattern from an operator.
16. The abnormality detection device according to claim 1, wherein said output
unit is operable to report to a predetermined destination via a transmission channel
that the abnormality has occurred.
17. The abnormality detection device according to claim 1,
wherein said transition analyzing unit is operable to calculate an appearance
frequency of the Markov chain as the analysis, and
said comparing unit is operable to judge whether or not an abnormality has occurred
by comparing the input pattern acquired by said input unit with a reference input
pattern for the appearance frequency.
18. The abnormality detection device according to claim 17,
wherein said comparing unit is operable to calculate an Euclid distance between
the input pattern acquired by said input unit and the reference input pattern for
the appearance frequency of the Markov chain, and to judge that an abnormality
has occurred when the calculated Euclid distance exceeds a predetermined value.
19. The abnormality detection device according to claim 18, wherein said comparing
unit is operable to calculate a likelihood of the input pattern acquired by said
input unit to the reference input pattern for the appearance frequency of the Markov
chain, and to judge that an abnormality has occurred when the calculated likelihood
is a predetermined value or less.
20. The abnormality detection device according to claim 1,
wherein said transition analyzing unit is operable to calculate a duration time
distribution of the Markov chain as the analysis, and
said comparing unit is operable to judge whether or not an abnormality has occurred
by comparing the input pattern acquired by said input unit with the reference input
pattern for the duration time distribution.
21. The abnormality detection device according to claim 20, wherein said comparing
unit is operable to calculate a likelihood of the input pattern acquired by said
input unit to the reference input pattern for the duration time distribution of
the Markov chain, and to judge that an abnormality has occurred when the calculated
likelihood is a predetermined value or less.
22. The abnormality detection device according to claim 1,
wherein the reference value is a collection of local reference values in each
time interval which satisfies a predetermined condition, and
said comparing unit is operable to judge whether or not an abnormality has occurred
by comparing the characteristic with the local reference value in each time interval
which satisfies the predetermined condition.
23. The abnormality detection device according to claim 22, further comprising:
a local reference value generating unit operable to generate the local reference
value in each time interval which satisfies the predetermined condition by acquiring
a plurality of the sequences of the input patterns through said input unit, resolving
the sequences in each time interval where the acquired input patterns are similar
to each other within a predetermined range, and collecting the resolved sequences
in each time interval which satisfies the predetermined condition; and
a local reference value selecting unit operable to predict and select an optimum
reference value from among all the local reference values belonging to each time
interval which satisfies the predetermined condition based on the input patterns
for abnormality detection,
wherein said comparing unit is operable to judge whether or not an abnormality
has occurred by comparing the characteristic with the local reference value selected
by said local reference value selecting unit in each time interval which satisfies
the predetermined condition.
24. The abnormality detection device according to claim 23, wherein said local
reference value generating unit is operable to sample a time interval with a high
correlation according to template matching by using a time window, and to resolve
the sequence in the each sampled time interval.
25. The abnormality detection device according to claim 23, wherein said local
reference value selecting unit is operable to calculate probability distributions
for each transition type in the sequences of the input patterns for abnormality
detection and the input patterns of the reference event, respectively, and to predict
and select the optimum local reference value based on a distance between the calculated
probability distributions.
26. An abnormality detection system comprising:
an abnormality detection device is operable to be located in a place where an
event for abnormality detection occurs;
a communication device operable to monitor an occurrence of an abnormality; and
a transmission channel operable to connect said abnormality detection device
and said communication device;
wherein said abnormality detection device includes:
an input unit operable to acquire a sequence of an input pattern that is data
depending upon the event;
a transition analyzing unit operable to analyze a characteristic of a transition
in the sequence of the input pattern acquired by said input unit;
a comparing unit operable to compare the characteristic of the transition analyzed
by said transition analyzing unit with a predetermined reference value, and to
judge that an abnormality has occurred in the event when the characteristic and
the reference value are not approximate to each other within a predetermined range;
and
an output unit operable to output an occurrence of an abnormality when said comparing
unit judges that the abnormality has occurred;
wherein said input unit includes a plurality of small motion detection sensors
located in a plurality of places in a house, and said input unit is operable to
acquire, as the input pattern, a sensor pattern that is a combination of data indicating
whether or not there is a small motion outputted from a plurality of said plurality
of small motion detection sensors;
wherein said transition analyzing unit is operable to calculate a Markov chain
in the sequence of the input pattern and to analyze a characteristic of the calculated
Markov chain as the analysis; and
wherein said communication device includes:
a receiving unit operable to receive a report from said abnormality detection
device; and
a showing unit operable to show an operator that the report has been received
when said receiving unit receives the report.
27. An abnormality detection system comprising:
an abnormality detection device operable to be located in a place where an event
for abnormality detection occurs;
a communication device operable to monitor an occurrence of an abnormality; and
a transmission channel operable to connect said abnormality detection device
and said communication device;
wherein said abnormality detection device includes:
an input unit operable to acquire a sequence of an input pattern that is data
depending upon the event; and
a sending unit operable to send the input pattern acquired by said input unit
to said communication device via said transmission channel;
wherein said input unit includes a plurality of small motion detection sensors
located in a plurality of places in a house, and said input unit is operable to
acquire, as the input pattern, a sensor pattern that is a combination of data indicating
whether or not there is a small motion outputted from a plurality of said plurality
of small motion detection sensors; and
wherein said communication device includes:
a receiving unit operable to receive the input pattern sent from said sending
unit of said abnormality detection device;
a transition analyzing unit operable to analyze a characteristic of a transition
in the sequence of the input pattern received by said receiving unit by calculating
a Markov chain in the sequence of the input pattern and analyzing a characteristic
of the calculated Markov chain as the analysis;
a comparing unit operable to compare the characteristic of the transition analyzed
by said transition analyzing unit with a predetermined reference value, and to
judge that an abnormality has occurred in the event when the characteristic and
the reference value are not approximate to each other within a predetermined range;
and
a showing unit operable to show an operator the occurrence of the abnormality
when said comparing unit judges that the abnormality has occurred.
28. The abnormality detection system according to claim 26, wherein said input
unit includes an operation state detection sensor operable to detect an operation
state of equipment which is located in the place, and to acquire a sensor signal
from said operation state detection sensor as the input pattern.
29. The abnormality detection system according to claim 26, wherein:
the place is a house;
said abnormality detection system further comprises:
a home network operable to connect a plurality of electrical household appliances
which are located in the house; and
a controller operable to control a plurality of the electrical household appliances
via said home network; and
said input unit is operable to detect operation states of a plurality of the
electrical household appliances via said home network and to acquire the detected
operation states as the input patterns.
30. The abnormality detection system according to claim 26, wherein:
the place is a house;
said abnormality detection system further comprises a home network operable to
connect a plurality of electrical household appliances which are located in the
house;
said abnormality detection device further includes a controller operable to control
a plurality of the electrical household appliances via said home network; and
said input unit is operable to detect operation states of a plurality of the
electrical household appliances via said home network and to acquire the detected
operation states as the input patterns.
31. An abnormality detection method for detecting an occurrence of an abnormality
in an event under observation, said abnormality detection method comprising:
acquiring a sequence of an input pattern that is data depending upon the event;
analyzing a characteristic of a transition in the sequence of the input pattern
acquired in said acquiring of the sequence of the input pattern;
comparing the characteristic of the transition analyzed in said analyzing of
the characteristic of the transition with a predetermined reference value, and
judging that an abnormality has occurred in the event when the characteristic and
the reference value are not approximate to each other within a predetermined range;
and
outputting an occurrence of the abnormality when said judging that the abnormality
has occurred judges that the abnormality has occurred;
wherein in said acquiring of the sequence of the input pattern, a plurality of
small motion detection sensors located in a plurality of places in a house are
utilized for acquiring the input pattern, and a sensor pattern that is a combination
of data indicating whether or not there is a small motion outputted from a plurality
of the small motion detection sensors is acquired as the input pattern; and
wherein in said analyzing of the characteristic of the transition, a Markov chain
in the sequence of the input pattern is calculated, and a characteristic of the
calculated Markov chain is analyzed as the analysis.
32. The abnormality detection method according to claim 31, wherein:
in said analyzing of the characteristic of the transition, an appearance frequency
for each type of the transition in the sequence of the input pattern is calculated
as the analysis; and
in said comparing of the characteristic of the transition, whether or not an
abnormality has occurred is judged by comparing the input pattern acquired in said
acquiring of the sequence of the input pattern with a reference input pattern for
the appearance frequency.
33. The abnormality detection method according to claim 31, wherein:
said analyzing of the characteristic of the transition, a duration time distribution
that is a distribution of time which is required for the transition in the sequence
of the input pattern is calculated as the analysis; and
in said comparing of the characteristic of the transition, whether or not an
abnormality has occurred is judged by comparing the input pattern acquired in said
acquiring of the sequence of the input pattern with a reference input pattern for
the duration time distribution.
34. The abnormality detection method according to claim 31, further comprising
transforming the sequence of the input pattern acquired in said acquiring of the
sequence of the input pattern into a cluster sequence that is a predetermined representative
input pattern;
wherein in said analyzing of the characteristic of the transition, the characteristic
in the cluster sequence which is transformed in said transforming of the sequence
of the input pattern is analyzed.
35. The abnormality detection method according to claim 31, wherein the reference
value is a value, for a reference event, which is obtained by
acquiring the sequence of the input pattern in advance in said acquiring of the
of the sequence of the input pattern, and
analyzing the characteristic in said analyzing of the characteristic of the transition.
36. The abnormality detection method according to claim 31, wherein the reference
value is a value specified by an operator.
37. The abnormality detection method according to claim 31, wherein in said acquiring
of the sequence of the input pattern, the sequence of the input pattern is acquired
from an operator.
38. The abnormality detection method according to claim 31, wherein said outputting
of the occurrence of the abnormality reports that the abnormality has occurred,
to a predetermined destination via a transmission channel.
39. The abnormality detection method according to claim 31, wherein:
the reference value is a collection of local reference values in each time interval
which satisfies a predetermined condition; and
in said judging that an abnormality has occurred, whether or not an abnormality
has occurred is judged by comparing the characteristic with the local reference
value in each time interval which satisfies the predetermined condition.
40. A program for detecting that an abnormality has occurred in an event under
observation, said program causing a computer to execute:
acquiring a sequence of an input pattern that is data depending upon the event;
analyzing a characteristic of a transition in the sequence of the input pattern
acquired in said acquiring of the sequence of the input pattern;
comparing the characteristic of the transition analyzed in said analyzing of
the characteristic of the transition with a predetermined reference value, and
judging that an abnormality has occurred in the event when the characteristic and
the reference value are not approximate to each other within a predetermined range;
and
outputting an occurrence of the abnormality when said judging that the abnormality
has occurred judges that the abnormality has occurred;
wherein in said acquiring of the sequence of the input pattern, a plurality of
small motion detection sensors located in a plurality of places in a house are
utilized for acquiring the input pattern, and a sensor pattern that is a combination
of data indicating whether or not there is a small motion outputted from a plurality
of the small motion detection sensors is acquired as the input pattern; and
wherein in said analyzing of the characteristic of the transition, a Markov chain
in the sequence of the input pattern is calculated, and a characteristic of the
calculated Markov chain is analyzed as the analysis.
41. The abnormality detection system according to claim 27, wherein said input
unit includes an operation state detection sensor operable to detect an operation
state of equipment which is located in the place, and to acquire a sensor signal
from said operation state detection sensor as the input pattern.
42. The abnormality detection system according to claim 27, wherein:
the place is a house;
said abnormality detection system further comprises:
a home network operable to connect a plurality of electrical household appliances
which are located in the house; and
a controller operable to control a plurality of the electrical household appliances
via said home network; and
said input unit is operable to detect operation states of a plurality of the
electrical household appliances via said home network and to acquire the detected
operation states as the input patterns.
43. The abnormality detection system according to claim 27, wherein:
the place is a house;
said abnormality detection system further comprises a home network operable to
connect a plurality of electrical household appliances which are located in the
house;
said abnormality detection device further includes a controller operable to control
a plurality of the electrical household appliances via said home network; and
said input unit is operable to detect operation states of a plurality of the
electrical household appliances via said home network and to acquire the detected
operation states as the input patterns.
Description
BACKGROUND OF THE INVENTION
(1) Field of the Invention
The present invention relates to an abnormality detection device, and more particularly
to a device, method, system and executable program which are suitable for detecting
an abnormality such as an unusual human activity which has occurred in an elderly
person who lives alone in a house.
(2) Description of the Related Art
Various attempts have been made for detecting an abnormality which has occurred
in an event such as the operation of a machine and human behavior. For example,
it is an important technology, particularly in rapidly aging Japan, to monitor
the activities of an elderly person who lives alone and to detect any occurrence
of his unusual activity. One of such conventional technologies is "Abnormality
Report System" which is disclosed in the Japanese Laid-Open Patent Application
No. 2001-67576.
This conventional system includes, for monitoring the daily life of an elderly
person or a sick person who lives alone, (a) a sensor unit which is placed in the
restroom in the house of the person that is subject to monitoring and which outputs
a predetermined signal when detecting the use of the restroom, (b) a first communication
means (a wireless terminal device) which is placed in the restroom and which outputs
a predetermined signal when receiving the signal from the sensor unit, and (c)
a second communication means (a main device) which includes a monitoring timer
that starts clocking when the signal is received from the first communication means
and which reports the occurrence of an abnormality to the monitoring center when
the signal from the first communication means is interrupted for a predetermined
time period or more. When the elderly person faints due to a disease or the like
and cannot move at all, he does not use the restroom for the predetermined time
period or more, and thereby, it is detected that something abnormal has occurred
in him.
However, the above-cited conventional system is based on his behavior characteristic
that he always goes to the restroom within the predetermined time period. Therefore,
steps are required to eliminate an exceptional case when the elderly person does
not go to the restroom for the predetermined time period or more even in normal
situations, such as when he goes out and he does his needs with a chamber pot.
Furthermore, since this conventional system only detects unusual human
activities in a building with a restroom, there is a problem in that the uses and
available opportunities thereof are extremely limited. In other words, the conventional
system can detect neither unusual human activities in a place without a restroom
or in a time period when the restroom is not used, nor abnormal motions of animals
such as a pet or machines which do not use a restroom.
SUMMARY OF THE INVENTION
In order to solve the aforesaid problem, the object of the present invention
is
to provide an abnormality detection device, method, system and executable program
with a variety of applications that can detect an occurrence of the abnormality
of objects and events under observation, independently of the types and numbers
thereof and the space (such as a place and a time zone) where they are observed.
In order to achieve the stated object, the abnormality detection device according
to the present invention is a device that detects the occurrence of an abnormality
in an event under observation. The abnormality detection device comprises: an input
unit operable to acquire a sequence of an input pattern that is data which depends
upon the event; a transition analyzing unit operable to analyze a characteristic
of a transition in the acquired sequence of the input pattern; a comparing unit
operable to compare the analyzed characteristic of the transition with a predetermined
reference value, and judge that an abnormality has occurred in the event when the
characteristic and the reference value are not approximate to each other within
a predetermined range; and an output unit operable to output the occurrence of
an abnormality when the comparing unit judges that an abnormality has occurred.
In other words, the occurrence of an abnormality is detected by acquiring a sequence
(time series) of input patterns that are the data depending upon changes of an
event such as a motion of a person or a thing to be observed, focusing on the transition
of the patterns in the sequence, sampling the characteristic amount in the transition
and comparing the characteristic amount with that of the normal case. Here, the
"event" means an event that can be represented as data which can be processed by
a computer based on a signal from an equipment sensor, a report (data entry) by
a person, and others, and is typically a "human activity" in a house.
When an abnormality is detected based on the human activities in the house,
the following approach is taken in the present invention.
A description of a personal daily life and the detection of unusual events in
his
life are made by using sensor information that is obtained in a house (an intelligent
house) where small motion sensors are placed in a plurality of predetermined places.
The objective here is that a computer understands the daily life which is individually
customized because human behavior varies greatly from person to person.
On the other hand, there is symbol processing where discretized environments
are
used for prediction. It is necessary for us to predict events in an outside world
for our lives, and such an intellectual function may be used for modeling and information
processing in the equipment which matches with human beings and supports them.
The intelligent house which is equipped with sensors that detect human presence
and activities is considered to be something that is equipped with human outward
sensory organs inwardly, and shows effectiveness as a place where the intellectual
information processing for supporting human activities is applied. Therefore, based
on the data which is actually measured in the intelligent house, unusual behavior
is detected by using the likelihood of the activity sequence as an information
processing model similar to that of human beings.
More specifically, discretization of continuous sensory data that is a basis
of symbol processing in human brains is performed, an environmental model is constructed
based on the transition between the discrete states, and then, the likelihood of
the actual activity is predicted based on that model. Accordingly, an abnormal
activity (an activity with a low likelihood) can be found based on the comparison
with the learned personal data, and therefore, an abnormal activity may be detected
without disturbing his daily life. It is preferable to use vector quantization
for discretization and a Markov process model for the environmental model.
Furthermore, based on the assumption that a human life is made up of
a plurality of daily activities and they are triggered according to the situation,
the sensor sequence is discretized into a reproducible time interval and a local
daily activity template in that time interval is generated. Since an image processing
method is applied to automatic sampling of time intervals, highly correlative time
intervals are sampled by template matching using a time window. As a result, daily
life representation with a hierarchy in the time domain can be constructed.
As a verification of the daily activity template, unusual activities were detected.
The unusual activities were detected by detecting the differences between the daily
activity template and the actual activities. More specifically, methods such as
(i) a comparison between the probability distribution based on the daily activity
template and the likelihood of the actual activities and (ii) a measurement of
the difference between the daily activity template that is calculated in the local
time interval and the global daily activity template using the probability distribution
distance are used so as to evaluate them.
The present invention is not only realized by dedicated hardware such as the
above-mentioned abnormality detection device, but is also realized as an abnormality
detection method including steps of the characteristic constituent elements, or
as a program that causes a general-purpose computer to execute these steps, or
as an abnormality detection system including the abnormality detection device and
the receiver of the abnormality report.
FURTHER INFORMATION ABOUT THE TECHNICAL BACKGROUND OF THE INVENTION
The following applications are incorporated by reference:
Japanese Patent Application Ser. No. 2001-392921 filed Dec. 25, 2001;
Japanese Patent Application Ser. No. 2001-111292 filed Apr. 12, 2002.
BRIEF DESCRIPTION OF THE DRAWINGS
These and other objects, advantages and features of the present invention will
become more apparent from the following description when taken in conjunction with
the accompanying drawings which illustrate a specific embodiment of the invention.
FIG. 1 is a block diagram showing an overall structure of the abnormality detection
system according to the embodiment of the present invention.
FIG. 2 is a plan view of an intelligent house showing locations of small motion
detection sensors.
FIG. 3 is a functional block diagram showing a structure of the abnormality
detection device according to the present invention.
FIG. 4 shows values of sensor signals from the small motion detection sensors
(sequences of sensor patterns).
FIG. 5 is a flowchart showing the operation of a Markov chain operating unit.
FIG. 6 is a flowchart showing a detailed calculation procedure for averaging
sensor patterns.
FIG. 7 is a flowchart showing a detailed calculation procedure for normalizing
the averaged sensor pattern.
FIG. 8 is a flowchart showing a detailed calculation procedure for vector quantizing
input patterns.
FIG. 9 is a plan view of the intelligent house showing locations of clusters.
FIG. 10 is a flowchart showing a calculation procedure of a cluster sequence.
FIG. 11 is a flowchart showing a calculation procedure of a transition number
matrix in an appearance frequency of a Markov chain.
FIG. 12 is a flowchart showing a first half of a calculation procedure of duration
time distribution of a Markov chain.
FIG. 13 is a flowchart showing the second half of the calculation procedure
of the duration time distribution of the Markov chain.
FIG. 14 is a data flow diagram showing a calculation procedure of distance in
the appearance frequency of the Markov chain that is performed by the distance
calculating unit of the comparing unit.
FIG. 15 is a flowchart showing a detailed calculation procedure for normalization
in the above distance calculation.
FIG. 16 is a flowchart showing a detailed calculation procedure for calculating
a Euclid distance in the above distance calculation.
FIG. 17 is a data flow diagram showing a calculation procedure of likelihood
in the appearance frequency of the Markov chain that is performed by the likelihood
calculating unit of the comparing unit.
FIG. 18 is a flowchart showing a detailed calculation procedure in the above
likelihood calculation.
FIG. 19 is a data flow diagram showing a calculation procedure of likelihood
for the duration time distribution of the Markov chain that is performed by the
likelihood calculating unit of the comparing unit.
FIG. 20 is a flowchart showing a detailed calculation procedure in the above
likelihood calculation.
FIG. 21 is a plan view of the intelligent house showing the transition number
matrix of a daily activity template in Experiment 1.
FIG. 22 is a plan view of the intelligent house showing the transition number
matrix of an unusual activity in Experiment 1.
FIG. 23 is a diagram showing the distance of the appearance frequency of the
Markov chain for each experiment number (sample) in the Experiment 1.
FIG. 24 is a diagram showing the distance of the appearance frequency of the
Markov chain for each experiment number (sample) in Experiment 2.
FIG. 25 is a diagram showing the likelihood for the duration time distribution
of the Markov chain in Experiment 3.
FIG. 26 is a plan view of the apartment style intelligent house showing the
transition number matrix of the daily activity template in Experiment 4.
FIG. 27 is a diagram showing the likelihood of the appearance frequency of the
Markov chain on every experiment day and time in Experiment 4.
FIG. 28 is a diagram showing the activities of the subject person based on his
self record in Experiment 5.
FIG. 29 is a diagram showing the likelihood of the appearance frequency of the
Markov chain on every experiment day and time in Experiment 5.
FIG. 30 is a functional block diagram showing a detailed structure of the Markov
chain operating unit of the abnormality detection system according to a modification
of the present invention.
FIG. 31 is a flowchart showing the operation of a local template generating
unit of the abnormality detection system shown in FIG. 30.
FIGS. 32A-D are diagrams showing small motion sensor sequences for explaining
a local template.
FIGS. 33A-J are diagrams showing the calculation results of the correlation
of all the times for four types of the sensor sequences as shown in FIGS. 32A-D.
FIGS. 34A-J are diagrams showing examples of the binarization and labeling
of the correlation shown in FIGS. 33A-J and the sampling of each time period thereof.
FIGS. 35A-H are diagrams showing examples of the sampling of similar sequence
intervals from four types of the sequences for morning scenarios.
FIGS. 36A-D are diagrams showing examples of the sampling of similar sequence
intervals from two types of the sequences for evening scenarios.
FIG. 37 is a flowchart showing the operation of a local template predicting
unit of the abnormality detection device shown in FIG. 30.
FIGS. 38A-H are diagrams showing reliability and predicted templates.
FIGS. 39A-H are diagrams showing input cluster sequences and predicted templates.
FIGS. 40A-F are diagrams showing the sequences for sampling six types of the
local templates, FIGS. 40G-L are diagrams showing local templates before selecting
the local templates corresponding to FIGS. 40A-F, and FIGS. 40M-R are diagrams
showing local templates after selecting the local templates corresponding to FIGS. 40A-F.
FIG. 41 is a diagram showing a comparison of the average log likelihood between
the case where a global template is used and where a local template is used in
Experiment 1.
FIG. 42 is a diagram showing a comparison of the average log likelihood between
the case where the global template is used and the local template is used in Experiment 2.
FIG. 43 is a diagram showing a comparison of the average log likelihood between
the case where the global template is used and where the local template is used
in Experiment ½.
FIG. 44 is a diagram showing the configuration of the abnormality detection
system according to a modification of the present embodiment that detects the occurrence
of an abnormality in human activities by using various sensor signals.
FIG. 45 is a diagram showing the configuration of the abnormality detection
system according to another modification of the present embodiment that performs
decentralized processing in which the intelligent house collects data and the monitoring
center or the like analyzes and judges the occurrence of an abnormality.
FIG. 46 is a diagram showing the configuration of the abnormality detection
system according to still another modification of the present embodiment including
the abnormality report device that collects the sensor signals via the home network.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The embodiment of the present invention will now be explained in detail with
reference to the drawings.
FIG. 1 is a block diagram showing an overall structure of the abnormality detection
system according to the embodiment of the present invention. The abnormality detection
system
10 is a system that detects the occurrence of an abnormality, that
is, unusual human activities, in a house and reports the detected abnormality to
a specific contact station. The abnormality detection system
10 is characterized
by detecting unusual activities without limiting the places in the house or the
time of day, and includes an intelligent house
20, a monitoring center
50
and a cell phone
60 which are connected to each other via a communication
network
40.
The intelligent house
20 is a two-story house shown in a plan view of
FIG.
2. The intelligent house
20 includes small motion detection
sensors
25a-25c that are placed at predetermined locations
(20 locations indicated by black circles in the example of FIG. 2) in the house,
and an abnormality detection device
30 that detects unusual human activities
in the house by monitoring sensor signals which are outputted from the small motion
detection sensors
25a-25c and reports its detection
results to the monitoring center
50 and a predetermined contact station
(the cell phone
60 in this case) via the communication network
40.
The small motion detection sensors
25a-25c are infrared
sensors that detect infrared rays emanating from the human body, for instance,
and when there exists a person in the detection space (in the restroom, for instance)
as determined by the orientation of each sensor and they detect the motion of the
person, the small motion detection output sensor signals indicating the detection
to the abnormality detection device
30.
The monitoring center
50 is a nursing-care center or the like that keeps
centralized monitoring on the occurrence of unusual activities in a plurality of
the target intelligent house
20, and includes a device that outputs, to
an alarm or a display device, its receipt of the report from the abnormality detection
device
30 via the communication network
40. The cell phone
60
is one of the contact stations that is registered in the abnormality detection
device
30, and receives the report that is sent from the abnormality detection
device
30 (such as an e-mail indicating that unusual activity has occurred
in the intelligent house
20).
FIG. 3 is a functional block diagram showing the structure of the abnormality
detection device
30 shown in FIG.
1. The abnormality detection device
30 is a controller which is connected to the small motion detection sensors
25a-25c in the house and the communication network
40, and includes an input unit
31, a data collecting unit
32,
a Markov chain operating unit
33, a comparing unit
34 and an output
unit
35.
The input unit
31 is an input device such as an operation panel, and is
used by the operator for giving the data to the abnormality detection device
30
instead of the small motion detection sensors
25a-25c,
or for setting various parameters for the abnormality detection device
30.
The data collecting unit
32 is comprised of a data logger or a hard disk
that receives and records the sensor signals that are outputted from the small
motion detection sensors
25a-25c. The data collecting
unit
32 includes a sample activity data storage unit
32a a
that stores the sensor signals that are outputted from the small motion detection
sensors
25a-25c and the input data that is outputted
from the input unit
31 as the data of sample activities according to predetermined
conditions, and a daily activity data storage unit
32b that stores
them as the data of daily activities.
Here, the "sample activity" means an activity that is to be a target of monitoring
the occurrence of an abnormality, and the "daily activity" means an activity under
the normal condition (the activity that is the measure (reference or standard)
for judging whether or not unusual activity has occurred).
The Markov chain operating unit
33 is a processing unit which is realized
by a CPU, a memory or the like that executes data processing based on the control
program, and calculates the transition number matrix and the duration time distribution
of a Markov chain in the daily activities as a characteristic amount by performing
operations (which will be described later) on the sensor patterns that are stored
in the daily activity data storage unit
32b prior to monitoring the
occurrence of an unusual activity (in the mode of creating the reference data).
In the mode of monitoring the occurrence of an unusual activity (monitoring mode),
the Markov chain operating unit
33 performs the same operation on the sequence
(time series) of the sensor patterns that are stored in the sample activity data
storage unit
32a, or directly on the sensor patterns which are inputted
by the small motion detection sensors
25a-25c, so as
to calculate the sequence of clusters, the transition number matrix of the Markov
chain and so on as the characteristic amount in the sample activities. The obtained
results are respectively outputted to the comparing unit
34.
"Markov chain" means the sequence of the events when the nth event is determined
in relation to the previous events in the sequence of the events, and in the present
embodiment, it corresponds to the sequence of the sensor pattern indicating the
human activities (such as places where he moves) in the house (including the input
pattern and the sequence of clusters which are obtained by performing a predetermined
operation on that sequence). The "transition number matrix" of a Markov chain is
a matrix indicating the number of transitions from the past events to that event.
The "duration time distribution" of a Markov chain is a histogram of the duration
time of each element (state transition) of the transition number matrix in the
subject sequence. The "input pattern" is a pattern that is obtained by averaging
and normalizing the sensor pattern (which will be described later). Further, the
"cluster" is a specific number of patterns which are representative of all the
input patterns, and are used for mapping an enormous types of input patterns into
a specific number (30 in the present embodiment) of the representative patterns.
The abnormality detection device
30 does not always require the distinction
of whether the subject event belongs to the daily activity or the sample activity,
and may process the stored and averaged data as the data in the daily activity.
In other words, the data collecting unit
32 may store the collected data
regardless of whether the collected data is the data of the sample activity or
the daily activity, and the Markov chain operating unit
33 may calculate
the above-mentioned characteristics by regarding the data that is stored for a
fixed time period and averaged as the data in the daily activity and the individual
(daily) data as the data in the sample activity.
The comparing unit
34 is a processing unit which is realized by a CPU
or a memory that executes the data processing based on the control program. The
comparing unit
34 has a function of comparing the daily activity and the
sample activity, that is, the characteristics pertaining to the appearance frequency
and the duration time of the state transition in the Markov chain thereof, judging
that an unusual activity has occurred when the difference between them exceeds
a predetermined threshold (or the similarity thereof is a predetermined threshold
or less), and notifying the output unit
35 of the result. The comparing
unit
34 includes a distance calculating unit
34a and a likelihood
calculating unit
34b.
The distance calculating unit
34a calculates the distance in the
appearance frequency of the Markov chain based on the two transition number matrixes
that are outputted from the Markov chain operating unit
33, that is, the
transition number matrix of the Markov chain that is obtained in the daily activity
and the transition number matrix of Markov chain obtained in the sample activity.
On the other hand, the likelihood calculating unit
34b calculates
the likelihood in the appearance frequency of the Markov chain based on the transition
number matrix of the Markov chain that is obtained in the daily activity which
is outputted from the Markov chain operating unit
33 and the cluster sequence
that is obtained in the sample activity, or the likelihood calculating unit
34b
calculates the likelihood in the duration time distribution of the Markov chain
based on the duration time distribution of the Markov chain that is obtained in
the daily activity which is outputted from the Markov chain operating unit
33
and the cluster sequence that is obtained in the sample activity.
The output unit
35 is a CPU, a modem or the like that executes communication
control based on the control program. Upon receipt of the notice from the distance
calculating unit
34a and the likelihood calculating unit
34b
in the comparing unit
34, that is, the notice of the occurrence of an
unusual activity, the output unit
35 reports the received notice to the
pre-registered contact station (such as the monitoring center
50 and the
cell phone
60) by using the input unit
31 or the like via the communication
network
40. For example, the output unit
35 makes a call by using
a cell phone, and sends a message indicating the occurrence of an unusual activity
to a pre-registered e-mail address.
The output unit
35 has option functions of, according to the prior setting
by the input unit
31, (1) outputting not only the message indicating the
occurrence of unusual activity but also the information indicating what type of
unusual activity has occurred (such as the information indicating where the unusual
activity has occurred, in appearance frequency and/or duration time distribution,
or distance and/or likelihood, the value and the occurrence time thereof), (2)
alarming or displaying the occurrence of an unusual activity by an alarm bell or
a display board which is mounted on the abnormality detection device
30,
and (3) repeating the report until receiving the receipt data from the contact station.
The operation of the abnormality detection system
10 configured as described
above will now be explained in detail.
FIG. 4 shows values of sensor signals that are outputted from the small motion
detection sensors
25a-25c (a sequence of a sensor pattern)
and collected by the data collecting unit
32. More specifically, FIG. 4
shows sensor signals ("
1" is indicated when a person exists and he is in
small motion) that are outputted from the twenty small motion detection sensors
25a-25c which are placed in the intelligent house
20,
and the times when the sensor signals are changed. As individual sensor pattern
consists of twenty-dimensional elements ("
1" or "
0"), and every time
one element changes, a new sensor pattern is stored and accumulated in the sample
activity data storage unit
32a or the daily activity data storage
unit
32b.
These sensor patterns do not include the information on the arrangement of
the sensors. "
1" is indicated while each of the small motion detection sensors
25a-25c is in the state of detecting a small motion
until not detecting any motion, and "
0" is indicated while each of the small
motion detection sensors
25a-25c is in the state of
not detecting anything. These binary information indicating whether or not the
sensor (whose location is unknown) detects a small motion and the information of
the time when the event has occurred are the raw data that is inputted to the abnormality
detection device
30.
FIG. 5 is a flowchart showing the operations of the Markov chain operating unit
33. FIG. 5 shows an overall operation procedure of calculating the transition
number matrix of a Markov chain and the duration time distribution of the Markov
chain for the sequence of the sensor pattern shown in FIG.
4. Each step
of this procedure will be explained in detail in the order of steps.
The Markov chain operating unit
33 first reads out the sequence of the
sensor pattern from the sample activity data storage unit
32a or
the daily activity data storage unit
32b depending upon the operation
mode (monitoring mode and/or reference data creating mode), averages and normalizes
the read sensor pattern, and then transforms the sensor pattern into an input pattern.
More specifically, the sensor pattern b
t which is to be processed
includes N steps (b
t is a sequence of b when t=1, . . . N). Since the
information from the small motion detection sensors
25a-25c
sometimes occurs at intervals of several seconds, it is highly possible that
appropriate weights cannot be assigned to human activities if such information
is used as an input event as it is. Therefore, the obtained sensor pattern b
t
is averaged in the time domain. Specifically, as shown in the following expression
1, the sensor pattern is divided into time windows, weighted addition is respectively
performed according to a Gaussian function, and then the sensor pattern normalized.
sensor pattern: b
t (J-dimensional vector, t=1, . . . ,N)
averaged pattern: x
t(J-dimensional vector, t=1, . . . ,N)
variance: σ
data size: N
time window size: K
##EQU1##
##EQU2##
The detailed calculation procedure for averaging as shown in the first equation
of the above expression is shown in the flowchart in FIG. 6, and the detailed calculation
procedure for normalizing as shown in the second equation of the expression is
shown in the flowchart in FIG.
7. In these figures, b
t and X
t
are the data having twenty-dimensional array elements and the duration time
of the pat
