Apparatus and method for hand motion tracking and handwriting recognition Number:7,394,460 from the United States Patent and Trademark Office (PTO) owispatent Apparatus and method for handwriting recognition including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry receiving the acceleration indicating output and providing a velocity indicating output and a handwritten multiple character recognizer receiving the velocity indicating output and providing a multiple character recognition output indication.<H recognition, handwriting, Apparatus, and, m, 7394460.html, Patent, pto, 7394460

Title: Apparatus and method for hand motion tracking and handwriting recognition

Abstract: Apparatus and method for handwriting recognition including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry receiving the acceleration indicating output and providing a velocity indicating output and a handwritten multiple character recognizer receiving the velocity indicating output and providing a multiple character recognition output indication.

Patent Number: 7,394,460 Issued on 07/01/2008 to Vardi

Inventors: Vardi; Micha (Raanana, IL)
Assignee: I.C. + Technologies Ltd. (Raana, IL)

Appl. No.: 10/964,998

Filed: October 14, 2004

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number	Issue Date
09829230	Apr., 2001	6831632

Current U.S. Class: 345/179 ; 178/19.01; 345/156; 382/189

Current International Class: G09G 5/00 (20060101)

Field of Search: 345/173-179,156,158,169 382/181,312,313,183-189 178/18.01,18.1,19.01

References Cited [Referenced By]

U.S. Patent Documents


4128829	December 1978	Herbst et al.
4141073	February 1979	Tan
4142175	February 1979	Herbst et al.
4513437	April 1985	Chainer et al.
4695831	September 1987	Shinn
4787051	November 1988	Olson
4839838	June 1989	LaBiche et al.
5077802	December 1991	Plamondon
5159321	October 1992	Masaki et al.
5247137	September 1993	Epperson
5294792	March 1994	Lewis et al.
5301222	April 1994	Fujiwara
5434371	July 1995	Brooks
5453758	September 1995	Sato
5517579	May 1996	Baron et al.
5587558	December 1996	Matsushima
5587906	December 1996	McIver et al.
5598187	January 1997	Ide et al.
5781661	July 1998	Hiraiwa et al.
5902968	May 1999	Sato et al.
5926567	July 1999	Collins et al.
5981884	November 1999	Sato et al.
6084557	July 2000	Ishida et al.
6084577	July 2000	Sato et al.
6101280	August 2000	Reynolds
6130666	October 2000	Persidsky
6181329	January 2001	Stork et al.
6188392	February 2001	O'Connor et al.
6201903	March 2001	Wolff et al.
6212296	April 2001	Stork et al.
6229102	May 2001	Sato et al.
6335727	January 2002	Morishita et al.
6625314	September 2003	Okamoto

Foreign Patent Documents


0 691 623	Oct., 1996	EP
2 219 119	Nov., 1989	GB
WO 94/09447	Apr., 1994	WO
WO 94/18663	Aug., 1994	WO
WO 95/21436	Aug., 1995	WO
WO 99/22338	May., 1999	WO

Other References

IBM Technical Disclosure Bulletin, vol. 19, No. 7, Dec. 1976, pp. 2816-2817, "Electronic Calculator Based on Character Recognition of Input from Stylus Acceleration Dynamics". cited by other .
Hideo Taguchi, et al., "On-line recognition of handwritten signatures by feature extraction of pen movement", Systems & Computers in Japan, Wiley, Hoboken, NJ, US, vol. 20, No. 10, Oct. 1989. cited by other.

Primary Examiner: Liang; Regina
Attorney, Agent or Firm: Mass; Clifford J.

Parent Case Text

This is a continuation of application Ser. No. 09/829,230 filed on Apr. 9, 2001, now U.S. Pat. No. 6,831,632 claims the benefit thereof and incorporates the same by reference.

Claims

The invention claimed is:

1. A method comprising: providing an acceleration indicating output from a hand-held device; receiving said acceleration indicating output and providing a proportionally normalized velocity indicating output; and receiving said velocity indicating output and providing a character recognition output indication.

2. A method according to claim 1 and also comprising at least one of a computation related to the character recognition output indication, a database operation related to the character recognition output indication, communication of character recognition output indication and a control operation based on the character recognition output indication.

3. Apparatus comprising: a hand-held device comprising at least one accelerometer providing an acceleration indicating output; computation circuitry receiving said acceleration indicating output and providing a proportionally normalized velocity indicating output; and a character recognizer receiving said velocity indicating output and providing a character recognition output indication.

4. Apparatus according to claim 3 and wherein said at least one accelerometer comprises a plurality of accelerometers providing outputs to acceleration processing circuitry, which provides said acceleration indicating output.

5. Apparatus according to claim 3 and also comprising at least one of: a computer, located within said hand-held device, receiving said character recognition output indication and performing computer operations in response thereto; a communication device, located within said hand-held device, receiving said character recognition output indication and performing communication operations in response thereto; a controller, located within said hand-held device, receiving said character recognition output indication and performing control operations in response to said character recognition output indication; and a memory located within said hand-held device, receiving said character recognition output indication and performing data base operations in response to said character recognition output indication.

6. Apparatus according to claim 5 and also comprising at least one of the following elements cooperating with at least one of said computer, said communication device, said controller and said memory: a microphone; a speaker; an earphone; a digital signal processor (DSP); a display; a control circuitry; a CPU; a memory; communications circuitry; a transceiver; an antenna; a baseband controller; a baseband interface; an Input/Output port; a wireless port; a Bluetooth port; wireless communications circuitry; an infra-red port; a serial port; a USB port; a smart-card; an analog to digital converter and data acquisition circuitry.

7. Apparatus comprising: a hand-held device comprising at least one accelerometer providing an acceleration indicating output; computation circuitry receiving said acceleration indicating output and providing a proportionally normalized velocity indicating output; a character recognizer receiving said velocity indicating output and providing a character recognition output indication; and a hand motion tracker receiving said velocity indicating output and providing a tracking output indication.

8. Apparatus according to claim 7 and wherein said at least one accelerometer comprises a plurality of accelerometers providing outputs to acceleration processing circuitry, which provides said acceleration indicating output.

9. Apparatus according to claim 7 and also comprising at least one of: a computer, located within said hand-held device, receiving at least one of said recognition output indication and said tracking output indication and performing computer operations in response thereto; a communication device, located within said hand-held device, receiving at least one of said recognition output indication and said tracking output indication and performing communication operations in response thereto; a controller, located within said hand-held device, receiving said character recognition output indication and performing control operations in response to said character recognition output indication; and a memory located within said hand-held device, receiving at least one of said recognition output indication and said tracking output indication and performing data base operations in response to said character recognition output indication.

10. Apparatus according to claim 9 and also comprising at least one of the following elements cooperating with at least one of said computer, said communication device, said controller and said memory: a microphone; a speaker; an earphone; a digital signal processor (DSP); a display; a control circuitry; a CPU; a memory; communications circuitry; a transceiver; an antenna; a baseband controller; a baseband interface; an Input/Output port; a wireless port; a Bluetooth port; wireless communications circuitry; an infra-red port; a serial port; a USB port; a smart-card; an analog to digital converter and data acquisition circuitry.

Description

FIELD OF THE INVENTION

The present invention relates to apparatus and methods for hand motion detection and handwriting recognition generally.

BACKGROUND OF THE INVENTION

The following U.S. Patents are believed to represent the current state of the art: 5,159,321; 5,247,137; 5,294,792; 5,434,371; 5,453,758; 5,517,579; 5,587,558; 5,598,187; 5,781,661; 5,902,968; 5,981,884; 6,084,557; 6,130,666;

The following PCT International Publications are believed to represent the current state of the art: WO 94/09447; WO 94/18663; WO 95/21436; WO 99/22338;

SUMMARY OF THE INVENTION

The present invention seeks to provide improved apparatus and methods for hand motion detection and handwriting recognition as well as hand held computing and communication devices employing such apparatus and methods.

There is thus provided in accordance with a preferred embodiment of the present invention an apparatus for handwriting recognition including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry receiving the acceleration indicating output and providing a velocity indicating output and a handwritten multiple character recognizer receiving the velocity indicating output and providing a multiple character recognition output indication.

There is also provided in accordance with a preferred embodiment of the present invention an apparatus for handwriting recognition including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry receiving the acceleration indicating output, discarding user specific components therein and providing a non-user specific output and a handwritten character recognizer receiving the non-user specific output and providing a character recognition output indication.

There is further provided in accordance with a preferred embodiment of the present invention an apparatus for hand motion tracking including a handheld device including at least one accelerometer providing an acceleration indicating output, computation circuitry receiving the acceleration indicating output and providing a proportionally normalized velocity indicating output and a hand motion tracker receiving the velocity indicating output and providing a tracking output indication.

There is further provided in accordance with another preferred embodiment of the present invention an apparatus for hand motion tracking including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry receiving the acceleration indicating output, discarding user specific components therein and providing a non-user specific output and a hand motion tracker receiving the non-user specific output and providing a tracking output indication.

There is also provided in accordance with yet another preferred embodiment of the present invention an apparatus for handwriting recognition and hand motion tracking including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry receiving the acceleration indicating output and providing a velocity indicating output, a handwritten character recognizer receiving the velocity indicating output and providing a multiple character recognition output indication and a hand motion tracker receiving the velocity indicating output and providing a tracking output indication.

There is also provided in accordance with a preferred embodiment of the present invention an apparatus for handwriting recognition and hand motion tracking including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry receiving the acceleration indicating output, discarding user specific components therein and providing a non-user specific output, a handwritten character recognizer receiving the non-user specific output and providing a character recognition output indication and a hand motion tracker receiving the non-user specific output and providing a tracking output indication.

There is also provided in accordance with yet a further preferred embodiment of the present invention a pen computer including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry located within the hand-held device and receiving the acceleration indicating output and providing a non-user specific output, a handwritten character recognizer located within the hand-held device and receiving the non-user specific output and providing a character recognition output indication and a computer located within the hand-held device for receiving the character recognition output indication as performing computer operations in response thereto.

There is further provided in accordance with a preferred embodiment of the present invention a pen communicator including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry located within the hand-held device and receiving the acceleration indicating output and providing a proportionally normalized velocity indicating output, a handwritten character recognizer located within the hand-held device and receiving the proportionally normalized velocity indicating output and providing a character recognition output indication, and a communication device located within the hand-held device for receiving the character recognition output indication and performing communication operations in response thereto.

Further in accordance with a preferred embodiment of the present invention the accelerometer includes a plurality of accelerometers providing outputs to acceleration processing circuitry, which provides the acceleration indicating output.

Still further in accordance with a preferred embodiment of the present invention the computation circuitry is also operable to perform at least one of Calculator, Database, Personal Information Management (PIM) and Personal Digital Assistant (PDA) functionalities, including at least one of address and phone book, data-bank, calendar, time management, calculator, expenses tracking and management, "to do list" management, note and memo taking, dictionary, translator and speller, smart-card, magnetic card, credit-card, electronic-wallet, remote-control and smart key functionalities and to provide a computation output resulting therefrom.

Further in accordance with a preferred embodiment of the present invention the apparatus for handwriting recognition also includes communication circuitry.

Preferably, the communication circuitry provides functionality for at least one of a telephone, paging, SMS, WAP, Email, Internet browsing and POTS functions.

Additionally in accordance with a preferred embodiment of the present invention the apparatus for handwriting recognition also includes at least one of a microphone, a speaker, an earphone, an audio interface, an audio processor, a digital signal processor, a display, an image processor, a control circuitry, a CPU, a MPU, a memory, a transceiver, an antenna, a baseband controller, a baseband interface, an Input/Output port, a wireless port, a Bluetooth output, an infra-red port, a serial port, a USB port and data acquisition circuitry.

Further in accordance with a preferred embodiment of the present invention the apparatus is employed as a writing implement.

Still further in accordance with a preferred embodiment of the present invention the computation circuitry provides at least one of proportional normalization of velocity, identifies minimum motion points, normalizes at least a portion of the acceleration indicating output to a level of an average of a first few samples of the acceleration indicating output, and determining a last intentional motion sample of the acceleration indicating output and eliminates samples following the last intentional motion sample of the acceleration indicating output.

Additionally in accordance with a preferred embodiment of the present invention the apparatus for handwriting recognition provides a multiple character recognition output indication which includes at least one of: compressing the velocity indicating output, separately analyzing information for the x and y velocity components, analyzing the directions of the velocity, calculating velocity thresholds, digitizing velocity components, comparing and matching between a digital dictionary and a digitized velocity record and using only part of the digitized velocity record, comparing and matching between a digital dictionary and a few different representation forms of digitized velocity using a prioritization procedure which takes place in case of disagreement between different comparing and matching results, filtering out short duration segments of the velocity components, indicating velocity value changes and ignoring time durations between the changes, performing a merger operation on velocity segments and binarizing velocity segments.

Further in accordance with a preferred embodiment of the present invention the hand motion tracker provides at least one of a three-dimensional tracking output indication, a two-dimensional tracking output indication and a character tracking output indication.

There is further provided in accordance with a preferred embodiment of the present invention a method for handwriting recognition. The method includes providing an acceleration indicating output from a hand-held device, receiving the acceleration indicating output and providing a velocity indicating output, and receiving the velocity indicating output and providing a multiple character recognition output indication.

Further in accordance with a preferred embodiment of the present invention the method also includes at least one of a computation related to the character recognition output indication, a database operation related to the character recognition output indication, communication of character recognition output indication and a control operation based on the character recognition output indication.

Still further in accordance with a preferred embodiment of the present invention the velocity indicating output is further proportionally normalized.

Additionally in accordance with a preferred embodiment of the present invention the method also includes normalizing at least a portion of the acceleration indicating output to a level of the average of the first few samples of the acceleration indicating output.

Further in accordance with a preferred embodiment of the present invention the method further includes determining a last intentional motion sample of the acceleration indicating output and eliminating samples following the last intentional motion sample of the acceleration indicating output.

Preferably determining of the last intentional motion sample includes examining a record of the output of at least one of the acceleration sensors in order to determine the average of the last few samples, thereafter, examining the record sequentially from the last few samples backwards in time direction until a variation from the average of at least a predetermined magnitude is encountered and determining a time location of this variation to be the time location of the last intentional motion.

Further in accordance with a preferred embodiment of the present invention the method of providing a velocity indicating output further includes identification of minimum motion points.

Still further in accordance with a preferred embodiment of the present invention the method also includes providing a multiple character recognition output indication includes compressing the velocity indicating output.

Additionally in accordance with a preferred embodiment of the present invention the method further includes providing a multiple character recognition output indication and includes separately analyzing information for the X and Y velocity components, Further in accordance with a preferred embodiment of the present invention the method also includes providing a multiple character recognition output indication includes analyzing directions of the velocity.

Still further in accordance with a preferred embodiment of the present invention the method includes providing a multiple character recognition output indication and includes comparing and matching between a digital dictionary and digitized velocity record using only part of the digitized velocity.

Alternatively the providing a multiple character recognition output indication includes comparing and matching between a digital dictionary and a few different representation forms of digitized velocity using a prioritization procedure which takes place in case of disagreement between different comparing and matching results.

Further in accordance with a preferred embodiment of the present invention the method further includes at least one of filtering out short duration segments of the velocity components, performing a merger operation on velocity segments and binarizing operation on velocity segments.

Still further in accordance with a preferred embodiment of the present invention the method also includes providing a multiple character recognition output indication includes calculating velocity thresholds.

Preferably calculating of velocity thresholds further includes dependency of the thresholds on the minimum and maximum values of the velocity components.

Further in accordance with a preferred embodiment of the present invention the method includes digitizing velocity components to provide digitized information.

Preferably the method also includes analyzing the digitized information for the velocity components to detect value changes therein, indicating each such change and ignoring time durations between changes.

There is also provided in accordance with another preferred embodiment of the present invention a method for handwriting recognition. The method includes providing an acceleration indicating output from a hand-held device, receiving the acceleration indicating output, discarding user specific components therein and providing a non-user specific output and receiving the non-user specific output and providing a character recognition output indication.

There is also provided in accordance with a preferred embodiment of the present a method for hand motion tracking. The method includes providing an acceleration indicating output from a hand-held device, receiving the acceleration indicating output and providing a proportionally normalized velocity indicating output and receiving the velocity indicating output and providing a tracking output indication.

Further in accordance with a preferred embodiment of the present invention the method also includes providing a tracking indicating output includes at least one of representing a multi-dimensional object, representing drawing in the air, representing drawing in on a surface, representing a character tracking in the air, and representing a character tracking on a surface.

Still further in accordance with a preferred embodiment of the present invention the method further includes indicating at least one of a computation based on the tracking output, operating a database on the tracking output indication, indicating communication of tracking output and a control operations based on the tracking output indications.

Additionally in accordance with a preferred embodiment of the present invention the method also includes normalizing at least a portion of the acceleration indicating output to a level of the average of the first few samples of the acceleration indicating output.

Further in accordance with a preferred embodiment of the present invention the method further includes determining the last intentional motion sample of the acceleration indicating output and eliminating samples following the last intentional motion sample of the acceleration indicating output,

Still further in accordance with a preferred embodiment of the present invention, determining of the last intentional motion sample includes examining a record of the output of at least one of the acceleration sensors in order to determine the average of the last few samples, thereafter, examining the record sequentially from the last few samples backwards in time direction until a variation from the average of at least a predetermined magnitude is encountered and determining a time location of this variation to be the time location of the last intentional motion.

Further in accordance with a preferred embodiment of the present invention the method also provided a velocity indicating output further includes identification of minimum motion points.

There is further provided in accordance with a preferred embodiment of the present invention a method for hand motion tracking. The method includes providing an acceleration indicating output from a hand-held device, receiving the acceleration indicating output, discarding user specific components therein and providing a non-user specific output and receiving the non-user specific output and providing a tracking output indication.

There is further provided in accordance with a preferred embodiment of the present invention a method for handwriting recognition and hand motion tracking. The method includes providing an acceleration indicating output from a hand-held device, receiving the acceleration indicating output and providing a velocity indicating output, receiving the velocity indicating output and providing a multiple character recognition output indication and receiving the non-user specific output and providing a character recognition output indication.

There is further provided in accordance with a preferred embodiment of the present invention a method for handwriting recognition and hand motion tracking. The method includes providing an acceleration indicating output from a hand-held device, receiving the acceleration indicating output, discarding user specific components therein and providing a non-user specific output and receiving the non-user specific output and providing a character recognition output indication.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood and appreciated more fully from the following detailed description, taken in conjunction with the drawings in which:

FIG. 1 is a simplified partially pictorial, partially block diagram illustration of handwriting recognition apparatus constructed and operative in accordance with a preferred embodiment of the present invention;

FIG. 2 is a simplified partially pictorial, partially block diagram illustration of hand motion tracking apparatus constructed and operative in accordance with a preferred embodiment of the present invention;

FIGS. 3A & 3B are together a simplified partially pictorial, partially block diagram illustration of handwriting recognition and hand motion tracking apparatus constructed and operative in accordance with a preferred embodiment of the present invention;

FIG. 4 is a simplified partially pictorial, partially block diagram illustration of a telephone employing handwriting recognition and hand motion tracking apparatus constructed and operative in accordance with a preferred embodiment of the present invention;

FIG. 5 is a simplified functional block diagram illustration of functionalities of the apparatus of FIGS. 1, 2, 3A, 3B & 4;

FIGS. 6A, 6B, 6C, 6D, 6E, 6F, 6G, 6H & 6I are flow charts, illustrating some of the functionalities of FIG. 5;

FIGS. 7A, 7B, 7C, 7D, 7E & 7F are acceleration diagrams which are useful in the understanding of the functionalities shown in FIGS. 6B & 6C;

FIGS. 8A & 8B are each a velocity diagram which is useful in the understanding of the functionality shown in FIG. 6D;

FIGS. 9A & 9B are each a diagram which is useful in the understanding of the functionality shown in FIG. 6E; and

FIGS. 10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H & 10I are diagrams which are useful in the understanding of the functionality shown in FIG. 6F.

FIGS. 11A, 11B, 11C, 11D, 11E, 11F, 11G, 11H, 11I 11J and 11K, are diagrams which are useful in the understanding of the functionality shown in FIGS. 6F & 6G; and

FIGS. 12A, 12B, 12C & 12D are diagrams which are useful in the understanding of the functionality shown in FIGS. 6F, 6G, 6H & 6I;

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Reference is now made to FIG. 1, which is a simplified partially pictorial, partially block diagram illustration of handwriting recognition apparatus constructed and operative in accordance with a preferred embodiment of the present invention. As seen in FIG. 1, there is provided apparatus for handwriting recognition, generally designated by reference numeral 100 and including a hand-held housing 102, typically formed of molded plastic and including therewithin at least one and preferably plural mutually orthogonal arranged accelerometers 104 providing outputs to an acceleration processing circuitry 106 which, in turn, preferably outputs to velocity processing circuitry 108, which provides a velocity indicating output.

A recognition circuitry 110, such as a handwriting character recognizer, receives the velocity indicating output from circuitry 108 and preferably provides a character recognition output indication. It is a particular feature of the present invention that a velocity indicating output is provided and employed for downstream recognition. The character recognition output indication is preferably supplied to a display 112, such as a LCD display, which is viewable by a user.

Additionally in accordance with a preferred embodiment of the present invention, computation circuitry 114 is provided and receives the character recognition output indication from circuitry 110. Computation circuitry 114 is preferably operable to perform mathematical functions, such as arithmetic functions, as illustrated based on inputs received from circuitry 110. Computation circuitry 114 preferably is operable to provide a computation output via display 112. Computation circuitry 114 may also be enabled to perform any other suitable type of computation, including for example, computations useful in Personal Information Management functionality.

Reference is now made to FIG. 2, which is a simplified partially pictorial, partially block diagram illustration of hand motion tracking apparatus constructed and operative in accordance with a preferred embodiment of the present invention. As seen in FIG. 2, there is provided apparatus for hand motion tracking, generally designated by reference numeral 200 and including a hand-held housing 202, typically formed of molded plastic and including therewithin at least one and preferably plural mutually orthogonal arranged accelerometers 204 providing outputs to acceleration processing circuitry 206 which, in turn, preferably outputs to velocity processing circuitry 208, which provides a velocity indicating output.

Tracking circuitry 210 receives the velocity indicating output from circuitry 208 and preferably provides a three-dimensional tracking output indication. Preferably, the three-dimensional output may be employed by a suitable output device.

Alternatively, the output of tracking circuitry 210 may be supplied to 3D-2D tracking output indication conversion circuitry or, as a further alternative, tracking circuitry 210 may provide a two-dimensional output. Preferably, the three-dimensional output may be employed by a suitable output device.

It is a particular feature of the present invention that a velocity indicating output is provided and employed for downstream tracking. The output of tracking circuitry 210 is supplied either directly or via 3D-2D tracking output indication conversion circuitry (not shown) to communication circuitry 214 which preferably provides a wireless communication output to a remote output device, such as, for example, a personal digital assistant 216.

It is appreciated that communication circuitry 214 may be any suitable communication circuitry and may provide wired or wireless transmission via a suitable antenna 218. A preferred type of communication is wireless Bluetooth communication, it being understood, however, that any suitable alternative thereto is also contemplated hereby, including, inter alia relatively long range communication.

As seen in FIG. 2, a three dimensional object can typically be represented simply by tracing one or more features of the outline thereof using the apparatus 200.

Reference is now made to FIGS. 3A & 3B, which are together a simplified partially pictorial, partially block diagram illustration of handwriting recognition and hand motion tracking apparatus constructed and operative in accordance with a preferred embodiment of the present invention. As seen in FIGS. 3A & 3B, there is provided apparatus for hand motion tracking and handwriting recognition, generally designated by reference numeral 300 and including a hand-held housing 302, typically formed of molded plastic and including therewithin at least one and preferably plural mutually orthogonal arranged accelerometers 304 providing outputs to acceleration processing circuitry 306 which, in turn, preferably outputs to velocity processing circuitry 308, which provides a velocity indicating output.

Tracking circuitry 310 receives the velocity indicating output from circuitry 308 and preferably provides a three-dimensional tracking output indication. Alternatively, tracking circuitry 310 may provide a two-dimensional output. As a further alternative, the three-dimensional output may be employed by any suitable output device. The output from the tracking circuitry 310 may be displayed or transmitted to another communication device and also used for handwriting recognition, as descried hereinbelow.

It is a particular feature of the present invention that a velocity indicating output is provided and employed for downstream tracking.

In addition to tracking circuitry 310, there is also preferably provided recognition circuitry 314, such as a handwriting character recognizer, receives the velocity indicating output from circuitry 308 and preferably provides a character recognition output indication. It is a particular feature of the present invention that a velocity indicating output is provided and employed for downstream recognition. The character recognition output indication is preferably supplied to a display 316, such as an LCD display, which is viewable by a user.

Additionally in accordance with a preferred embodiment of the present invention, computation circuitry 318 is provided and receives the character recognition output indication from circuitry 314 via a bus 315. Computation circuitry 318 is preferably operable to perform mathematical functions, such as arithmetic functions and personal organizer functions as illustrated based on inputs received from circuitry 314. Computation circuitry 318 preferably is operable to provide a computation output via the bus 315 to the display 316 and to the communication circuitry 320. Computation circuitry 318 may also be enabled to perform any other suitable type of computation and to have database and Personal Digital Assistant functionality.

In accordance with a preferred embodiment of the present invention, the output of tracking circuitry 310 is supplied either directly or via 3D-2D tracking output indication conversion circuitry (not shown) to communication circuitry 320, along with the output of one or both of computation circuitry 318 and recognition circuitry 314. Communication circuitry 320 preferably provides a wireless communication output via an antenna 322 or other suitable communication port to a remote output device, such as, for example, a similar device 350 or any other suitable output device. It is appreciated that communication circuitry 320 may be any suitable communication circuitry and may provide wired or wireless transmission. A preferred type of communication is at least partially wireless telephone or IP communication, it being understood, however, that any suitable alternative thereto is also contemplated hereby.

As seen in FIG. 3A, a user can represent three dimensional objects such as flowers simply by tracing one or more features of the outline thereof or drawing them in the air or on a substrate using the apparatus 300. The representation of the three dimensional image, typically in two-dimensional form, may be sent to a similar output device 350 at a remote location, as illustrated, where it may be viewed on the display 316 of that device.

As seen in FIG. 3B, the user of the remote apparatus 350 may, in response to receipt of the representation of the three dimensional image, transmit a response back to the sender. In the illustrated case, the response is a handwritten message, transmitted wirelessly to the original sending user and appearing on display 316 of his apparatus 300.

Reference is now made to FIG. 4, which is a simplified partially pictorial, partially block diagram illustration of a hand-held communicator, such as a telephone employing handwriting recognition and hand motion tracking apparatus constructed and operative in accordance with a preferred embodiment of the present invention. As seen in FIG. 4, there is provided a hand-held communicator, such as a telephone, generally designated by reference numeral 400 and preferably including apparatus for hand motion tracking and handwriting recognition located within a hand-held housing 402. Housing 402 is preferably formed of molded plastic and includes therewithin at least one and preferably plural mutually orthogonal arranged accelerometers 404 providing outputs to acceleration processing circuitry 406 which, in turn, preferably outputs to velocity processing circuitry 408, which provides a velocity indicating output.

Recognition circuitry 410, such as a handwriting character recognizer, preferably receives the velocity indicating output from circuitry 408 and preferably provides a character recognition output indication. It is a particular feature of the present invention that a velocity indicating output is provided to circuitry 410 and employed for downstream recognition therein. The character recognition output indication from circuitry 410 is preferably supplied via a bus 412 to computation and control circuitry 414, a baseband controller 418 and to a display 416, such as an LCD display, which is viewable by a user.

Computation and control circuitry 414 is preferably operable to perform mathematical functions, such as arithmetic functions and personal organizer functions, as illustrated based on inputs received from circuitry 410. Computation and control circuitry 414 preferably is operable to provide a computation output via display 416 and the bus 412. Computation and control circuitry 414 may also be enabled to perform any other suitable type of computation, database or Personal Digital Assistant functionality.

Conventional cellular telephone functionality may be provided by computation and control circuitry 414, baseband controller 418 and a baseband and audio interface 420. Interface 420 typically interfaces with a microphone 422 and a speaker 424 as well as with a conventional RF transceiver 426. The RF transceiver 426 transmits and receives via a suitable antenna 428. It is noted that the output of recognition circuitry 410 may be employed for dialing, as pictorially illustrated in FIG. 4, typically by supplying an output from recognition circuitry 410 via baseband controller 418 to the RF transceiver 426 that transmits via the antenna 428.

It is a particular feature of the embodiment of FIG. 4 that the output of recognition circuitry 410 and possibly of computation and control circuitry 414 may be used as a dialing input for a telephone. It is thus appreciated, that, as illustrated in FIG. 4, a user may dial the telephone by writing in the air or on any substrate using the telephone as a writing implement.

It is appreciated that the communicator 400 may be any suitable type of communicator and may provide wired or wireless transmission. A preferred type of communication is at least partially wireless telephone or IP telephony communication, it being understood, however, that any suitable alternative thereto is also contemplated hereby.

Reference is now made to FIG. 5, which is a simplified functional block diagram illustration of a preferred embodiment of the present invention showing the functionalities of the apparatus of FIGS. 1, 2, 3A, 3B & 4. It is appreciated that in any given application some of the functionalities described hereinbelow, with respect to FIG. 5, may not be needed and thus may not be present.

As seen in FIG. 5, preferably three accelerometers, operating as respective X, Y and Z acceleration sensors, respectively designated by reference numerals 500, 502 and 504, preferably output to data acquisition circuitry 506. Data acquisition circuitry 506 preferably includes operational amplifiers 508, 510 and 512, receiving the outputs of acceleration sensors 500, 502 and 504 respectively and providing outputs via respective low pass filters 514, 516 and 518 to an A-D converter 520. It is appreciated that digital acceleration sensors, providing a DCM (Duty Cycle Modulated signal) output may be employed instead of sensors providing analog outputs. In such a case, the A-D converter 520 is obviated and other aspects of the data acquisition circuitry 506 may be changed accordingly.

The output of the A-D converter 520 is preferably supplied to a dual port motion memory 522 which communicates via a control bus and a data bus, here respectively designated by reference numerals 524 and 526.

A controller 530, which typically receives inputs from a write/on/off switch 532 and a contact sensor 534, provides control inputs via the control bus 524 to the data acquisition circuitry 506, to the dual port motion memory 522 and to acceleration processing circuitry 536, which receives a data output from dual port motion memory 522 via the data bus 526. Controller 530 may also provide a power management function for the remainder of the circuitry of FIG. 5, which may include power savings when a sensed activity level is below a useful level due to the absence of intentional hand motion.

The acceleration processing circuitry 536 provides an output to velocity processing circuitry 538 preferably via data bus 526 and the dual port motion memory 522. Velocity processing circuitry 538 also receives a control input preferably from controller 530 via control bus 524.

The output of velocity processing circuitry 536 is preferably supplied via data bus 526 to tracking circuitry 540 and to recognition circuitry 542, both of which also receive control inputs from controller 530 preferably via control bus 524. The tracking circuitry 540 preferably provides an output via an output bus 544 to communication and Input/output circuitry 546, application and output memory 548 and display 550.

Recognition circuitry 542 preferably comprises velocity compression circuitry 552 which receives data and control inputs from respective data and control busses 526 and 524 and which provides an output to identification circuitry 554. Identification circuitry 554 typically also receives a data input from a dictionary database 556 and a control input from control bus 524. Identification circuitry 554 also preferably outputs via output bus 544 to communication and Input/Output circuitry 546, to application and output memory 548 and to display 550.

Controller 530 preferably cooperates with application processing circuitry 558, such as circuitry which provides Personal Information Management (PIM) and Personal Digital Assistant (PDA) circuitry including at least one of address and phone book, data-bank, calendar, time management, calculator, expenses tracking and management, "to do list" management, note and memo taking, dictionary, translator and speller functionalities. It may also provide one or more of smart-card, magnetic card, credit-card, electronic-wallet, remote-control and smart key functionalities. When combined with a communication functionality, it may serve for editing, browsing, receiving and sending, uni- and bi-directional paging, SMS, WAP, Email, POTS and other types of communications. Circuitry 558 preferably which interfaces with application and output memory 548, which in turn communicates via output bus 544 and control bus 524, as does controller 530 and a display 550, which is coupled to controller 530.

Communication and input/output circuitry 546 may handle wireless port, infra-red port, serial port, USB port and any other suitable type of communication. It preferably interfaces with the controller 530 via the control bus 524 and output bus 544, exchanges data with application and output memory 548 and sends data to display 550, via output bus 544. A wireless embodiment of the communication circuitry 546 appears in FIG. 4.

It is appreciated that there may be other architectures, memory organizations, processing schemes and buses structures which also implement the functionalities shown in FIG. 5.

The operation of the apparatus of FIG. 5 is now be described in greater detail with reference to FIGS. 6A, 6B, 6C, 6D, 6E, 6F, 6G, 6H &, 6I, which are flow charts, illustrating some of the functionality of FIG. 5 and to FIGS. 7A, 7B, 7C, 7D, 7E & 7F, which are acceleration diagrams which are useful in the understanding of the functionality shown in FIGS. 6B & 6C, FIGS. 8A & 8B, which are each a velocity diagram which is useful in the understanding of the functionality shown in FIG. 6D, FIGS. 9A & 9B, which are each a diagram which is useful in the understanding of the functionality shown in FIG. 6E and FIGS. 10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H & 10I are diagram which is useful in the understanding of the functionality shown in FIG. 6E.

Reference is made in the course of the description which follows also to FIGS. 11A, 11B, 11C, 11D, 11E, 11F, 11G, 11H, 11I, 11J & 11K, which are diagrams which are useful in the understanding of the functionality shown in FIGS. 6F, 6G, 6H & 6I and to FIGS. 12A, 12B, 12C & 12D are diagrams which are useful in the understanding of the functionality shown in FIGS. 6F, 6G, 6H & 6I.

Turning initially to FIG. 6A, which relates to the operation of data acquisition circuitry 506, dual port motion memory 522 and controller 530, it is seen that initially one of the two buffers in dual port motion memory 522 is selected by controller 530. Initial levels of the outputs of acceleration sensors 500, 502 and 504 are stored in the selected active buffer.

A determination is then made by controller 530 of whether a motion plane in three-dimensional space is detected or whether a write indication is otherwise provided. This detection may be made in one of typically three ways. By operation of contact sensor 534, by analysis of the outputs of acceleration sensors 500, 502 and 504 and by operation of the write/on/off switch 532. Upon detection of a motion plane, a new record is started and outputs of acceleration sensors 500, 502 and 504 are stored. Additional samples are stored until the outputs of the acceleration sensors 500, 502 and 504, or the contact sensor 534 indicate the absence of a motion plane, or a write signal is absent or until memory capacity is exceeded.

Upon termination of storing, the record is ended and processed, typically by acceleration processing circuitry 536. A typical record contains a large number of samples of the outputs of acceleration sensors 500, 502 and 504. Further records are processed in much the same manner, it being appreciated that by the use of two active buffers, the contents of one may be processed, while the other is being filled.

Referring now to FIG. 6B which relates to the operation of acceleration processing circuitry 536, it is seen that preferably, the first few and last few samples in each record are eliminated. This may be readily seen by viewing FIGS. 7A and 7B, which illustrate a record containing the outputs of the X and Y acceleration sensors 500 and 502 respectively, prior to and subsequent to elimination of the first few and last few samples therein. Following elimination of the first few and last few samples in the record, the remaining outputs of the acceleration sensors in the record are preferably normalized to a calibration level, as can be seen in FIG. 7C for the outputs of the X and Y acceleration sensors 500 and 502 respectively. Clearly, if three accelerometers are employed a Z acceleration sensor output is processed in a similar manner.

Following the normalization, an average of the first few remaining samples, typically about 10-30 samples, is taken. The remaining record is then normalized to a level of the average of the first few remaining samples. The result of this normalization for the outputs of the X and Y acceleration sensors 500 and 502 respectively is shown in FIG. 7D. The normalization of the acceleration record, according to the average of the first samples, as exemplified by the difference between the FIGS. 7C and 7D is a particular feature of the present invention.

Following this normalization according to the average of the first samples, the last intentional motion sample is detected, as is described hereinbelow with reference to FIG. 6C. The samples following the detected last intentional motion sample are eliminated, as illustrated for the outputs of the X and Y acceleration sensors 500 and 502 respectively in FIG. 7E. Noise is also filtered out, as illustrated for the outputs of the X and Y acceleration sensors 500 and 502 respectively in FIG. 7F and the resulting processed acceleration record is supplied for velocity processing by circuitry 538 (FIG. 5).

Reference is now made to FIG. 6C, which illustrates detection of the last intentional motion sample. A record of the output of at least one of the three acceleration sensors 500, 502 and 504, such as the record shown in FIG. 7D in respect of the output of the X acceleration sensor is examined to determine the average of the last few remaining samples.

Following determination of this average, the preceding samples is examined sequentially from the last few samples in a backwards in time direction until a variation from the average of at least a predetermined magnitude is encountered. The time location of this variation is determined to be the time location of the last intentional motion.

The foregoing process may be carried out in parallel for the outputs of all or some of the three acceleration sensors 500, 502 and 504. The existence of a variation from the average of at least a predetermined magnitude for even the output of just one of the acceleration sensors 500, 502 and 504 may be considered sufficient for finding the last intentional motion in the record for the outputs of all of the acceleration sensors 500, 502 and 504.

Reference is now made to FIG. 6D, which illustrates velocity processing in circuitry 538 in accordance with a preferred embodiment of the invention. The output of the acceleration processing circuitry 536, illustrated for example in FIG. 7F, is integrated over time to produce a velocity record. The velocity record corresponding to the acceleration input of FIG. 7F, appears in FIG. 8A.

The average velocity at the end of the velocity record, corresponding to the last few remaining acceleration samples, typically about 5-10 samples, is calculated.

The velocity of the entire velocity record, e.g. of all values represented in FIG. 8A is now proportionally normalized between the beginning and end of the record, where the proportion of normalization applied at each sample within a velocity record is relative to the distance of that sample from the beginning and end of the record. FIG. 8B illustrates the proportionally normalized values VX1 and VY1 of FIG. 8A, for the X and Y components respectively. A suitable formula for obtaining the values VX1 and VY1 of FIG. 8B from the values VX and VY, respectively, of FIG. 8A is: VX1(n)=VX(n)-n/N.times.(X component of average velocity at N) VY1(n)=VY(n)-n/N.times.(Y component of average velocity at N),

where n is the index of samples within the record, n=1, . . . N.

The X and Y components of the average velocity at N (as computed in the second step of the method of FIG. 6D) are the average X and Y velocity values, respectively, where the averages are each taken over the last few (typically 5-10) velocity values in the velocity record of FIG. 8A, corresponding to the last few acceleration samples. In case of three dimensional motion a similar formula is applied also for the Z component of the velocity. The proportional normalization of the velocity record, as exemplified by the difference between FIGS. 8A and 8B is a particular feature of the present invention.

It is also a particular feature of a preferred embodiment of the present invention that, for a long track or complex track, the identification of minimum motion points along the track may be performed similarly to the process described hereinabove with reference to FIG. 6C and proportional