System and method for volume control management in a personal telephony recorder Number:7,065,198 from the United States Patent and Trademark Office (PTO) owispatent

Title: System and method for volume control management in a personal telephony recorder

Abstract: A system and method for recording a telephone conference and replaying a portion of the recording during the conference. Users participate by connecting through different types of networks using a device having a communication line connection. The recording can be in audio format, text format, or both. Thus, users can recall and replay textual information in addition to the recorded audio. Other information-such as time and user data-may also be recorded along with the audio and text. Users in the conference are identified to enable the association with them each user's contribution to the conference. The user or the user's device can assist by providing identification information. User identification may also be accomplished by associating each user's contribution with the particular line the user is calling from. Caller ID information may also be used to identify the user. Voice analysis may also performed to accomplish user identification.

Patent Number: 7,065,198 Issued on 06/20/2006 to Brown,   et al.

---

Inventors:	Brown; Michael Wayne (Georgetown, TX); McIntyre; Joseph Herbert (Austin, TX); Moore; Victor S. (Boynton Beach, FL); Paolini; Michael A. (Austin, TX); Winters; Scott Lee (Austin, TX)
Assignee:	International Business Machines Corporation (Armonk, NY)
Appl. No.:	279674
Filed:	October 23, 2002

---

Current U.S. Class:	379/202.01 ; 379/85; 379/87; 379/88.02
Current International Class:	H04M 3/42 (20060101)

---

References Cited [Referenced By]

---

U.S. Patent Documents

	4481383		November 1984		Madon
	4577067		March 1986		Levy et al.
	4937856		June 1990		Natarajan
	5313523		May 1994		Kawauchi
	5353351		October 1994		Bartoli et al.
	5440624		August 1995		Schoof, II
	5471318		November 1995		Ahuja et al.
	5539741		July 1996		Barraclough et al.
	5548335		August 1996		Mitsuhashi et al.
	5600714		February 1997		Eppler, Jr. et al.
	5604798		February 1997		Sacchetti et al.
	5627936		May 1997		Prasad et al.
	5657246		August 1997		Hogan et al.
	5668863		September 1997		Bieselin et al.
	5692213		November 1997		Goldberg et al.
	5710591		January 1998		Bruno et al.
	5778053		July 1998		Skarbo et al.
	5867574		February 1999		Eryilmaz
	5916302		June 1999		Dunn et al.
	5940476		August 1999		Morganstein et al.
	5987106		November 1999		Kitamura
	6163692		December 2000		Chakrabarti et al.
	6212522		April 2001		Himmel et al.
	6222909		April 2001		Qua et al.
	6298129		October 2001		Culver et al.
	6343313		January 2002		Salesky et al.
	6363145		March 2002		Shaffer et al.
	6389114		May 2002		Dowens et al.
	6407988		June 2002		Agraharam et al.
	6477366		November 2002		Valentine et al.
	6529875		March 2003		Nakajima et al.
	6563914		May 2003		Sammon et al.
	6661879		December 2003		Schwartz et al.
	6674842		January 2004		Weinman, Jr.
	6795536		September 2004		Ronca
	6832350		December 2004		Bates et al.
	6850609		February 2005		Schrage
	6859448		February 2005		Roy
	2001/0033644		October 2001		Offer
	2002/0089901		July 2002		Morohoshi et al.
	2003/0114147		June 2003		Goss et al.
	2003/0227540		December 2003		Monroe

Foreign Patent Documents

	61274462		Dec., 1986		JP
	1143511		Jun., 1989		JP
	4196898		Jul., 1992		JP
	5241588		Sep., 1993		JP
	4287461		Oct., 1993		JP
	8251276		Sep., 1996		JP
	8316953		Nov., 1996		JP

Other References

US. Appl. No. 09/240,962, filed on Jan. 29, 1999, Rodriguez et al. cited by other.

Primary Examiner: Tsang; Fan
Assistant Examiner: Sing; Simon
Attorney, Agent or Firm: Van Leeuwen & Van Leeuwen Walker; Mark S.

---

Claims

---

What is claimed is:

1. A method of managing telephone call participants, said method comprising: initiating a telephone call between a plurality of participants; receiving, during the telephone call, a plurality of voice inputs from the participants; identifying, during the telephone call, the participant corresponding to each of the received voice inputs; recording the voice inputs and the identifiers corresponding to the participants into a memory area, wherein the identifiers indicate the voice inputs received from each of the individual participants; adjusting, during the telephone call, a volume level of a first voice input received from a first participant, wherein the adjusting is based on the identity of the first participant; and transmitting, during the telephone call, the adjusted voice input to a second participant.

2. The method as described in claim 1 wherein the identifying further comprises: determining a communication line to which a selected participant is connected; and associating the communication line to the selected participant.

3. The method as described in claim 1 wherein the identifying further comprises: capturing one or more voice signatures corresponding to the voice inputs received from one or more of the participants; associating each captured voice signature with a different one of the participants; subsequently receiving voice inputs from the participants; comparing the subsequently received voice inputs with the captured voice signatures; and selecting at least one of the participants based upon the comparison.

4. The method as described in claim 1 further comprising: creating a transcript report after completion of the telephone call, the creating including: retrieving, from the memory area, the participant identifiers and the voice data received from the identified participants in the same order in which the voice data was recorded; converting the retrieved voice data to a textual form; and displaying the participant corresponding to each participant identifier and the corresponding converted textual form of the voice data.

5. The method as described in claim 1 further comprising: receiving one or more fields of participant data from one or more of the participants, wherein at least one of the fields of participant data is selected from the group consisting of a participant's name, a participant's identifier, a participant's role, a participant's organization, a participant's contact information, a participant's email address, a participant's phone number, a participant's job title, a participant's address, and a participant's nickname.

6. The method as described in claim 1 further comprising: receiving, during the telephone call, an incoming volume adjustment request from the second participant, the incoming volume adjustment request including an incoming volume adjustment value, and wherein the adjusting is further based on the incoming volume adjustment value.

7. The method as described in claim 1 further comprising: receiving, during the telephone call, an outgoing volume adjustment request from the first participant, the outgoing volume adjustment request including an outgoing volume adjustment value, and wherein the adjusting is further based on the outgoing volume adjustment value.

8. An information handling system comprising: one or more processors; a memory area accessible by the processors for storing telephone call data; a transmitter for transmitting voice input received over a data network; a receiver for receiving voice data from the data network; and a call participant tool for managing telephone call participants, the call participant tool including: means for initiating a telephone call between a plurality of participants through the data network; means for receiving, during the telephone call, a plurality of voice inputs from the participants over the data network; means for identifying, during the telephone call, the participant corresponding to each of the received voice inputs; means for recording the voice inputs and the identifiers corresponding to the participants into the memory area, wherein the identifiers indicate the voice inputs received from each of the individual participants; means for adjusting, during the telephone call, a volume level of a first voice input received from a first participant, wherein the adjusting is based on the identity of the first participant; and means for transmitting, during the telephone call, the adjusted voice input to a second participant.

9. The information handling system as described in claim 8 wherein the means for identifying further comprises: means for determining a communication line to which a selected participant is connected; and means for associating the communication line to the selected participant.

10. The information handling system as described in claim 8 wherein the means for identifying further comprises: means for capturing one or more voice signatures corresponding to the voice inputs received from one or more of the participants; means for associating each captured voice signature with one a different one of the participants; means for subsequently receiving voice inputs from the participants; means for comparing the subsequently received voice inputs with the captured voice signatures; and means for selecting at least one of the participants based upon the comparison.

11. The information handling system as described in claim 8 further comprising: means for creating a transcript report after completion of the telephone call, the creating including: p2 means for retrieving, from the memory area, the participant identifiers and the voice data received from the identified participants in the same order in which the voice data was recorded; means for converting the retrieved voice data to a textual form; and means for displaying the participant corresponding to each participant identifier and the corresponding converted textual form of the voice data.

12. The information handling system as described in claim 8 further comprising: means for receiving one or more fields of participant data from one or more of the participants, wherein at least one of the fields of participant data is selected from the group consisting of a participant's name, a participant's identifier, a participant's role, a participant's organization, a participant's contact information, a participant's email address, a participant's phone number, a participant's job title, a participant's address, and a participant's nickname.

13. The information handling system as described in claim 8 further comprising: means for receiving, during the telephone call, an incoming volume adjustment request from the second participant, the incoming volume adjustment request including an incoming volume adjustment value, and wherein the adjusting is further based on the incoming volume adjustment value.

14. The information handling system as described in claim 8 further comprising: means for receiving, during the telephone call, an outgoing volume adjustment request from the first participant, the outgoing volume adjustment request including an outgoing volume adjustment value, and wherein the adjusting is further based on the outgoing volume adjustment value.

15. A computer program product stored on a computer operable media for managing telephone call participants, said computer program product comprising: means for initiating a telephone call between a plurality of participants; means for receiving, during the telephone call, a plurality of voice inputs from the participants; means for identifying, during the telephone call, the participant corresponding to each of the received voice inputs; means for recording the voice inputs and the identifiers corresponding to the participants into a memory area, wherein the identifiers indicate the voice inputs received from each of the individual participants; means for adjusting, during the telephone call, a volume level of a first voice input received from a first participant, wherein the adjusting is based on the identity of the first participant; and means for transmitting, during the telephone call, the adjusted voice input to a second participant.

16. The computer program product as described in claim 15 wherein the means for identifying further comprises: means for determining a communication line to which a selected participant is connected; and means for associating the communication line to the selected participant.

17. The computer program product as described in claim 15 wherein the means for identifying further comprises: means for capturing one or more voice signatures corresponding to the voice inputs received from one or more of the participants; means for associating each captured voice signature with a different one of the participants; means for subsequently receiving voice inputs from the participants; means for comparing the subsequently received voice inputs with the captured voice signatures; and means for selecting at least one of the participants based upon the comparison.

18. The computer program product as described in claim 15 further comprising: means for creating a transcript report after completion of the telephone call, the creating including: means for retrieving, from the memory area, the participant identifiers and the voice data received from the identified participants in the same order in which the voice data was recorded; means for converting the retrieved voice data to a textual form; and means for displaying the participant corresponding to each participant identifier and the corresponding converted textual form of the voice data.

19. The computer program product as described in claim 15 further comprising: means for receiving one or more fields of participant data from one or more of the participants, wherein at least one of the fields of participant data is selected from the group consisting of a participant's name, a participant's identifier, a participant's role, a participant's organization, a participant's contact information, a participant's email address, a participant's phone number, a participant's job title, a participant's address, and a participant's nickname.

20. The computer program product as described in claim 15 further comprising: means for receiving, during the telephone call, an incoming volume adjustment request from the second participant, the incoming volume adjustment request including an incoming volume adjustment value, and wherein the adjusting is further based on the incoming volume adjustment value.

21. The computer program product as described in claim 15 further comprising: means for receiving, during the telephone call, an outgoing volume adjustment request from the first participant, the outgoing volume adjustment request including an outgoing volume adjustment value, and wherein the adjusting is further based on the outgoing volume adjustment value.

---

Description

---

BACKGROUND OF THE INVENTION

1. Technical Field

The present invention relates in general to a system and method for providing personal telephony recorder services. More particularly, the present invention relates to a system and method for identifying participants of a conference call and controlling volume to and from participants.

2. Description of the Related Art

Voice communication is the most common and one of the oldest forms of real-time remote communications. Real-time remote forms of communications are very good alternatives to face-to-face meetings, in which real-time communication is an important aspect. Voice communications are used for casual conversations, to conduct business, to summon for help in an emergency, to access special services (such as banking, retrieving messages), etc.

There are numerous types of devices operating over numerous types of networks to facilitate voice communications. Most of the voice-capable networks are also capable of transmitting data. The most common voice communication device is the traditional telephone operating over the Public Switched Telephone System (PSTN), also known as the Plain Old Telephone System (POTS). Through the PSTN, telephones are linked using complex switching systems at central offices or exchanges that establish a pathway for voice to be transmitted and received between one or more of the telephones. With appropriate devices, such as modems, for examples, the PSTN can be used for the transmission of data. The PSTN is still one of the most reliable networks for voice communication.

Voice communication can also be facilitated over the Internet or other such networks. Computers connected to the Internet first convert the voice into digital information and then convert the information into data packets. The packets are created according to the Transmission Control Protocol (TCP), a set of rules used with the Internet Protocol (IP) to send data in the form of packets between computers over the Internet. IP handles the actual delivery of the data while TCP keeps track of the individual data packets, into which the voice or other data is divided, for efficient routing through the Internet. The process of transmitting voice over the Internet or other such networks is called voice-over-IP. Voice communication through the Internet is not as reliable as it is through the PSTN. Internet-type networks were designed for data transmission where "real-time" transmissions are not necessary. The speed with which the packets move from one user to the other is very dependent on the type of connection each user establishes to the Internet, the type of computers/communication lines that exist between the two users, the amount of traffic through the internet, etc.

Mobile phones and the wireless mobile network provide yet another method for voice communication. Through short-wave analog or digital transmissions, a user establishes a wireless connection from a mobile telephone to a nearby transmitter. Generally, mobile telephone service is available in urban areas and along major highways. As the mobile telephone user moves from one cell or area of coverage to another, the mobile telephone is transferred from one transmitter to the next. Today, the mobile network may be accessed not only by traditional personal mobile phones but also by Personal Data Assistants (PDAs), notebook computers with special communications cards, combination devices, etc. Many of these networks are capable of also transmitting through a number of existing protocols. Voice communication through the mobile network is also not as reliable as voice communication through the PSTN. Depending on the geography, certain areas may have better receptions than others. In large cities, for example, reception may be affected, for example, by big buildings, etc. A user who moves into a no-reception "pocket" can be "dropped" from the call. A user may also be dropped while being transferred from one transmitter to the next. For example, a transmitter may be at full capacity and thus not be able to handle additional users.

Satellites provide another medium through which voice can be transmitted. A satellite is a specialized wireless receiver/transmitter launched by a rocket and placed in orbit around the earth. There are hundreds of satellites currently in operation. Geostationary satellites, the most common type of satellite, orbit the earth directly over the equator remaining over the same spot at all times. A geostationary satellite can be accessed using an antenna aimed at the spot in the sky where the satellite hovers. A low-earth-orbit (LEO) system employs a large fleet of satellites in a circular orbit at a constant altitude of a few hundred miles over the geographic poles. An LEO satellite system operates similarly to a mobile phone network where users are transferred from satellite to satellite. As with any other wireless system of communication, reliability is a concern. The connection to the satellite may be affected by such things as weather, obstacles between the user and the satellite (such as being inside a building).

These and other types of networks through which voice may be transmitted are linked with one another to facilitate voice communication across all of the networks. For example, a mobile phone user may establish a telephone call with a user connected through the PSTN, a user having a satellite phone, a user connected through the Internet, etc. In addition, communication may be established between more than two users. Some telephone devices and services are "three-way" capable and establish communication between three users. Certain devices and services have the capability to conference three or more users. A telephone conference allows multiple parties to talk to each other in real time.

Typically, a conference leader contacts a telecommunications service provider and reserves a conference bridge, a computer-controlled device for interconnecting callers. The user may reserve a certain number of telephone lines at a specific date and time. The conference leader may provide each user with an access number and/or password/access code. The users may dial in from any type of voice-capable communications device that can access the bridge. The leader may also select dial-out service for some or all of the other users, where the leader provides bridge with the users' phone numbers, and the bridge either automatically or through an operator dials each user's telephone number at the scheduled time of the conference to connect the user to the conference bridge.

As the number of users increases, it becomes increasingly difficult to carry out the conference efficiently. At times, certain users may not be able to initially join the conference or a user may be dropped from a conference. The user, for example, may not be able to connect to (or may lose connection to) the conference due to problems with the user's handset or device, due to problems with one or more of the networks or due to too high network traffic, etc. In addition, the user may not be able to connect because of unexpected circumstances or because the user's handset is not functioning. When a user joins late or after having been dropped, the other users must interrupt the conference to "brief" the user or the user must join the conference without the benefit of the missed information. A user may also require briefing simply because the user did not clearly hear some of the information (for example, due to a bad connection), or because the user was absentminded, or because the user heard the conversation but simply did not understand it. It would also be desirable to be able to distinguish each user's contribution to the conference so as to be able to provide better information.

What is desired, therefore, is a method and system that can provide individual telephone users with one or more ways to review information relating to the conversation. Further, a method and system are desired that can perform the review of the information in real time subsequently allowing the user to return back to the live conference. The users should also be able to control the review process through voice commands over regular telephone devices. Finally, it would be desirable to have a way of identifying each user (either automatically or with the user's assistance) in order to track each user's to the conference.

SUMMARY

It has been discovered that a telephony recording system (PTR) can record a telephone conference and can replay the recording after the end of the conference or during the telephone conference. The PTR is capable of establishing a telephone conference between two or more users. The users can connect to the PTR from different types of networks. One user may, for example, connect through the mobile network, another may connect via satellite, and another may connect through the internet. Each user may connect to the PTR using a device having one or more types of communication lines. For example, a PDA may connect to the PTR through a voice line and a data line.

The PTR is also capable of recording the conference in audio format, text format (obtained by converting the audio to text), or both. If text is recorded in real time, the users will have the option of recalling the textual information in addition to recalling the recorded audio. Other information-such as time and user data-may also be recorded along with the audio and text. In one embodiment, both text and audio may be compressed (in real time if required) to save storage space.

Furthermore, the PTR is capable of identifying the users in the conference and associating with them each user's contribution to the conference. In some cases, the user or the user's device can assist by providing identification information. In these cases, if a user has not automatically provided identification information, the PTR may prompt the user (or query the user's device) for the information. The user may, for example, provide the information by voice or by inputting the information using the user's device. In another example, the user's device may communicate the information by sending data to the PTR using a predetermined communication protocol.

If the user or the user's device does not assist in the identification process, the PTR can attempt to identify the user using other methods. If for example, a user is calling from a unique, identifiable line, each user's contribution may be associated with that particular line. If, in addition, Caller ID information is sent over the line, that information may be used to further identify the user.

In another embodiment, the PTR could identify each user by analyzing the audio stream. The voice analysis data may also be compared to past data in order to associate a user with past user information and past conference data.

Once the users are identified, each user's contribution is tagged with a user identifier. Being able to distinguish the different users is very valuable. For example, users can now have the capability of individually adjusting the volume of the other users'incoming voice.

The foregoing is a summary and thus contains, by necessity, simplifications, generalizations, and omissions of detail; consequently, those skilled in the art will appreciate that the summary is illustrative only and is not intended to be in any way limiting. Other aspects, inventive features, and advantages of the present invention, as defined solely by the claims, will become apparent in the non-limiting detailed description set forth below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention may be better understood, and its numerous objects, features, and advantages made apparent to those skilled in the art by referencing the accompanying drawings. The use of the same reference symbols in different drawings indicates similar or identical items.

FIG. 1 is a high level network diagram of a personal telephony recorder system;

FIG. 2 is a is a block diagram of a personal telephony recorder system;

FIG. 3 is a is a hierarchy diagram of components used in a personal telephony recorder system;

FIG. 4 is a high level flowchart of adding participants to a telephone conference utilizing a personal telephony recorder system;

FIG. 5 is a data diagram of data maintained by a personal telephony recorder system;

FIG. 6 is high level flowchart of a personal telephony recorder system;

FIG. 7A is a system diagram of a client-based personal telephony recorder used by a primary user;

FIG. 7B is a system diagram of a network-based proxy used by primary and secondary users to provide personal telephony recorder services;

FIG. 8 is a high level system diagram of a personal telephony recorder proxy system;

FIG. 9 is a network diagram of a personal telephony recorder proxy system using proxy dialing with PSTN-centric telephones;

FIG. 10 is a network diagram of a personal telephony recorder proxy system using proxy dialing with PSTN-centric telephones as well as a Session Initiation Protocol (SIP) based telephone;

FIG. 11 is a signal diagram of a personal telephony recorder proxy system using proxy dialing with PSTN-centric telephones as well as a Session Initiation Protocol (SIP) based telephone;

FIG. 12 is a high level flowchart of a personal telephony recorder proxy service handling requests from users;

FIG. 13 is a flowchart showing steps taken in setting up a new conference call using a personal telephony recorder proxy service;

FIG. 14 is a flowchart showing the handling of user requests received at a personal telephony recorder proxy service;

FIG. 15 is a flowchart showing steps taken in joining a call to a teleconference being managed by a personal telephony recorder proxy service;

FIG. 16 is a high level network diagram for personal telephony recorder services;

FIG. 17 is a flowchart showing steps taken in recording a call using a personal telephony recorder;

FIG. 18 is a flowchart showing steps taken in processing user requests received at a personal telephony recorder;

FIG. 19 is a flowchart showing steps taken to convert stored voice data to textual data;

FIG. 20 is a flowchart showing high level steps taken to process a user's data retrieval request;

FIG. 21 is a flowchart showing steps taken to handle a basic personal telephony recorder request received from a user;

FIG. 22 is a flowchart showing steps taken to manage a call library using a personal telephony recorder;

FIG. 23 is a flowchart showing steps taken to record voice and voice metadata using a personal telephony recorder;

FIG. 24 is a flowchart showing steps taken to playback voice data using a personal telephony recorder;

FIG. 25 is a high level system diagram identifying participants in a personal telephony recorder call and handling participant-oriented adjustments;

FIG. 26 is a flowchart showing steps taken to identify users participating in a personal telephony recorder conference call;

FIG. 27 is a flowchart showing steps taken to adjust the volume of voice data received from or sent to individual participants;

FIG. 28 is a high level system diagram of setting and maintaining bookmarks corresponding to voice data recorded using a personal telephony recorder;

FIG. 29 is a flowchart showing steps taken in setting and maintaining bookmarks corresponding to recorded voice data;

FIG. 30 is a high level diagram of a personal telephony recorder processing voice commands received from a user;

FIG. 31 is a flowchart showing steps taken by a personal telephony recorder receiving and filtering voice commands received from a user;

FIG. 32 is a flowchart showing steps taken by a personal telephony recorder processing voice commands received from a user;

FIG. 33 is a high level diagram of a personal telephony recorder forwarding portions of a telephone call;

FIG. 34 is a higher level flowchart showing steps taken by a personal telephony recorder handing forwarding requests received from a user;

FIG. 35 is a flowchart showing steps taken by a personal telephony recorder in forwarding text data;

FIG. 36 is a flowchart showing steps taken by a personal telephony recorder in forwarding voice data;

FIG. 37 is a flowchart showing steps taken by a personal telephony recorder in forwarding portions of a call during the telephone call;

FIG. 38 is a network diagram showing a personal telephony recorder rejoining a participant that was dropped from a telephone conference;

FIG. 39 is a flowchart of steps taken by a personal telephony recorder in handling participants dropped from a telephone conference;

FIG. 40 is a flowchart of steps taken by a personal telephony recorder in playing back prior voice recordings for a user joining a conference call;

FIG. 41 is a system diagram of a user data mining words and phrases from call data recorded using a personal telephony recorder;

FIG. 42 is a flowchart of steps taken in creating an index of words and phrases during a call data mining operation;

FIG. 43 is a flowchart of steps taken annotating call text during a call data mining operation;

FIG. 44 is a flowchart of steps taken processing information mined from recorded telephone calls;

FIG. 45 is a flowchart showing steps taken in searching call data for a query request;

FIG. 46 is a flowchart showing steps taken in data mining words and phrases from a call library that includes numerous call recordings;

FIG. 47 is a flowchart showing steps taken in creating a custom report specification used to retrieve data found in call data files;

FIG. 48 is a flowchart showing steps taken in generating a custom report by retrieving data from call data files;

FIG. 49 is a flowchart showing steps taken in generating a transcription report from a call data file; and

FIG. 50 is a block diagram of an information handling system capable of implementing the present invention.

DETAILED DESCRIPTION

The following is intended to provide a detailed description of an example of the invention and should not be taken to be limiting of the invention itself. Rather, any number of variations may fall within the scope of the invention which is defined in the claims following the description.

FIG. 1 is a high level network diagram of a personal telephony recorder system. Personal telephony recorder 100 is used to record telephone data to various users and to provide the users with information. This information can include previously recorded call data which can be retrieved either during a telephone call or after a telephone call. In addition, personal telephony recorder 100 can receive information from computer network 115. One example of such a computer network is the Internet. Data received from the computer network can include voice data received from network connected telephony devices as well as non-voice information, such as the results of a search that the user requested. Personal telephony recorder 100 also provides services to participants engaged in a teleconference. For example, if one of the participants is dropped from the conference call, the personal telephony recorder informs the other participants of the drop. When the user reconnects to the personal telephony recorder, the device provides the reconnected participant with the ability to listen to the portion of the call that was missed.

Personal telephony recorder 100 can be a client-centric or network-centric device. In a client-centric application, the personal telephony recorder is connected to the user's computer or telephone system. Conversely, in a network-centric application, the personal telephony recorder is connected a network, such as telephone network 110 or computer network 120, and clients access the personal telephony recorder by logging into the personal telephony recorder or by being connected to the personal telephony recorder through a telephone call. In a network-centric application, therefore, the personal telephony recorder is available to the user regardless of the telephone currently being used.

Different devices connect to personal telephony recorder 100 in a variety of ways. Traditional telephone connects to a call being managed by a personal telephony recorder through telephone network 100, such as a public switched telephone network (PSTN).

Mobile telephone 140 and personal digital assistant (PDA) 170 can connect to either telephone network 110 or computer network 120. A gateway can be used to connect these devices from a wireless network to either the telephone network or the computer network.

Computer systems, such as personal computer 160 and laptop computer 150 typically connect to computer network 120. However, these devices are also capable of utilizing telephone network 110 using peripherals such as a modem.

FIG. 2 is a is a block diagram of a personal telephony recorder system. Personal telephony recorder 200 includes a number of components used to record call data and to provide services to users both during and after a telephone call. Personal telephony recorder users 205 speak into a microphone, such as the microphone provided on a telephone or a microphone connected to a computer system. Voice receiver component 210 receives analog speech from the user and sends the analog speech signal to command filter 215. Command filter 215 uses voice recognition software to recognize voice commands that might be included in the analog speech. When a command is identified, command filter 215 sends the analog speech to voice to text converter 245 which converts the command and words surrounding the command into a textual form. Voice to text converter 245, in turn, sends the textual form of the command and words surrounding the command (parameters) to command processor 250 for processing. In addition, a copy of the voice signal is stored in call buffer 255 so that the voice signal is able to be retrieved and processed at a later time (e.g., in response to a query request).

Returning to command filter 215, if the speech received from the user is not a command, command filter 215 passes the analog voice signal to analog transmitter 220. Analog transmitter 220 transmits the user's analog voice signal through network 225 to one or more participants 230. Network 225 may include a telephone network, such as a public switched telephone network (PSTN), and may include a computer network, such as the Internet.

Voice receiver 235 receives analog voice data from participants 230 through network 225. A copy of the received voice data is stored in call buffer 255. In one embodiment, participants other than the personal telephony recorder users are permitted to issue voice commands. In this embodiment, the voice signal received from the participants also passes through command filter 215 so that commands received from the participants can be identified and processed. Voice data is sent from voice receiver 235 to analog transmitter 240 which, in turn, transmits the analog voice data to personal telephony recorder users 205.

Returning to command processor 250, the command processor receives voice commands from voice to text converter 245. In addition, command processor 250 receives digital command signals from digital receiver 280. Digital commands can be received from a personal telephony recorder user using traditional telephone equipment (e.g., pressing various keys on the keypad, etc.). Digital commands can also be received from a computer system, or computer network, connected to the personal telephony recorder, such as computer system 282.

Command processor 250 retrieves call data from call buffer 255 in order to process some commands. The command processor may also use voice to text converter 245 and speech synthesizer 275. Voice to text converter 245 is used to convert analog call data to textual data which can then be processed or sent to computer systems using digital transmitter 285. In addition, the command processor can be programmed to receive all voice, including both voice data and voice commands, and convert the voice data to text using voice to text converter 245. Voice data that is not a command can be displayed, in a near real-time fashion, using digital transmitter 285 and either email/computer system 282 or a personal telephony recorder system that has a display device. In this manner, the personal telephony recorder user can follow the conference call by reading the data displayed on the display device. Command processor also stores additional data in nonvolatile storage area 260. Nonvolatile storage area can be nonvolatile memory, optical, magnetic storage, or any memory able to retain data value in a non-powered state. In addition, memory can be used in place of the nonvolatile storage, typically offering faster access and retrieval, but lacking the ability to retain values when power is interrupted.

Nonvolatile storage 260 is used to store voice data, bookmark data (marking places within the voice data), translation data (digital forms of the analog voice data), queries and command that have been requested, and data regarding participants of the calls, such as the participants' names, companies, telephone numbers, and the like.

Command processor 250 also interfaces with dropped line handler 265 to notify call participants when a participant gets dropped from a teleconference. Dropped line handler 265 also uses dropped line buffer 270 to set bookmarks corresponding to when a participant is dropped and rejoins a teleconference as well as data regarding the playback of voice data that was missed by the caller before he or she rejoined the call. For example, when a dropped participant rejoins a teleconference, dropped call handler will retrieve the voice data that transpired while the participant was not connected and allow the participant to listen to the missed voice data.

FIG. 3 is a is a hierarchy diagram of components used in a personal telephony recorder system. Personal telephony recorder 300 includes establish calls component 310 for establishing a telephone call or teleconference. This is performed somewhat differently depending on whether the personal telephony recorder is acting in a proxy role (connected to a network rather than any one specific participant) or is connected to a particular participant. Establish calls component 310 includes subcomponents to setup service in a proxy environment 315, components to connect participants to one another 320, and components to identify individual participants 325.

Another personal telephony recorder component is record calls component 330 for recording voice data transmitted during a telephone or conference call. Command processing component 340 includes a number of subcomponents for responding to requests and commands received by the personal telephony recorder from participants and users. These subcomponents include bookmarking, data retrieval, dropped line handling, and data mining.

Bookmarking component 345 is used to allow the personal telephony recorder user the ability to set bookmarks identifying where in a telephone call a certain topic was discussed. In addition, bookmarks are used to retrieve a portion of a recorded telephone call for forwarding the portion. Bookmarks are also automatically generated when a participant is dropped from a conference call (marking the point at which the participant was dropped) as well as used for marking the point at which the user rejoined a conference call.

Data retrieval component 350 is used to retrieve various types of call data and perform various functions with the retrieved data. Further subcomponents provide this functionality. These subcomponents include basic retrieval component 355, call forwarding component 360, and specialized retrieval component 375. Of these subcomponents, the forwarding component includes two subcomponents--text forwarding subcomponent 365 and voice forwarding component 370.

Another command processing component is dropped line handling component 380. The dropped line handling component detects when a teleconference participant is dropped from a telephone call and, upon the dropped participant rejoining the call, allows the participant the ability to listen to the portion of the call that the participant missed.

Data mining component 385 is used to cull information from the call data. The call data information is used by data mining subcomponents for creating reports (subcomponent 390) and processing ad hoc queries (subcomponent 395).

FIG. 4 is a high level flowchart of adding participants to a telephone conference utilizing a personal telephony recorder system. Processing commences at 400 whereupon the first participant in the telephone call is identified (predefined process 405, see FIG. 26 for processing details). A determination is made as to whether there are more participants to identify (decision 410). If there are more participants, decision 410 branches to "yes" branch 412 which loops back to identify the next participant (predefined process 415, see FIG. 26 for processing details). This looping continues until there are no more participants to identify, at which point decision 410 branches to "no" branch 418.

Voice data and signals are received (step 420) from telephone network 425 (for those participants that are remote from the personal telephony recorder) and from telephone(s) 428 (for those participants that are directly connected to the personal telephony recorder). A determination is made as to whether the received voice and/or signal data includes a personal telephony recorder command (decision 430). If a command was received, decision 430 branches to "yes" branch 432 whereupon the personal telephony recorder processes the received command (predefined process 435, see FIG. 20 for processing details). On the other hand, if a command was not received (i.e., normal voice traffic was received), decision 430 branches to "no" branch 442 whereupon the participant from whom the voice data was received is identified (step 445). This identification can be based upon the line from which the data was received or may be made by analyzing the vocal characteristics of the participants voice. An identifier corresponding to the participant and the received voice data is stored (step 450) in call buffer memory area 455.

A determination is made as to whether the received voice data was from a locally connected personal telephony recorder user or from another participant connected to the personal telephony recorder through a telephone network (decision 460). If the voice data was received from a locally connected personal telephony recorder user, decision 460 branches to "yes" branch 462 whereupon the voice data is transmitted to the other participants (step 465) through telephone network 425. On the other hand, if the voice data was received from the telephone network, decision 460 branches to "no" branch 472 whereupon the voice data is transmitted to the locally connected personal telephony recorder user (step 475) through locally attached telephone speaker 428.

After receiving the last command or voice data, a determination is made as to whether the participants have ended the telephone call (decision 485). If the call has not ended, decision 485 branches to "no" branch 486 which loops back to process the next command or voice data. This looping continues until the call has ended, at which point decision 485 branches to "yes" branch 488 whereupon the call data stored in buffer 455 is stored (step 490) on nonvolatile storage device 492 in order to retain the call data indefinitely. Processing thereafter ends at 495.

FIG. 5 is a data diagram of data maintained by a personal telephony recorder system. Buffer data 500 includes a variety of information maintained by the personal telephony recorder. Call buffer 510 includes voice data received during the course of a telephone call. The call buffer includes addresses 515 and raw (analog) voice data 520 that was received. The analog voice data is stored sequentially so that the first voice data stored is stored towards the top of the call buffer while later retrieved voice data is stored towards the bottom of the buffer.

Participant data 525 includes information regarding the participants. Participants are assigned a unique identifier 535 so that the participant's identity can be tracked during the course of a telephone call. Participant data also includes descriptive information 540 regarding the participant. Descriptive information can include participants' names, telephone numbers, company names, addresses and the like. Descriptive information can also include voice signature data used to identify the participant using voice recognition software.

Participant data 525 also includes participant call tracking data 545 that tracks contributions made to a telephone call by the various participants. Tracking data 545 includes a pointer (550) to the address within the voice data where a contribution was made as well as the participant's unique identifier 555. In addition, a second pointer can be maintained tracking when the participant finished talking and another participant began talking.

Bookmark data 560 is used to mark places within the voice data. For example, during a lengthy conference call, a personal telephony recorder user may wish to bookmark where in the call specific terms were discussed. In this manner, the user can return to the portion of the call at a later time without having to scan through the other portions of the call and without having to take time consuming and lengthy notes during the call. Bookmark data 560 includes bookmark identifier 565 that is assigned to uniquely identify the bookmark, pointer 570 used to mark the location of the bookmark (i.e., the address) within call buffer 510. Bookmark data 560 also includes an optional bookmark description 575 which is used by the user to store a description of the bookmark. In the example above, a bookmark description may be "discussion of terms."

Line drop data 580 is used to store data regarding participants that are dropped from a conference call. Line drop data 580 includes a line drop identifier which uniquely identifies the line drop event. Drop pointer 584 which indicates the location, or address, within the call buffer when the participant was dropped. Drop timestamp 586 stores the time at which the participant was dropped. Rejoin pointer 588 indicates the position of the call buffer when the participant rejoined the conference call. Thus, playing the data that was stored in the call buffer between drop pointer 584 and rejoin pointer 588 would play the portion of the call that was missed by the participant from the time the participant was dropped to the time the participant rejoined the call. Rejoin timestamp 590 stores the time at which the participant rejoined the call. Replay pointer 592 is used to keep track of how much of the call buffer missed by the participant has been replayed to the participant.

FIG. 6 is high level flowchart of a personal telephony recorder system. Processing commences at 600 whereupon A determination is made as to whether the user is engaging in a new (live) telephone call or is requesting data regarding previously recorded telephone calls (decision 610). If the user is engaging in a new, or live, call, decision 610 branches to "yes" branch 615 whereupon a call is established either using a locally connected personal telephony recorder device or a network accessible (proxy) personal telephony recorder device (predefined process 620). During the telephone call, the call data is stored (predefined process 630) in call storage 640. Commands received by personal telephony recorder users during the call are processed (predefined process 650) using previously recorded call data 640 as well as call metadata 660 which includes data regarding the telephone call (e.g., the participants).

On the other hand, if the user is requesting data regarding previously recorded telephone calls, decision 610 branches to "no" branch 675 whereupon post-call commands and requests are received by the user and processed (predefined process 680) using previously recorded call