JP2001268669A - Device and method for equipment control using mobile telephone terminal and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an equipment control system which is general to equipments and specified to users. SOLUTION: When a user speaks, a voice recognition part 5 compares voice features extracted by a voice feature extraction part 2 with a voice feature dictionary held in a user information storage part 4 and a conversation control part 8 recognizes a key work that the user speaks from recognition candidates set with status information, etc., of the equipment. The conversation control part 8 sends a command for controlling the equipment to an equipment control part 9 according to the recognized key word to control the equipment 10. The conversation control part 8 generates a speech sentence, limits a user speech, and explains and changes the key word in addition to the conversion from the voice recognition result to the command and the limitation of the voice recognition candidates.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device control apparatus, a method, and a recording medium using a mobile telephone terminal. More specifically, the present invention relates to a mobile telephone terminal that performs voice recognition, voice synthesis, and interactive processing. The present invention relates to a device control apparatus, a method, and a recording medium using a mobile telephone terminal, which control the device using a mobile telephone terminal as a personalized user interface of a device requiring complicated operations.

[0002]

2. Description of the Related Art Japanese Patent Laid-Open No. 5-130181 discloses that
An invention relating to a cordless telephone device having an infrared remote control function such as a V device or an air conditioner is described. Japanese Patent Application Laid-Open No. Hei 10-93665 discloses an invention relating to an indoor device control device for controlling a plurality of devices using a mobile telephone terminal. In this device, a function list is transmitted from the device side, and the user can select it with a mobile phone and instruct an operation.

[0003]

In today's modern day where people are surrounded by a variety of devices, it is necessary to remember user interfaces designed separately for each device. I have. Therefore, we carry one user interface unit that is general-purpose for the device and specialized for the user,
It would be ideal if a plurality of devices could be controlled by this. There is a great merit of using a mobile telephone terminal having a communication function and excellent portability as such a general-purpose user interface.

[0004] In the "cordless telephone device" described in Japanese Patent Application Laid-Open No. 5-130181, infrared communication is performed between the cordless telephone device and the device due to its high affinity with home appliances. However, since a plurality of devices are operated with a limited number of buttons on the telephone, the user load is still large, for example, it is necessary to memorize a plurality of correspondences between button operations and device operations.

[0005] In the "indoor equipment control apparatus using a mobile telephone terminal" described in Japanese Patent Application Laid-Open No. Hei 10-93665, a user transmits a list of controllable functions from the equipment side to the mobile telephone so that the user can view the list. Operation can be selectively performed, so that the memory load for the operation can be reduced, and the number of compatible devices can be expanded. However, when there are a large number of functions of the device, it is not easy to select a desired function from these functions using only a display device and buttons similar to those of an ordinary mobile phone. Further, when a large number of parameters need to be input for a setting operation or the like, a general-purpose interface tends to repeat complicated operations. Also, when considering control of various devices, it is difficult to convey a function unique to each device to the user in a simple display, and it is also difficult to store all the operations. Even if you can control multiple devices with a general-purpose user interface,
Since an operation system specialized for an individual cannot be constructed, the load on the user is large.

On the other hand, speech recognition has the following problems. Although it is difficult to obtain a high recognition rate for any utterance without specializing in pronunciation of a specific user, it is necessary for devices shared by an unspecified number of people to accurately recognize everyone's voice. In addition, since words used in speech change depending on individual differences, contexts, and situations, there is always ambiguity in the conversion to device operation commands, and in situations where the speaker and the context of the speaker cannot be specified. This leads to an increase in recognition candidates, leading to a further reduction in the recognition rate.

[0007] The present invention has been made in view of the above circumstances, and has the following objects. Claims 1, 16,
The inventions of the inventions 31 and 32 make it possible to use various functions of a device without increasing a user's memory load by transmitting a device control command through a dialogue through a mobile telephone terminal.

According to the inventions of claims 2, 17, 31, and 32, various functions of the equipment can be used without increasing the memory load of the user by transmitting the equipment control command by voice dialogue through the mobile telephone terminal. At the same time, we aim to improve the voice recognition rate by performing voice recognition specialized for individuals even for an unspecified number of users.

According to the inventions of claims 3, 18, 31, and 32, by simultaneously recognizing a plurality of keywords, an operation instruction including parameter setting which tends to be complicated can be easily used.

According to the inventions of claims 4, 19, 31, and 32, even a complicated function which cannot be understood only by character information with a limited display area can help the user to understand by a voice explanation.

According to the invention of claims 5, 20, 31, and 32, the user changes the correspondence between the command and the keyword, thereby constructing an operation system capable of storing the information without difficulty for each user.
Reduce memory load.

According to the inventions of claims 6, 21, 31, and 32, the utterance history and the command history of each user are stored in the mobile telephone terminal, thereby estimating the context of the user and eliminating the ambiguity of the user utterance. , Is converted into an appropriate device operation command.

[0013] According to the invention of claims 7, 8, 22, 23, 31, and 32, the knowledge acquired from the operation of similar devices can be easily reused, so that it is appropriate for each user and low in stress. Implement a user interface.

The invention according to claims 9, 24, 31, and 32 generates a message that can be easily understood by the user based on the knowledge acquired in the past.

[0015] The invention of claims 10, 25, 31, 32 is
We aim to improve the speech recognition rate by focusing on vocabulary that is easy for the user to speak.

The invention of claims 11, 26, 31, 32 is
Guide operations that are comfortable for the user along with the familiar workflow.

The invention according to claims 12, 27, 31, 32 is
By predicting the order of the utterances, the recognition target is narrowed down, and the speech recognition rate is improved.

The invention of claims 13, 28, 31, 32 is
Prevents erroneous operations by emphasizing and guiding functions specific to the operation target device.

The invention of claims 14, 29, 31, 32 is
Improve operation efficiency by setting appropriate defaults.

The invention of claims 15, 30, 31, and 32 is
It is possible to perform an operation focusing on a setting item that the user attaches importance to.

[0021]

The invention according to claim 1 comprises an equipment control unit for controlling one or a plurality of equipment, and a mobile telephone terminal for each user, which acts as an interface for the user. A command representing an instruction, a parameter necessary for operation setting of the device, and a relationship between the command and the parameter are stored, and a command selected by a user by the mobile phone terminal is notified to the device control unit. A device control device using a mobile telephone terminal, wherein the device control device controls the device by input recognition means for recognizing an input including a keyword, output synthesis means for sending an output message to a user, and status information of the device. Limiting a recognition candidate of the input recognition means, generating a device control command from a recognition result of the input recognition means, Dialogue control means for generating an output message in response to an information request from a device, and device control means for controlling the device by the device control command generated from the dialogue control device and sending status information of the device to the dialogue control device And the following.

According to a second aspect of the present invention, there is provided a device control unit for controlling one or a plurality of devices, and a mobile phone terminal for each user, the command representing an operation instruction to the device. The parameters necessary for the operation setting of the device and the relationship between the command and the parameter are stored, the command selected by the user by the mobile telephone terminal is notified to the device control unit, and the device is controlled by the device control unit. A device control device using a mobile telephone terminal, a voice recognition unit for recognizing a user utterance including a keyword, a voice synthesis unit for sending a voice output message to a user, and the voice recognition unit based on status information of the device. Restrict recognition candidates for,
A device control command is generated from the recognition result of the voice recognition unit, a dialogue control unit that generates an output message in response to an information request from the device, and the device is controlled by a device control command generated from the dialogue control unit. Device control means for sending status information of the device to the dialog control means.

According to a third aspect of the present invention, in the second aspect of the present invention, there are a plurality of keywords recognized by the voice recognition means at the same time, and by recognizing a plurality of keywords, an operation instruction combining a plurality of the commands, It is characterized in that a plurality of parameters can be set simultaneously.

According to a fourth aspect of the present invention, in the second aspect of the present invention, the mobile telephone terminal adds, to the keyword,
It is characterized in that it holds explanation information on the operation of the device corresponding to the keyword, and presents the held explanation information by voice or visual display at the request of the user.

According to a fifth aspect of the present invention, in the second aspect of the present invention, the correspondence between the keyword and the command can be changed by operating the mobile telephone terminal.

According to a sixth aspect of the present invention, in the second aspect of the present invention, the dialog control means holds a history of a user's utterance, a history of a device utterance and a command history up to the present time, and adds the history to the status information of the device. The present invention is characterized in that recognition candidates in the voice recognition means are restricted by using the held history.

According to a seventh aspect of the present invention, there is provided a mobile telephone, comprising: a device control unit for controlling one or a plurality of devices; and a mobile telephone terminal for each user, which controls the user. A device control device using a terminal, a history storage unit for storing an operation history and a dialog history, and analyzing data stored in the history storage unit to output at least one of a vocabulary list and a workflow A user model analyzing unit, a device classifying unit that classifies the used device into a device class having a typical function group according to the purpose, the work content, and the like, and a result of the analysis by the device classifying unit, the analysis result of the user model analyzing unit. User model storage means for storing in association with the user model, and dialog processing means for performing a dialog process using data stored in the user model storage means It is obtained by further comprising a.

[0028] The invention according to claim 8 comprises an equipment control unit for controlling one or a plurality of equipment, and a mobile telephone terminal for each user, which controls the equipment by voice dialogue. A device control device using a telephone terminal, comprising: a history storage unit for storing an operation history and a dialog history; and analyzing data stored in the history storage unit to determine at least one of a vocabulary list and a workflow. A user model analyzing means for outputting, a device classifying means for classifying a used device into a device class having a typical function group according to a purpose, a work content, and the like; and an analysis result of the user model analyzing means for classification of the device classifying means. User model storage means for storing in association with a result, and interactive processing for performing interactive processing using data stored in the user model storage means It is obtained by comprising: the stage, a.

A ninth aspect of the present invention is characterized in that, in the eighth aspect of the present invention, a voice output message is generated using a vocabulary list associated with the device classification to which the operation target device belongs.

A tenth aspect of the present invention is characterized in that, in the eighth aspect of the present invention, a speech recognition candidate is generated using a vocabulary list associated with the device classification to which the operation target device belongs.

[0031] The invention of claim 11 is characterized in that, in the invention of claim 8, a voice output message is created using a workflow associated with the device classification to which the device to be operated belongs.

A twelfth aspect of the present invention is characterized in that, in the eighth aspect of the present invention, a speech recognition candidate is generated using a workflow associated with the device classification to which the device to be operated belongs.

According to a thirteenth aspect of the present invention, in the invention of the eighth aspect, a detailed voice output message is generated for a function not included in the function group associated with the device classification to which the operation target device belongs, and the message is emphasized. It is characterized by generating synthesized speech.

According to a fourteenth aspect of the present invention, in the invention of the eighth aspect, the workflow is such that the setting values set by the target user with respect to the setting items to be set by the user when using the device are different even when using a different device. When the change is small and stable, the setting value of the setting item is created as a default value.

The invention of claim 15 is characterized in that, in the invention of claim 8, the workflow is created by lowering the priority of the setting process for a setting item whose change from the default value is small. Things.

The invention according to claim 16 comprises a device control unit for controlling one or a plurality of devices, and a mobile telephone terminal for each user, which serves as an interface for the user, and a command indicating an operation instruction to the device. The parameters necessary for the operation setting of the device and the relationship between the command and the parameter are stored, the command selected by the user by the mobile telephone terminal is notified to the device control unit, and the device is controlled by the device control unit. A device control method using a mobile telephone terminal, comprising: an input recognition step of recognizing an input including a keyword; an output combining step of sending an output message to a user; and recognition of the input recognition step from status information of the device. Limiting the candidates, generating a device control command from the recognition result of the input recognition step, A dialogue control step of generating an output message in response to an information request from a device, and a device control step of controlling the device by the device control command generated in the dialogue control step and sending status information of the device to the dialogue control step And the following.

The invention according to claim 17 comprises a device control unit for controlling one or a plurality of devices, and a mobile telephone terminal for each user, which serves as an interface for the user, and a command indicating an operation instruction to the device. The parameters necessary for the operation setting of the device and the relationship between the command and the parameter are stored, the command selected by the user by the mobile telephone terminal is notified to the device control unit, and the device is controlled by the device control unit. A device control method using a mobile telephone terminal, a voice recognition step of recognizing a user utterance including a keyword, a voice synthesis step of sending a voice output message to a user, and the voice recognition step from status information of the device. And the device control command is restricted based on the recognition result of the voice recognition step. And an interactive control step of generating an output message in response to an information request from the device, and controlling the device by a device control command generated in the interactive control step, and sending status information of the device to the interactive control step And a device control step.

The invention of claim 18 is the invention of claim 17, wherein there are a plurality of keywords recognized in the voice recognition step at the same time, and by recognizing a plurality of keywords, an operation instruction combining a plurality of the commands, It is characterized in that a plurality of parameters can be set simultaneously.

According to a nineteenth aspect, in the seventeenth aspect, the mobile telephone terminal holds, in addition to the keyword, explanation information on an operation of a device corresponding to the keyword, and the held explanation. Information is presented by voice or visual display at the request of the user.

[0040] The invention of claim 20 is the invention of claim 17, wherein the correspondence between the keyword and the command can be changed by operating the mobile telephone terminal.

According to a twenty-first aspect of the present invention, in the seventeenth aspect, the interaction control step holds a history of a user utterance, a history of a device utterance, and a command history up to the present time, and adds the history to the status information of the device. The present invention is characterized in that recognition candidates in the voice recognition step are limited by using the held history.

According to a twenty-second aspect of the present invention, there is provided a mobile telephone, comprising: a device control unit for controlling one or a plurality of devices; and a mobile phone terminal for each user, which controls the user. A device control method using a terminal, comprising: a history storing step of storing an operation history and a dialog history; and analyzing data stored in the history storing step to output at least one of a vocabulary list and a workflow. A user model analysis step, a device classification step of classifying a used device into a device classification having a typical function group according to purpose, work content, and the like, and a classification in the device classification step of an analysis result in the user model analysis step. Storing a user model storing step in association with the result; and storing the data stored in the user model storing step. Dialogue processing step of performing an interactive process with use, in which characterized in that it comprises a.

According to a twenty-third aspect of the present invention, there is provided a device control unit for controlling one or a plurality of devices, and a mobile telephone terminal for each user, which controls an interface of the user. A device control method using a telephone terminal, comprising: a history storage step for storing an operation history and a dialog history; and analyzing data stored in the history storage step to output at least one of a vocabulary list and a workflow. A user model analyzing step, a device classifying step of classifying a used device into a device class having a typical function group according to a purpose, a work content, and the like, and an analysis result in the user model analyzing step in the device classifying step. Storing the user model in association with the classification result; and storing the data stored in the user model storing step. Dialogue processing step of performing a conversation process using the one in which was characterized in that it comprises a.

According to a twenty-fourth aspect of the present invention, in the twenty-third aspect, a voice output message is generated using a vocabulary list associated with the device classification to which the device to be operated belongs.

A twenty-fifth invention is characterized in that, in the twenty-third invention, a speech recognition candidate is generated using a vocabulary list associated with the device classification to which the device to be operated belongs.

A twenty-sixth invention is characterized in that, in the twenty-third invention, a voice output message is created using a workflow associated with the device classification to which the device to be operated belongs.

In a twenty-seventh aspect, in the twenty-third aspect, a voice recognition candidate is generated using a workflow associated with the device classification to which the operation target device belongs.

According to a twenty-eighth aspect, in the twenty-third aspect, a detailed voice output message is generated and emphasized for a function not included in the function group associated with the device classification to which the operation target device belongs. It is characterized by generating synthesized speech.

According to a twenty-ninth aspect of the present invention, in the twenty-third aspect of the present invention, the workflow is such that the setting value set by the target user with respect to the setting item to be set by the user when using the device is different even when using a different device When the change is small and stable, the setting value of the setting item is created as a default value.

The invention according to claim 30 is characterized in that, in the invention according to claim 23, the workflow is created by lowering the priority of the setting process for setting items whose change from the default value is small. Things.

According to a thirty-first aspect of the present invention, there is provided a program recording a program for functioning as a device control device using the mobile telephone terminal according to any one of the first to fifteenth aspects.

The invention of claim 32 is the invention of claims 16 to 30.
A program for realizing the device control method using the mobile telephone terminal described in any one of the above items is recorded.

[0053]

FIG. 1 is a block diagram showing a configuration of a device control device according to an embodiment of the present invention. The mobile phone terminal and the device control unit, which are components of the device control device according to the present embodiment, can communicate by any kind of wireless communication such as radio waves or infrared rays, and this communication can be used directly or via a certain base station. It is possible. Further, the device control unit may be built in the device 10 or may be of a separate housing connected by some kind of interface. Also, one for one device
It is possible to configure so as to be controlled by one device control unit, or to control a plurality of devices by one device control unit.

In this embodiment, the operation unit 1, the voice feature extraction unit 2, the voice synthesis unit 3, and the user information storage unit 4 are provided in the mobile telephone terminal, and the voice recognition unit 5, the voice feature generation unit 6, and the device information storage unit 7, a dialog control unit 8, and a device control unit 9 are provided in a device control unit. In this configuration, the processing at the mobile telephone terminal is DS
P (Digital Surround Procedures)
sor)
It aims to improve processing efficiency. On the other hand, the operation unit 1,
By including a voice feature extraction unit 2, a voice synthesis unit 3, a user information storage unit 4, a voice recognition unit 5, a voice feature generation unit 6, a device information storage unit 7, and a dialogue control unit 8 in a mobile telephone terminal, device control is performed. While reducing the processing volume of the unit,
By reducing the amount of communication data between the mobile telephone terminal and the device control unit, it is possible to facilitate connection of the device control unit to many devices and to promote the spread of the device. Thus, the implementation of each unit is not fixed to either the mobile telephone terminal or the device control unit.

As an implementation of the present invention, a command from a mobile phone as a mobile phone terminal may be sent to a device control unit connected to a device to control the device. Alternatively, each device may be controlled from a device control unit connected to (or including) the telephone by interacting with a home telephone using a mobile telephone. The present invention is a device in which an unspecified number of users perform a somewhat complicated operation,
The effect is particularly exhibited, and examples thereof include the operation of a copying machine in a convenience store and the operation of a karaoke machine in a karaoke box. The operation of each unit in the present embodiment when the copying machine is the control target device 10 will be described below.

The user information storage unit 4 stores unique information for each user. Specifically, it is a speech feature dictionary, an utterance history, a command history, a user keyword, and the like. The operation unit 1 inputs information of the device 10 to the device information storage unit 7 or assists in registration.

The device information storage unit 7 stores information unique to the device controlled by the device control unit. Specifically, device I
D, a correspondence table of keywords, commands, parameters, and descriptions. FIG. 2 is a diagram illustrating an example of device-specific information according to the present embodiment. In this example, in the above correspondence table, as keywords, keyword list, keyword change, margin / margin / binding margin, both sides, enlargement /
Zoom, word, n centimeters / n mm, even / odd, n
The percentages are listed. For example, the keyword "margin /
The command for "margin / binding margin" is #MARGIN
Since "width" is required for the keyword, the command $ WIDTH is used as a parameter. The description of the keyword is associated with the display or utterance of “Reduce and adjust the binding margin”. In the table, n represents an arbitrary numeric expression, and word represents an arbitrary linguistic expression.

The voice feature extraction unit 2 extracts voice features required for voice recognition from a voice waveform input from a microphone of the mobile telephone terminal and converted into a digital signal. The speech features and the method of extracting them are not different from those used in the existing speech recognition technology, so that detailed description is omitted here.

The speech recognition unit 5 compares the speech features extracted by the speech feature extraction unit 2 with the speech feature dictionary stored in the user information storage unit 4, and selects one of the recognition candidates set by the dialog control unit 8. Recognize keywords uttered by the user. At this time, a plurality of keywords can be recognized simultaneously. For example, from a user utterance “reduce to A4 and copy 10 copies on both sides”, “A4”, “reduce”, “double-sided”,
A keyword such as "10 copies" is obtained. Since the recognition process is not different from that used in the existing speech recognition technology, a detailed description is omitted here.

The functions of the dialogue control unit 8 are roughly classified into three. These are conversion of speech recognition results into commands, limitation of speech recognition candidates, generation of utterance sentences of the device and limitation of user utterances, and explanation and modification of keywords. This will be described below in order.

In a multifunctional device, the number of commands also increases, and the keyword corresponding to one command is 1
If not all candidates are the target recognition candidates of the speech recognition unit 5, they become enormous, which leads to an increase in processing time and a decrease in recognition rate.

Therefore, the dialog control unit 8 includes the device control unit 9
From the status information of the device 10 obtained from the user, the utterance history and the command history obtained from the user information storage unit 4,
The recognition candidates are limited and sent to the voice recognition unit 5. As an example of limiting recognition candidates based on status information, if a copying operation is currently being performed, various setting keywords are not recognized as recognition candidates, but "stop", "interrupt", "cancel", and the like are candidates. As an example of limiting candidates from the utterance history, if the user of the mobile phone terminal used the keyword “margin” in the previous use, “margin” is set preferentially instead of “margin” as a recognition candidate and so on. As an example of limiting candidates from the command history, when a copy setting command has not been issued and a sorter use command has been issued, a keyword corresponding to the copy setting command is preferentially set.

The speech recognition results obtained by the speech recognition unit 5 from the recognition candidates selected as described above are stored in the correspondence table of the device information storage unit 7 (see FIG. 2) and the user / keyword of the user information storage unit 4. , And is sent to the device control unit 9. However, if the utterance is not recognized in the command requiring the parameter, the device utterance requesting the parameter is synthesized. FIG. 3 is a flowchart for explaining a process of executing a command from the speech recognition result obtained by the speech recognition unit 5 in FIG. First, a keyword is recognized by the voice recognition unit 5 (step S1), and it is determined from the correspondence table stored in the device information storage unit 7 whether a parameter is required (step S2). If no parameters are required, the command is executed as it is (step S6). If parameters are required, step S3
It is determined whether or not there is a parameter voice. If there is no parameter voice, the command is executed (step S6). If there is a parameter voice, a description of the parameter is output (step S5) and the command is executed (step S6).

A device utterance sentence is generated from the parameter request,
The voice utterance is sent to the voice feature generation unit 6 to urge the user to utter, and the utterance of the user is limited to narrow down recognition candidates for the next utterance. At this time, the utterance history and the command history in the user information storage unit 4 are referred to as necessary, and a simple and easy-to-understand expression according to the context of the user is selected. For example, if the user has already used the keyword “margin”, the device utterance uses the keyword “margin” instead of “margin”. Further, by outputting a device utterance sentence "What is the copy?", The user utterance "~" is suppressed and the next utterance recognition candidates are narrowed down.

The user does not initially know what keyword can do what. Therefore, in the present embodiment, the meta command% KW for explaining the keyword list
A LIST is prepared (see FIG. 2). % KWLIS
When T is executed, the dialogue control unit 8 generates all keywords and their descriptions as device utterances. Also, a meta command% KWCHANGE for changing the expression of the keyword when the expression is uncomfortable for the user is prepared. When% KWCHANGE is executed, the parameter $ NEWKW, which is a new keyword, is used as a user keyword together with the device ID and command as the user keyword in the user information storage unit 4.
Is stored. The description and change of these keywords can be realized by a visual operation using a liquid crystal screen and buttons provided in the mobile telephone terminal, in addition to the voice method described here. Moreover, it is also feasible to use them in combination.

The voice feature generation unit 6 generates a voice feature that defines phonemic / prosodic information from the device utterance sentence generated by the dialogue control unit 8. This can be used at a symbol level such as a phonetic symbol, or at a parameter level based on a column such as a phoneme number, a duration time, and a pitch frequency. Since this process is not different from that used in the existing text-to-speech synthesis technology, a detailed description is omitted here.

The voice synthesizing unit 3 generates a voice waveform from voice characteristics. Since this process is not different from that used in the existing rule speech synthesis technology, detailed description is omitted here.

The device control unit 9 sends a control signal to the device 10 based on the device control command and the parameter selected by the dialog control unit 8. Further, it receives a status signal from the device 10 and sends it to the dialogue control unit 8.

In this embodiment, not only the explanation and the change of the keyword but also all the processes are displayed on the display provided in the mobile telephone terminal and operated by using the buttons. It is also possible to adopt a configuration in which processing by voice and processing by display and buttons are used together.

FIG. 4 is a block diagram showing a configuration of a device control device according to another embodiment of the present invention. The mobile phone terminal and the device control unit, which are components of the device control device according to the present embodiment, can communicate with each other by some kind of wireless communication such as radio waves or infrared rays, and this communication can be used directly or via a base station. is there. The device control unit may be built in the device 20, but may be realized as a separate housing connected by some interface. Also, one for one device
It is possible to configure so as to be controlled by one device control unit, or to control a plurality of devices by one device control unit.

In this embodiment, a mobile telephone terminal is provided with an operation unit 11, a voice feature extraction unit 12, a voice synthesis unit 13, and a user model storage unit 14, and a voice recognition unit 15, a voice feature generation unit 16, and a device information storage unit. 17, a dialog control unit 18, a device control unit 19, a user model analysis unit 21, and a history storage unit 22 are provided in a device control unit. In this configuration, processing at the mobile phone terminal is performed by a DSP (Digital Surround PDS).
The purpose of this is to improve the processing efficiency by grouping the devices suitable for the use of the processor. On the other hand,
Operation unit 11, voice feature extraction unit 12, voice synthesis unit 13, user model storage unit 14, voice recognition unit 15, voice feature generation unit 16, device information storage unit 17, user model analysis unit 2
1, the history storage unit 22 and the dialogue control unit 18 are included in the mobile telephone terminal, thereby reducing the processing amount of the equipment control unit and reducing the communication data amount between the mobile telephone terminal and the equipment control unit. A configuration that facilitates connection of the device control unit to another device and promotes the spread of the present device is also possible. Thus, the implementation of each unit is not fixed to either the mobile telephone terminal or the device control unit.

The present invention is particularly effective in a device in which an unspecified number of users perform a somewhat complicated operation.
Examples of such operations include the operation of a copier in a convenience store and the operation of a karaoke machine in a karaoke box. Hereinafter, the operation of each unit in the present embodiment when the copying machine is the control target device 20 will be described.

The operation unit 11 inputs device information to the device information storage unit 17 or assists registration. The device information storage unit 17 stores information unique to the device 20 controlled by the device control unit. Specifically, device ID, keyword,
It stores a correspondence table of commands, parameters, and descriptions, and also stores the device classification to which the device 20 belongs. FIG. 2 shows an example of the above correspondence table. In FIG. 2, the keywords include a keyword list, a keyword change,
Margin / binding allowance, both sides, enlargement / zoom, word, n
Cm / n mm, even / odd, and n percent. For example, the command for the keyword "margin / margin / binding margin" is #MARGIN, and "width" is required for the keyword, so the command $ WIDTH is used as a parameter. The description of this keyword is associated with the display or utterance of "Reduce and adjust the binding margin." The device classification is obtained by previously grouping devices having high similarities such as purpose, function, and operability, such as a copier and a facsimile, a tape recorder and a CD player, and a jukebox and a karaoke device.

The voice feature extraction unit 12 extracts voice features necessary for voice recognition from a voice waveform input from a microphone of the mobile telephone terminal and converted into a digital signal. The speech features and the method of extracting them are not different from those used in the existing speech recognition technology, so that detailed description is omitted here.

The speech recognition unit 15 compares the speech features extracted by the speech feature extraction unit 12 with the speech feature dictionary stored in the user information storage unit (user model storage unit) 14, and is set by the dialog control unit 18. The keywords uttered by the user are recognized from the recognized candidates. At this time, a plurality of keywords can be recognized simultaneously. For example, from a user's utterance “Reduce to A4 and copy 10 copies on both sides”,
Keywords such as "A4", "reduced", "double-sided", and "10 copies" are obtained. Since the recognition process is not different from that used in the existing speech recognition technology, a detailed description is omitted here.

The voice feature generation unit 16 generates voice features that define phoneme / prosodic information from the device utterance sentence generated by the dialogue control unit 18. This can be used at a symbol level such as a phonetic symbol, or at a parameter level based on a column such as a phoneme number, a duration time, and a pitch frequency. Since this process is not different from that used in the existing text-to-speech synthesis technology, a detailed description is omitted here.

The voice synthesizing unit 13 generates a voice waveform from voice characteristics. Since this process is not different from that used in the existing rule speech synthesis technology, detailed description is omitted here.

The device controller 19 controls the device 2 based on the device control commands and parameters selected by the dialog controller 18.
Send a control signal to 0. The status signal from the device 20 is received and sent to the dialogue control unit 18.

The user model storage unit 14 stores unique information for each user. In the present embodiment, a vocabulary list and a workflow are stored in addition to the voice feature dictionary. When operating a device having a similar purpose or function, it is considered that the user understands the operation method using the same metaphor. The data summarizing the behavior of the user with respect to the device group understood by the same metaphor is the user model referred to here, and in the present embodiment, includes a vocabulary list and a workflow.

FIG. 5 is a flowchart for explaining the process of guiding / predicting a user's operation and utterance in the dialog control unit 18 of FIG. The dialog control unit 18 guides and predicts user operations and utterances according to the workflow and the vocabulary list. The guidance is preferably performed by a guidance message in which the phonemes and prosody are changed and emphasized as usual (for example, a high voice, a loud voice), and the prediction result is set as a speech recognition candidate. The dialog control unit 18 sequentially processes operation scenes in the workflow. First, a user model based on device classification is selected (step S1). If the target device 20 is unfamiliar to the user (step S1
4) A guidance message corresponding to the operation scene is generated (step S15). At this time, words used in the operation scene are selected from the vocabulary list (step S12), and these words are preferentially set as speech recognition candidates (step S13). In step S14, if the target device 20 is a device that is familiar to the user, no guidance message is generated. continue,
In step S16, it is determined whether or not the scene is the last operation scene. If the scene is not the last operation scene, the process of the next scene (step S17) is performed (return to step S12). If it is the last operation scene in step S16, the process is terminated.

FIG. 6 is a diagram for explaining an example of use of the apparatus control device of FIG. 4, and is a diagram for explaining an example in which a user accustomed to the operation of a copying machine operates a facsimile for the first time. It can be seen from the device information that the device classification of the target device 20 is a hard copy system. Therefore, the user model storage unit 1
4 to obtain a hard copy workflow. This is analyzed and recorded in the past by operating the same hard copy type copying machine. The operation scene sequence of this workflow is condition setting, document setting, test, and execution. Commands used in each case,
The correspondence between words and priorities is obtained from the vocabulary list, and guidance messages are generated and speech recognition candidates are set. Since the priority is a degree specified in the past of the parameter or the command, the setting is performed in consideration of this. Also, where
The device information indicates that the orientation of the document set is downward as a special note. Since this is different from the information of the user model, the guidance message emphasizes this.

In this way, a facsimile to be operated for the first time can be operated by the same metaphor operation as the copying machine. For parameters not specified at the end of a scene, default values are calculated and used. The default value is calculated from the history of the value specified in the past.

The history storage unit 22 temporarily stores the history of a series of operations and conversations. User model analysis unit 2
In step 1, the user model is updated based on the data in the history storage unit 22. In the present embodiment, the user model includes a workflow and a vocabulary list, but the present invention is not limited to this configuration. These two can be used independently, and a configuration including only one of them is also possible. First, the workflow will be described. The user model analysis unit 21 divides the detailed operation history sequence stored in the history storage unit 22 for each operation scene and outputs the result as a workflow. For example, continuous key input is put together as a condition setting scene. Here, consider a typical workflow in a copying machine. A sequence is conceivable in which a document is set, conditions such as density, magnification, and number of sheets are set, and a start button is pressed. However, on the other hand, a sequence in which a document is set after setting the conditions and the start button is pressed may be considered. In addition, if the user is careful, a sequence of taking one test copy first, setting the number of copies, and taking a real copy may be considered. In general, a workflow selected by the user is specified from a plurality of workflows. If an operation is performed in a different flow on the device 20 for which workflow data already exists,
Although the result of the prediction by the dialog control unit 18 is incorrect, the workflow is updated here with the operation history actually taken.

Next, the vocabulary list will be described below.
Although there are generally a plurality of words used to express one command or parameter, the words used by the user are almost limited. Therefore, the words used by the user are checked based on the conversation history data in the history storage unit 22, and the vocabulary list is updated according to the user. Not all parameters are specified. The priority is increased for items specified in the operation, and the priority is reduced for items not specified. The value of the designated parameter is recorded as data for calculating a default value.

In the present embodiment, not only the explanation and the change of the keyword but also all the processes are displayed on the display provided in the mobile telephone terminal and operated by using the buttons. It is also possible to adopt a configuration in which processing by voice and processing by display and buttons are used together.

Although some embodiments of the present invention have been described as device control devices, the present invention is not limited to this, and a program in which a program to function as these device control devices is incorporated in a mobile telephone terminal. But it can be implemented.

[0087]

According to the invention of claims 1, 16, 31, and 32, various device functions are transmitted without increasing the load on the user by transmitting device control commands through dialogue through the mobile telephone terminal. Will be available.

According to the inventions of claims 2, 17, 31, and 32, various functions of the device can be used without increasing the load on the user by transmitting the device control command by voice conversation through the mobile telephone terminal. It becomes possible,
The voice recognition rate is improved by performing voice recognition specialized for individuals even for an unspecified number of users.

According to the invention of claims 3, 18, 31, and 32, by simultaneously recognizing a plurality of keywords,
The operation instruction including the parameter setting which tends to be complicated can be easily used.

According to the inventions of claims 4, 19, 31, and 32, even a complicated function that cannot be easily understood only by character information with a limited display area can be easily understood by the user by explanation by voice.

According to the inventions of claims 5, 20, 31, and 32, the user changes the correspondence between the command and the keyword, thereby constructing an operation system capable of storing the information without difficulty for each user, and reducing the storage load. .

According to the inventions of claims 6, 21, 31, and 32, the utterance history and the command history of each user are stored in the mobile telephone terminal, thereby estimating the context of the user and reducing the ambiguity of the user utterance. Can be resolved and converted to an appropriate device operation command.

According to the seventh, eighth, twenty-second, twenty-third, and thirty-second aspects of the present invention, the interactive processing is performed based on a user model of a device that has been used and is similar to the device to be operated. Operation can be realized.

According to the ninth, twenty-fourth, thirty-second, and thirty-second inventions, a vocabulary used in the past for a device similar to the operation target device is used to generate a message that can be understood only by known linguistic knowledge. it can.

According to the tenth, twenty-fifth, thirty-first, and thirty-second inventions, the vocabulary used in the past is used for a device similar to the operation target device, thereby narrowing down the recognition target to a vocabulary that is easy for the user to speak. Is possible, which leads to an improvement in the speech recognition rate.

According to the invention of claims 11, 26, 31, and 32, a smooth operation can be expected by outputting a guide message according to a workflow assembled for devices similar to the device to be operated.

According to the invention of claims 12, 27, 31, and 32, it is possible to narrow down recognition targets by predicting the order of speech in accordance with a workflow assembled for devices similar to the operation target device. This leads to an improvement in the speech recognition rate.

According to the inventions of claims 13, 28, 31, and 32, an operation with few errors can be expected by outputting a message that is not standard but is detailed and emphasizes a function specific to the operation target device. I can do it.

According to the invention of claims 14, 29, 31, and 32, for a setting item whose setting value is always stable, a workflow is created with the setting value as a default value, thereby reducing wasteful and efficient. Operation can be expected.

According to the invention of claims 15, 30, 31, and 32, for a setting item whose change from the default value is small,
By creating a workflow by lowering the priority of the setting process, it is possible to operate the setting items that the user values more importantly with more time.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a device control device according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of device-specific information according to an embodiment of the present invention.

FIG. 3 is a flowchart illustrating a process of executing a command from a speech recognition result obtained by a speech recognition unit in FIG. 1;

FIG. 4 is a block diagram illustrating a configuration of a device control device according to another embodiment of the present invention.

FIG. 5 is a flowchart illustrating a process of guiding / predicting a user operation and an utterance in the dialogue control unit of FIG. 4;

6 is a diagram for explaining an example of use of the device control device of FIG. 4, and is a diagram for explaining an example in which a user accustomed to operating a copying machine first operates a facsimile.

[Explanation of symbols]

1, 11: operation unit, 2, 12: voice feature extraction unit, 3, 1
3: Voice synthesis unit, 4: User information storage unit, 5, 15: Voice recognition unit, 6, 16: Voice feature generation unit, 7, 17: Device information storage unit, 8, 18: Dialogue control unit, 9, 19 ... Device control unit, 10, 20 ... Device, 14 ... User model storage unit
21: User model analysis unit, 22: History storage unit.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (reference) G10L 3/00 571V F term (reference) 5D015 AA04 BB01 KK01 KK04 LL02 LL06 LL10 5K027 AA11 BB02 FF22 FF28 HH26 MM16 5K048 AA04 AA14 BA01 DB04 EB02 FB10 FB12 FB15 9A001 DD11 HH16 HH17 HH18 HH34 JJ12

Claims (32)

[Claims] 1. A device control unit for controlling one or a plurality of devices, and a mobile phone terminal for each user, which serves as an interface for the user, wherein a command indicating an operation instruction to the device and an operation setting of the device are provided. Required mobile phone terminal, storing the relationship between the command and the parameter, notifying the device control unit of the command selected by the user by the mobile phone terminal, and controlling the device by the device control unit. An apparatus control apparatus that uses the input control unit, an input recognition unit that recognizes an input including a keyword, an output synthesis unit that sends an output message to a user, and restricts a recognition candidate of the input recognition unit from status information of the device. A device control command is generated from the recognition result of the input recognition unit, and an output message is generated in response to an information request from the device. Dialogue control means for generating a message, and device control means for controlling the device by the device control command generated from the dialogue control device and sending status information of the device to the dialogue control device. Equipment control device using a mobile telephone terminal. 2. A device control unit for controlling one or a plurality of devices, and a mobile telephone terminal for each user, which serves as an interface for the user, wherein a command indicating an operation instruction to the device and an operation setting of the device are provided. Required mobile phone terminal, storing the relationship between the command and the parameter, notifying the device control unit of the command selected by the user by the mobile phone terminal, and controlling the device by the device control unit. A device control device that uses a voice recognition unit that recognizes a user utterance including a keyword, a voice synthesis unit that sends a voice output message to a user, and limits recognition candidates of the voice recognition unit from status information of the device. Then, a device control command is generated from the recognition result of the voice recognition unit, and the device control command is generated by an information request from the device. Dialogue control means for generating an output message, and device control means for controlling the device by a device control command generated from the dialogue control device and sending status information of the device to the dialogue control device. An equipment control device using a mobile phone terminal. 3. A plurality of keywords recognized by the voice recognition unit are simultaneously recognized. By recognizing a plurality of keywords, an operation instruction combining a plurality of the commands,
3. The device control apparatus using a mobile telephone terminal according to claim 2, wherein a plurality of said parameters can be set simultaneously. 4. The mobile telephone terminal holds, in addition to the keyword, explanation information on an operation of a device corresponding to the keyword, and stores the held explanation information by voice or visual display according to a user's request. 3. The device control device using a mobile telephone terminal according to claim 2, wherein the device control device is provided. 5. The device control device using a mobile telephone terminal according to claim 2, wherein the correspondence between the keyword and the command can be changed by operating the mobile telephone terminal. 6. The dialogue control means holds a history of a user utterance, a history of a device utterance, and a command history up to the present time, and uses the held history in addition to status information of the device, thereby 3. The device control device using a mobile telephone terminal according to claim 2, wherein recognition candidates in the voice recognition means are limited. 7. A device using a mobile telephone terminal, comprising: a device control unit for controlling one or a plurality of devices; and a mobile telephone terminal for each user, which controls an interface of the user. A control device, comprising: history storage means for storing an operation history and a dialogue history; and user model analysis means for analyzing data stored in the history storage means and outputting at least one of a vocabulary list and a workflow. And a device classifying means for classifying the used device into a device class having a typical function group according to the purpose, work content, and the like; and storing an analysis result of the user model analyzing means in association with a classification result of the device classifying means. User model storage means; and dialogue processing means for performing dialogue processing using data stored in the user model storage means. Device control device using the mobile telephone terminal, characterized in. 8. A mobile telephone terminal, comprising: a device control unit for controlling one or a plurality of devices; and a mobile telephone terminal for each user, which controls the user by voice dialogue. A device control apparatus, comprising: a history storage unit for storing an operation history and a dialog history; and a user model analysis for analyzing data stored in the history storage unit and outputting at least one of a vocabulary list and a workflow. Means, a device classifying means for classifying the used device into a device class having a typical function group according to purpose, work content, and the like; and storing an analysis result of the user model analyzing unit in association with a classification result of the device classifying unit. User model storage means for performing an interactive process using data stored in the user model storage means. Device control device using the mobile telephone terminal, characterized in that. 9. The device control device using a mobile telephone terminal according to claim 8, wherein a voice output message is generated using a vocabulary list associated with the device classification to which the device to be operated belongs. 10. The device control apparatus using a mobile telephone terminal according to claim 8, wherein a speech recognition candidate is generated using a vocabulary list associated with the device classification to which the device to be operated belongs. 11. The device control device using a mobile telephone terminal according to claim 8, wherein a voice output message is created using a workflow associated with the device classification to which the device to be operated belongs. 12. The device control apparatus using a mobile telephone terminal according to claim 8, wherein a speech recognition candidate is generated using a workflow associated with the device classification to which the device to be operated belongs. 13. A detailed voice output message is generated for a function not included in a function group associated with the device classification to which an operation target device belongs, and an enhanced synthesized voice is generated. An apparatus control device using the mobile telephone terminal according to claim 8. 14. The workflow according to claim 1, wherein the setting values set by the target user with respect to the setting items to be set by the user when using the device are small and stable even when using a different device. 9. The device control device using a mobile telephone terminal according to claim 8, wherein the setting value of the setting item is created as a default value. 15. The apparatus using a mobile telephone terminal according to claim 8, wherein the workflow is created by lowering the priority of the setting process for a setting item whose change from a default value is small. Control device. 16. A device control unit for controlling one or a plurality of devices, and a mobile telephone terminal for each user, which serves as an interface for the user, wherein a command indicating an operation instruction to the device and an operation setting of the device are provided. Required mobile phone terminal, storing the relationship between the command and the parameter, notifying the device control unit of the command selected by the user by the mobile phone terminal, and controlling the device by the device control unit. An apparatus control method using: an input recognition step of recognizing an input including a keyword; an output synthesis step of sending an output message to a user; and restricting recognition candidates of the input recognition step from status information of the apparatus, A device control command is generated from the recognition result of the input recognition step, and information from the device is generated. A dialogue control step of generating an output message in response to a request; and a device control step of controlling the device by the device control command generated in the dialogue control step and sending status information of the device to the dialogue control step. A device control method using a mobile telephone terminal, characterized by comprising: 17. A device control unit for controlling one or a plurality of devices, and a mobile phone terminal for each user, which serves as an interface for the user, wherein a command indicating an operation instruction to the device and an operation setting of the device are provided. Required mobile phone terminal, storing the relationship between the command and the parameter, notifying the device control unit of the command selected by the user by the mobile phone terminal, and controlling the device by the device control unit. A method of controlling a device that uses a voice recognition step of recognizing a user utterance including a keyword, a voice synthesis step of sending a voice output message to a user, and limiting recognition candidates in the voice recognition step from status information of the device. Generating a device control command from the recognition result of the voice recognition step; A dialogue control step of generating an output message in response to an information request from the device, and a device control step of controlling the device by a device control command generated in the dialogue control step and sending status information of the device to the dialogue control step; A device control method using a mobile telephone terminal. 18. The method according to claim 18, wherein a plurality of keywords are recognized at the same time in the voice recognition step, and by recognizing the plurality of keywords, an operation instruction combining a plurality of the commands and a plurality of the parameters can be simultaneously set. An apparatus control method using the mobile telephone terminal according to claim 17. 19. The mobile phone terminal holds, in addition to the keyword, explanation information on an operation of a device corresponding to the keyword, and stores the held explanation information by voice or visual display at the request of the user. 18. The device control method using a mobile telephone terminal according to claim 17, wherein the device control method is provided. 20. The apparatus control method using a mobile telephone terminal according to claim 17, wherein the correspondence between the keyword and the command can be changed by operating the mobile telephone terminal. 21. The dialogue control step holds the history of the user utterance, the history of the device utterance, and the command history up to the present time, and uses the held history in addition to the status information of the device. 18. The apparatus control method using a mobile telephone terminal according to claim 17, wherein recognition candidates in the voice recognition step are limited. 22. A device using a mobile telephone terminal, comprising: a device control unit for controlling one or a plurality of devices; and a mobile telephone terminal for each user, which controls the user interface. A control method, comprising: a history storing step of storing an operation history and a dialog history; and a user model analyzing step of analyzing data stored in the history storing step and outputting at least one of a vocabulary list and a workflow. A device classifying step of classifying the used device into a device class having a typical function group according to the purpose, work content, and the like; and storing the analysis result in the user model analysis step in association with the classification result in the device classification step. User model storage step to perform, and interactive processing using the data stored in the user model storage step. Interactive processing steps and, a comprise device control method using a mobile telephone terminal, characterized by comprising performing. 23. A mobile telephone terminal, comprising a device control unit for controlling one or a plurality of devices and a mobile telephone terminal for each user, which controls said user by voice dialogue. A device control method, comprising: a history storage step of storing an operation history and a dialog history; and a user model analysis step of analyzing data stored in the history storage step and outputting at least one of a vocabulary list and a workflow. And a device classifying step of classifying the used device into a device class having a typical function group according to the purpose, work content, etc. A user model storage step for storing, and a pair using the data stored in the user model storage step. Equipment control method using a mobile telephone terminal, characterized in that it comprises a dialogue processing step of performing processing, the. 24. The device control method using a mobile telephone terminal according to claim 23, wherein a voice output message is generated using a vocabulary list associated with the device classification to which the device to be operated belongs. 25. The device control method using a mobile telephone terminal according to claim 23, wherein a speech recognition candidate is generated using a vocabulary list associated with the device classification to which the device to be operated belongs. 26. The device control method using a mobile telephone terminal according to claim 23, wherein a voice output message is created using a workflow associated with the device classification to which the operation target device belongs. 27. The device control method using a mobile telephone terminal according to claim 23, wherein a voice recognition candidate is generated using a workflow associated with the device classification to which the device to be operated belongs. 28. A function that generates a detailed voice output message for a function that is not included in the function group associated with the device classification to which the operation target device belongs, and generates an enhanced synthesized voice. 23. A device control method using the mobile telephone terminal according to 23. 29. The workflow according to claim 1, wherein the setting value set by the target user with respect to a setting item to be set by the user when using the device is small and stable even when using a different device. 24. The method according to claim 23, wherein the setting value of the setting item is created as a default value.
An apparatus control method using the mobile phone terminal described in the above. 30. The apparatus using a mobile telephone terminal according to claim 23, wherein the workflow is created by lowering the priority of the setting process for a setting item whose change from a default value is small. Control method. 31. A computer-readable recording medium in which a program for causing the mobile telephone terminal according to claim 1 to function as a device control device is recorded. 32. A computer-readable recording medium on which a program for implementing the device control method using the mobile telephone terminal according to claim 16 is recorded.