JP4722010B2 - Information processing apparatus, information processing method, information processing program, and computer-readable recording medium recording the information processing program - Google Patents

Description

  The present invention relates to an information processing device, an information processing method, an information processing program, and a computer-readable recording medium on which the information processing program is recorded, and in particular, an information processing device, an information processing method, and an information processing that are restricted by a graphical user interface The present invention relates to a program and a computer-readable recording medium on which an information processing program is recorded.

  Conventionally, when an image taken by a mobile phone camera is transmitted to a friend by the mobile phone, the following procedure is performed. The first procedure is to start up photographing application software installed on the mobile phone. The second procedure is to take an image. The third procedure is to start application software for creating mail. The fourth procedure is to create a mail. The fifth procedure is to start application software for address display. The sixth procedure is to input the other party's mail address. The seventh procedure is to confirm that the photographed image is attached to the mail. The eighth procedure is to send mail using application software for creating mail. That is, unless a plurality of application software is started, the user cannot even transmit an image immediately after shooting.

  One method for easily executing a plurality of operations is to use a so-called “macro”. However, macros have the following disadvantages: The first drawback is that it can only be used inside certain application software. The second drawback is that it is necessary to determine the contents to be recorded in advance and perform the operation as it is, or to edit with an editor.

  Patent Document 1 discloses a computer operation support apparatus that supports the operation of a computer. The computer operation support device disclosed in Patent Literature 1 specifies an operation history recording processing unit that records the operation contents performed on a computer as an operation history, and information indicating a position where user input is performed in the operation history An operation history editing processing unit that edits the operation history, and inputs the operation history to the computer to reproduce the operation history, and accepts input from the user when reproducing the operation history at the specified position. And an operation history reproduction processing unit for reproducing the operation contents of the operation history according to the received user input contents and supporting the operation of the computer.

According to the invention disclosed in Patent Document 1, a user who has not mastered how to use a computer can use the computer by referring to the operation contents of other users.
JP 2003-29891 A

  For example, a computer operation support apparatus such as that disclosed in Patent Document 1 is known in terms of presenting operations performed by the user in the past. However, since the operation support device shown in Patent Document 1 is intended to educate people who are unfamiliar with the operation, the user may even send an image immediately after shooting unless a plurality of application software is started. Can not. The same can be said for other various operations. In other words, in the invention disclosed in Patent Document 1, in order to perform an operation using a plurality of applications, the user must perform operations for starting and ending the applications one by one. There is a problem.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an information processing apparatus, an information processing method, an information processing program, and information that can simplify a complicated operation using a plurality of applications. An object of the present invention is to provide a computer-readable recording medium in which a processing program is recorded.

  In order to achieve the above object, according to an aspect of the present invention, an information processing apparatus includes a storage unit, a display unit, an input unit, and a control unit. The storage means stores first application software, second application software, and a plurality of types of information each representing an operation realized by at least one of the first application software and the second application software. The first application software is application software for generating data. The second application software is application software for processing data. The display means displays an image. The input means inputs information corresponding to any of a plurality of types of information. The control means operates in accordance with the first application software and the second application software, and controls the display means. The control means includes operation means, first generation means, second generation means, third generation means, storage control means, and display control means. The operation means operates according to at least one of the first application software and the second application software so as to correspond to the information input to the input means. The first generation unit generates information representing data generated by the operation unit when the operation unit operates according to the first application software. The second generating means generates information representing the content of the processing for the data when the operating means processes the data according to the second application software. The third generation unit generates information representing the operation performed by the operation unit based on the information generated by the first generation unit and the information generated by the second generation unit. The storage control unit controls the storage unit so as to store the information generated by the third generation unit as one type of a plurality of types of information. The display control means controls the display means so as to display images corresponding to a plurality of types of information.

  In addition, it is preferable that the third generation unit described above includes a combination unit for combining the information generated by the first generation unit and the information generated by the second generation unit.

  Alternatively, it is desirable that the combination means described above includes a first control means and a second control means. The first control unit controls the storage unit so as to store the information generated by the first generation unit. The second control unit controls the storage unit to store the information generated by the second generation unit in association with the information generated by the first generation unit.

  In addition to the first application software, the second application software, and a plurality of types of information, the storage means described above includes character string information corresponding to data and character string information representing the contents of processing. It is desirable to memorize. In addition, the display control means controls the display means so as to display images corresponding to the character string information corresponding to the data, the character string information indicating the contents of the processing, and the plurality of types of information. It is desirable to include means for

  Alternatively, the display control unit described above preferably includes a combining unit and a unit for controlling the display unit. The synthesizing unit synthesizes information on a character string having a character string corresponding to the data and a character string representing the content of the processing based on a combination of the information representing the data and the information representing the content of the processing. The means for controlling the display means controls the display means so as to display an image representing the information synthesized by the synthesizing means.

  Alternatively, the character string corresponding to the data described above preferably includes a character string representing a word corresponding to the data.

According to another aspect of the present invention, an information processing method includes a first generation step, a second generation step, a third generation step, a storage step, a display step, an input step, and an operation step. including. The first generation step generates information representing data when operating according to the first application software. The first application software is application software for generating data. The second generation step generates information representing the content of the processing for the data when operating according to the second application software. The second application software is application software for processing data. In the third generation step, information representing an operation is generated based on the information generated in the first generation step and the information generated in the second generation step. In the storage step, the information generated in the third generation step is stored as one of a plurality of types of information each representing an operation. The display step displays images corresponding to a plurality of types of information. In the input step, information corresponding to any of a plurality of types of information is input. The operation step operates according to at least one of the first application software and the second application software so as to correspond to the information input in the input step.

  According to another aspect of the present invention, an information processing program includes a first generation step, a second generation step, a third generation step, a storage step, a display step, an input step, and an operation step. The computer is caused to execute each step including The first generation step generates information representing data when operating according to the first application software. The first application software is application software for generating data. The second generation step generates information representing the content of the processing for the data when operating according to the second application software. The second application software is application software for processing data. In the third generation step, information representing an operation is generated based on the information generated in the first generation step and the information generated in the second generation step. In the storage step, the information generated in the third generation step is stored as one of a plurality of types of information each representing an operation. The display step displays images corresponding to a plurality of types of information. In the input step, information corresponding to any of a plurality of types of information is input. The operation step operates according to at least one of the first application software and the second application software so as to correspond to the information input in the input step.

  According to another aspect of the present invention, the recording medium is a computer-readable recording medium on which an information processing program is recorded. The information processing program causes a computer to execute each step including a first generation step, a second generation step, a third generation step, a storage step, a display step, an input step, and an operation step. . The first generation step generates information representing data when operating according to the first application software. The first application software is application software for generating data. The second generation step generates information representing the content of the processing for the data when operating according to the second application software. The second application software is application software for processing data. In the third generation step, information representing an operation is generated based on the information generated in the first generation step and the information generated in the second generation step. In the storage step, the information generated in the third generation step is stored as one of a plurality of types of information each representing an operation. The display step displays images corresponding to a plurality of types of information. In the input step, information corresponding to any of a plurality of types of information is input. The operation step operates according to at least one of the first application software and the second application software so as to correspond to the information input in the input step.

  An information processing apparatus, an information processing method, an information processing program, and a computer-readable recording medium recording the information processing program according to the present invention can simplify a complicated operation using a plurality of applications.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<First embodiment>
The information communication device according to the first embodiment of the present invention will be described below.

With reference to FIG. 1 and FIG. 2, a mobile phone 100 which is an aspect of the mobile communication terminal according to the present invention will be described. FIG. 1 is a front view of the folding mobile phone 100 with the housing opened. FIG. 2 is a diagram showing the cellular phone 100 in a state where the housing is opened from the back side.

  As shown in FIG. 1, the mobile phone 100 includes a first housing 104, a second housing 106, a speaker 174, a main display 150, a sub camera 142, a main operation unit 120, and a microphone 172. Including. The main operation unit 120 includes a cursor movement key 122 and a 12 key 124.

  As shown in FIG. 2, the mobile phone 100 includes a sub display 160, a sub operation unit 130, a main camera 140, and an LED (Light Emitting Diode) 176 in the first housing 104. In addition, the mobile phone 100 includes an antenna 108 in the second housing 106. The sub operation unit 130 includes cursor movement keys 1302 and 1306 displayed on the sub display 160 and an enter key 1304.

  With reference to FIG. 3, the configuration of mobile phone 100 according to the present embodiment will be further described. FIG. 3 is a block diagram showing the configuration of the mobile phone 100.

  As shown in FIG. 3, in addition to the configuration shown in FIGS. 1 and 2, the mobile phone 100 includes a communication unit 102, a control unit 110, a flash memory 144, a RAM (Random Access Memory) 146, data ROM (Read Only Memory) 148, an audio signal processing unit 170, a data communication IF (Interface) 178, a memory card driving unit 180, and a vibrator 184 are included. A memory card 182 is attached to the memory card drive unit 180.

  When the antenna 108 receives a signal, the signal is input to the communication unit 102. The communication unit 102 executes a predetermined process on the input signal and outputs it. A signal output from the communication unit 102 is input to the control unit 110. On the other hand, a signal for transmission output from the control unit 110 is input to the communication unit 102. When the communication unit 102 executes a predetermined transmission process and outputs it, the output signal is wirelessly transmitted via the antenna 108.

  As described above, the main operation unit 120 includes, for example, a cursor movement key 122, a 12 key 124, and other buttons. When the user inputs an instruction via the main operation unit 120, a signal corresponding to the instruction is output from the main operation unit 120 and input to the control unit 110. Control unit 110 performs a predetermined process on the input signal. Note that the user instruction includes, for example, an instruction to move the cursor, an instruction to confirm and execute selection of an item by the cursor, and other instructions.

  In addition, main operation unit 120 accepts information representing an instruction for executing a specific process from among a plurality of information displayed on main display 150. In addition, the main operation unit 120 receives an instruction to confirm the selection for the selected information. Thereby, the process selected by the user is specified.

  As described above, the sub operation unit 130 includes, for example, a key for moving the cursor among keys of the main operation unit 120 or a key for selecting and executing a menu designated by the user. When the user inputs an instruction via the sub operation unit 130, a signal corresponding to the instruction is output from the sub operation unit 130 and input to the control unit 110. Control unit 110 executes a predetermined process based on the signal. When both the main operation unit 120 and the sub operation unit 130 are operated when the mobile phone 100 is in the opened state, one of the predetermined operation units (for example, the main operation unit 120) is determined. May be preferentially executed. Thereby, it is possible to prevent erroneous operation of the mobile phone 100 even when mutually exclusive instructions are input.

  The main camera 140 is, for example, a solid-state image sensor CCD (Charge Coupled Device). The main camera 140 images a subject and generates and outputs a video signal of the subject. The output signal is input to the control unit 110. Control unit 110 executes a predetermined process on the signal and outputs the signal to main display 150 or sub display 160. This process includes a process of generating image display data in accordance with the size of the image display area of the main display 150 or the sub display 160, for example.

The sub camera 142 is, for example, a solid-state image sensor CCD. The user moves the sub camera 142
When the subject is photographed via the video signal, a video signal is output from the sub camera 142, and the signal is input to the control unit 110. The control unit 110 executes predetermined processing on the input signal and outputs it to the main display 150 or the sub display 160. This process includes a process of generating image display data in accordance with the size of the image display area of the main display 150 or the sub display 160, for example.

  When the user speaks to the microphone 172, an audio signal corresponding to the speech is output from the microphone 172. The output signal is input to the audio signal processing unit 170. The audio signal processing unit 170 executes predetermined signal processing and outputs the signal processing to the control unit 110. This processing includes, for example, adjustment of the output level of the audio signal. The control unit 110 outputs this signal to the communication unit 102 and wirelessly transmits the signal via the antenna 108. Alternatively, control unit 110 outputs the signal to data communication IF 178.

  The audio signal received by the mobile phone 100 via the antenna 108 is input to the audio signal processing unit 170 via the communication unit 102 and the control unit 110. The audio signal processing unit 170 converts the signal based on a predetermined process and outputs it to the speaker 174.

  The flash memory 144 stores data for executing each process. The data stored in the flash memory 144 includes a queue database, a word database, which will be described later, a database indicating which data among the data constituting the operation history queue is associated with which application software, Data for displaying a menu by main display 150 and a database representing a relationship between an operation represented by the menu and a signal output from main operation unit 120 or sub operation unit 130 are included.

  The RAM 146 temporarily stores processing data executed by the control unit 110. The processing data includes data representing the operation history queue being generated. The control unit 110 reads data stored in the RAM 146 and executes a predetermined process. This processing includes generation of display data for displaying an image.

  The data ROM 148 stores application software (for example, a mail editing program, an image processing program, an audio reproduction program, and other programs) that the mobile phone 100 has. These programs are read by the control unit 110, and after predetermined processing is executed, data corresponding to each processing is written in the RAM 146. The main display 150 or the sub display 160 displays an image based on the data.

  The main display 150 is a liquid crystal display, for example, but may be another display device. The main display 150 displays an image based on the data read by the control unit 110.

  Similarly, the sub display 160 is a liquid crystal display, for example, but may be another display device. The sub display 160 displays an image based on the data read by the control unit 110. Here, the images displayed on the main display 150 and the sub display 160 may be the same, or different images may be displayed. For example, an image of a subject photographed by the main camera 140 or the sub camera 142 is displayed on one of the displays, and data stored in the flash memory 144 or other storage device (for example, contents of received mail) is displayed on the other display. ) May be displayed.

  The LED 176 is an embodiment of a light emitting element. The LED 176 emits light based on a predetermined emission color when it detects an incoming call or mail reception of the mobile phone 100. In this case, the LED 176 is not limited to a specific color, and may include a plurality of colors. Further, the light emitting element is not limited to the LED 176, and may be another aspect.

  The data communication IF 178 is an interface for data communication via, for example, a communication cable (not shown). The mode of this interface is not particularly limited.

Vibrator 184 is an oscillating device for notifying incoming calls, mails received and other conditions. When reception of a telephone or mail is detected, control unit 110 oscillates vibrator 184 based on a predetermined vibration frequency. The mode of oscillation may be switched according to the caller or mail sender. Further, the oscillation device is not limited to the vibrator 184.

  The memory card driving unit 180 is for driving a memory card 182 capable of reading and writing removable data. In other words, when the memory card 182 is inserted into the memory card driving unit 180, the data stored in the memory card 182 is read out and temporarily written in the RAM 146. As described above, by providing a data recording medium that can be attached to and detached from the mobile phone 100, data set by the user and other data stored in each storage device of the mobile phone 100 are transferred to other mobile communication terminals. be able to.

  FIG. 4 is a functional block diagram of the control unit 110. In the case of the present embodiment, each function of the control unit 110 shown in FIG. 4 is realized by the control unit 110 executing a program.

  Referring to FIG. 4, the functions that control unit 110 takes charge of include operation unit 1100, first generation unit 1102, second generation unit 1104, third generation unit 1106, storage control unit 1108, and display control. Part 1110. The operation unit 1100 represents a function that operates according to at least one of the first application software and the second application software so as to correspond to information input to the main operation unit 120 and the sub operation unit 130. In the present embodiment, “first application software” means application software for generating data to be used by a user. “Second application software” means application software for processing data to be used by a user. Accordingly, the flash memory 144, the RAM 146, and the data ROM 148 store a plurality of types of first application software and a plurality of types of second application software. The first generation unit 1102 represents a function of generating information representing data generated by the operation unit 1100 when the operation unit 1100 operates according to the first application software. The second generation unit 1104 represents a function of generating information representing the content of processing on data when the operation unit 1100 processes data according to the second application software. Examples of “information representing the contents of processing on data” include information representing an object and information representing an action. The third generation unit 1106 represents a function of generating information representing the operation performed by the operation unit 1100 based on the information generated by the first generation unit 1102 and the information generated by the second generation unit 1104. The storage control unit 1108 represents a function of controlling the RAM 146 so as to store the information generated by the third generation unit 1106 as one of a plurality of types of information each representing an operation. The display control unit 1110 controls the main display 150 and the sub-display 160 so as to display a plurality of types of information based on the character string information corresponding to the data and the character string information indicating the contents of the processing. Represents the function to perform.

  The third generation unit 1106 includes a first control unit 1120 and a second control unit 1122. The first control unit 1120 represents a function of controlling the RAM 146 so as to store the information generated by the first generation unit 1102. The second control unit 1122 represents a function of controlling the RAM 146 so as to store the information generated by the second generation unit 1104 in association with the information generated by the first generation unit 1102.

  The display control unit 1110 includes a synthesis unit 1224 and an image control unit 1226. Combining unit 1224 represents a function of combining character string images based on a combination of information representing data generated by operation unit 1100 and information representing the contents of processing on the data. This character string is a character string having a character string corresponding to data and a character string representing the contents of processing. The image control unit 1226 represents a function of controlling the main display 150 and the sub display 160 so as to display the image synthesized by the synthesis unit 1224.

As described above, the function of mobile phone 100 according to the present embodiment is realized by software executed by each hardware and control unit 110. Such software may be stored in advance in a storage device such as the flash memory 144, the RAM 146, and the data ROM 148. Alternatively, the information may be stored and distributed in a memory card 182 or other removable information recording medium, read from the information recording medium by the memory card driving device 180, and temporarily stored in the flash memory 144. The software is
The data is read from the flash memory 144 to the RAM 146 and executed by the control unit 110. The hardware configuration of the mobile phone 100 shown in FIG. 3 corresponds to a computer device within the above-described configuration and the operation thereof, and is general within this range. Therefore, it can be said that the most essential part of the present invention is software recorded in the flash memory 144, the RAM 146, the data ROM 148, the memory card 182 and other storage devices. Since the operation of each hardware of this computer device is well known, the description thereof will not be repeated here.

  Note that the mobile phone 100 may not have a foldable configuration. The mobile phone 100 does not necessarily have a plurality of display devices. Further, the mobile phone 100 does not necessarily have the memory card driving device 180. Further, the configuration of the storage device is not limited to the configuration shown in FIG. 3, and the mobile phone 100 only needs to have at least one storage device. Further, each function that the control unit 110 takes charge of may be realized by an IC or other hardware.

  FIG. 5 is a diagram showing the configuration of the queue database stored in the flash memory 144. The queue database includes a plurality of operation history queues. In the present embodiment, the operation history queue is data representing processing executed by the control unit 110. Referring to FIG. 5, in the case of the present embodiment, operation history queue includes data representing a subject as a code, data representing an object as a code, and data representing an operation as a code.

  FIG. 6 is a diagram showing the configuration of the word database stored in the flash memory 144. The data included in the word database represents the meaning of the data included in the queue database. Referring to FIG. 6, in the case of the present embodiment, the word database includes data representing codes, data representing information associated with the codes, and data that can be directly used by control unit 110 when executing the application. , Data representing a character string to be displayed instead of the code on the main display 150 or the sub display 160. For example, a code “001” is associated with information representing a mail address “mistera@abc.def.ne.jp”. The control unit 110 uses this data as it is as data representing the subject of the mail. When the operation history queue includes a code “001”, the main display 150 or the sub display 160 displays a character string “Mr. A”. The code “010” is associated with information representing an address “& H00FFAA00”. “& H00FFAA00” means an address in the RAM 146 where still image data is stored. The control unit 110 uses the address “& H00FFAA00” as it is to read out still image data. When the code “010” is included in the operation history queue, the main display 150 or the sub display 160 displays a character string “still image data”. The code “111” is associated with information representing the operation “telephone.exe” by a file name. The control unit 110 uses this data as it is as data for selecting an application to be executed. When the code “111” is included in the operation history queue, the main display 150 or the sub display 160 displays a character string “call”. The code “000” is a so-called null code. In the present embodiment, the character strings included in the word database represent information in the same natural language, that is, Japanese.

  Referring to FIG. 7, the program executed by mobile phone 100 performs the following control with respect to generation of the operation history queue.

In step S2000, RAM 146 stores data representing an operation as a code in accordance with the control of first generation unit 1102. The stored data is data corresponding to the contents of the application software to be activated. This data is generated by the first generation unit 1102 based on the data stored in the flash memory 144. This process is executed by the first generation unit 1102 as part of information processing based on application software that is executed in parallel with the application software that is about to start execution and that controls the user interface. When the data is stored, the operation unit 1100 starts executing the application software. In the following description, application software that is being executed by the operation unit 1100 and that is not application software responsible for storing data constituting the operation history queue is referred to as “the application”.

  In step S2002, first generation unit 1102 determines whether or not the application has been activated in conjunction with the termination of other application software, based on the data representing the operation history queue being generated. In the case of this embodiment, one piece of data constituting the operation history queue is associated with each application software. When all the data representing the operation history queue being generated corresponds to the “associated” data, the first generation unit 1102 determines that the application has been started in conjunction with the termination of the other application software. When there is no data representing the operation history queue being generated, and when any of the data representing the operation history queue being generated does not correspond to the “associated” data, the other application software is terminated. The first generation unit 1102 determines that the application is activated without being linked. Which application software is associated with which data among the data constituting the operation history queue is stored in the flash memory 144 as a database representing that fact. Information indicating which application software is activated when the application is activated is stored in the RAM 146. Thus, the first generation unit 1102 can determine whether any data representing the operation history queue being generated corresponds to the “associated” data. If it is determined that the application has been started in conjunction with the termination of the other application software (YES in step S2002), the process proceeds to step S2004. If not (NO in step S2002), the process proceeds to step S2006.

  In step S2004, second generation unit 1104 reads data stored at a predetermined address in RAM 146. This data represents the address of the RAM 146 in which data representing the operation history queue being generated is stored. As a result, the control unit 110 can refer to or update data representing the operation history queue being generated.

  In step S2006, the first generation unit 1102 stores a predetermined value at a predetermined address in the RAM 146. This value represents the first address of the RAM 146 in which data constituting the operation history queue is stored when a new operation history queue is generated. Data stored at addresses subsequent to this address constitutes an operation history queue. As a result, data is stored after this address, so that the operation history queue that has been generated is effectively deleted.

  In step S2008, operation unit 1100 operates in accordance with the contents of the application. At this time, if data is stored in a “predetermined area” of the RAM 146 in step S3005 described later, the operation unit 1100 executes the application using the data. If no data is stored in the area, the operation unit 1100 prompts the user to input data necessary to execute the application while the application is being executed. For example, when the new application is application software for transmitting mail and data representing the mail address of the transmission destination is stored in step S3005, operation unit 1100 stores the data stored in step S3005. To cause the communication unit 102 to send an email. If such data is not stored in step S3005, operation unit 1100 prompts the user to input a destination email address as part of the processing based on the application.

In step S2010, the first generation unit 1102 determines whether the application is application software for generating data to be used by the user, that is, first application software. The first generation unit 1102 determines this based on data representing application software stored in the RAM 146. That is, the first generation unit 1102 determines this based on whether or not the data representing the application software includes data representing a reserved word for generating data to be used by the user. Examples of reserved words for generating data for use by the user include reserved words that display a message “Do you want to register?”. If it is determined that the application is application software for generating data to be used by the user (YES in step S2010), the process proceeds to step S2012. If not (NO in step S2010), the process proceeds to step S2020.

  In step S2012, operation unit 1100 outputs or saves data according to a command represented by the application. When outputting data, the operation unit 1100 stores the data in the RAM 146 and then outputs the stored data.

  In step S2014, control unit 110 executes processing for accepting selection of application software. This processing corresponds to processing from step S3000 to step S3006 described later.

  In step S2016, the operation unit 1100 terminates the processing of the application when there is a user operation for instructing the main operation unit 120 or the sub operation unit 130 to start the next application software. When the processing of the application is completed, the RAM 146 updates data representing the operation with a code in accordance with the control of the operation unit 1100. The updated data corresponds to the contents of the application software that is activated next. Which application software is activated next corresponds to a user operation on the main operation unit 120 or the sub operation unit 130. For example, when the user performs an operation for sending mail, the software that is activated next is application software for sending mail. When the data representing the operation in code is updated, the operation unit 1100 activates application software corresponding to a user operation on the main operation unit 120 or the sub operation unit 130. Thereby, the new said application will be started. When the RAM 146 stores data in step S3005 described later, the operation unit 1100 performs processing using the data when executing the new application. When the RAM 146 does not store data necessary for executing the new application in step S3005, the operation unit 1100 is based on application software that is executed in parallel with the application and controls the user interface. As part of information processing, the user is prompted to input necessary data. For example, if the new application is application software for transmitting mail, if data representing the mail address of the destination is stored in step S3005, operation unit 1100 uses it to communicate with communication unit 102. Send mail. When such data is not stored in step S3005, the operation unit 1100 executes a process for prompting input of a destination mail address as part of the process based on the application.

  In step S2018, second generation unit 1104 adds new data to the data representing the operation history queue being generated. The data to be added is data representing an object by a code. For example, if the data stored in step S2012 is still image data, the code data added in this step is data representing an address where the still image data is stored. The meaning of the code data added in this step can be specified by the word database. When the meaning of the code is not included in the word database, the second generation unit 1104 prompts the user to input information representing the meaning of the code by operating the main operation unit 120. As a specific process for prompting the user, the second generation unit 1104 controls the main display 150 to display a message prompting the user to input the meaning of the code. When the meaning of the code is input, the second generation unit 1104 is configured to associate the input information with the data added to the data representing the operation history queue being generated and store it as a part of the word database. Controls the flash memory 144.

  In S2020, operation unit 1100 executes the last process of the application. In this case, the data representing the operation as a code, which is the data stored in RAM 146 in step S2000 or the data updated in step S2016, remains stored in RAM 146.

In S2022, second generation unit 1104 adds new data to the data representing the operation history queue being generated. The data to be added is data representing the target person with a code and data representing an action with a code. Data representing the subject as a code is stored in advance in a predetermined area of the RAM 146. The data representing the operation as a code is the data stored in the RAM 146 in step S2000 or the data updated in step S2016. The process of storing these data in a predetermined area of the RAM 146 is executed by the second generation unit 1104 as part of information processing based on application software that is executed in parallel with the application and controls the user interface. The second generation unit 1104 copies the data representing the operation history queue being generated to the area of the RAM 146 and adds the data representing the target person and the data representing the action by the code. To do. By adding these data, data representing the operation history queue is completed. The second generation unit 1104 adds the completed data to the queue database stored in the flash memory 144. Thereby, the process based on the said application is complete | finished.

  Referring to FIG. 8, the program executed on mobile phone 100 performs the following control with respect to generation of data for displaying a menu.

  In step S3000, composition unit 1224 searches for an operation history queue that can be displayed as a menu on main display 150 or sub display 160 in accordance with a user operation on main operation unit 120 or sub operation unit 130. The database to be searched is a queue database. The search key is a code representing the target person in the operation history queue. The synthesizing unit 1224 searches the reserved word representing the process of storing the data representing the target person in the predetermined area of the RAM 146 from the data representing the application, and then the address where the retrieved reserved word is stored. Data of an address whose difference is a predetermined value is read. As a result, a code representing the subject is read out. Reading the code representing the subject by this method is based on the premise that the difference between the address where the searched reserved word is stored and the address where the code representing the subject is stored is constant. It goes without saying that the method for reading the code representing the subject is not limited to this method. The synthesizing unit 1224 identifies an operation history queue whose code representing the subject matches the search key in the queue database by searching. When such an operation history queue is specified, the RAM 146 stores data representing such an operation history queue in a predetermined area under the control of the combining unit 1224.

  In step S3001, the synthesizer 1224 refers to the data stored in the RAM 146 to determine whether there is an operation history queue specified by the search in step S3000. Thereby, the composition unit 1224 determines whether or not there is an operation history queue that can be presented to the user as a part of the menu. If there is an operation history queue identified by the search in step S3000, that is, if there is an operation history queue that can be presented to the user as part of the menu (YES in step S3001), the process proceeds to step S3002. If not (NO in step S3001), the process proceeds to step S3003.

In step S3002, composition unit 1224 creates data representing a menu to be displayed on main display 150. The procedure for creating the data is the procedure from the first procedure to the fifth procedure. In the first procedure, the synthesis unit 1224 reads data for the main display 150 to display a menu, which is stored in the flash memory 144. In the second procedure, combining unit 1224 reads data representing the operation history queue stored in RAM 146 in step S3000. In the third procedure, the synthesizer 1224 selects data that defines the word order and the like by repeatedly determining the contents of the data representing the operation history queue. In the case of the present embodiment, if the natural language used for the menu is Japanese, the code representing the object is “NULL”, and the code representing the action is “110” (in the case of the present embodiment, an address book) When the application software is executed, the code representing the operation becomes “110”), the character string displayed on the main display 150 is a character string “object data edit”. Data is selected that specifies that the column continues. The natural language used for the menu is Japanese,
If the code representing the object is “NULL” and the code representing the action is “111”, the character string displayed on the main display is the character string representing the subject followed by the character string “Call”. The data that prescribes is selected. If the natural language used for the menu is Japanese and the code representing the subject is “NULL”, the character string displayed on the main display is a character string representing an object, a character “O”, and a character representing an action. Data defining a word order or the like is selected so as to be in the order of columns. The contents of the data defining these selection criteria and word order are different from those described above if the natural language used for the menu is a language other than Japanese. For example, if the natural language used for the menu is English, the code representing the object is “NULL”, and the code representing the action is “110”, the character string displayed on the main display is “edit about”. Data defining that the character string is followed by a character string representing the subject is selected. In the fourth procedure, the synthesis unit 1224 selects the data corresponding to the data representing the operation history queue stored in the RAM 146 in step S3000 among the “data representing the character string to be displayed instead of the code” included in the word database. Read out. In the fifth procedure, the synthesizer 1224 synthesizes the data read in the fourth procedure according to the data selected in the third procedure, and adds it to “data for displaying the menu”. As a result, the added data follows to some extent the natural language of the character strings included in the word database, that is, the Japanese word order and grammar in the present embodiment. Thus, if the data read in the fourth procedure is “send mail”, “Mr. A”, and “still image data”, the character string “send still image data to Ms. A” is displayed in the menu. Will be displayed as part of. When the data for displaying the menu is created, the synthesizer 1224 updates the database representing the relationship between the operation represented by the menu and the signal output from the main operation unit 120 or the sub operation unit 130. This database is stored in the flash memory 144. The synthesizer 1224 updates the database through the first procedure and the second procedure described below. In the first procedure, the composition unit 1224 includes data representing the operation history queue and the main operation unit 120 from a database representing the relationship between the operation represented by the menu and the signal output from the main operation unit 120 or the sub operation unit 130. Alternatively, the data representing the relationship with the signal output from the sub operation unit 130 is deleted. In the second procedure, the synthesis unit 1224 adds data representing the relationship between the data representing the operation history queue stored in step S3000 by the RAM 146 and the signal output from the main operation unit 120 or the sub operation unit 130. The combining unit 1224 selects which data representing the operation history queue corresponds to which signal among the signals output from the main operation unit 120 or the sub operation unit 130 according to a predetermined rule. Thereby, the database representing the relationship between the operation represented by the menu and the signal output from main operation unit 120 or sub operation unit 130 is updated. When data representing the menu is created and the database representing the relationship between the operation and the signal is updated, image control unit 1226 outputs the data representing the menu to main display 150. The main display 150 displays a menu based on the data created and output by the display control unit 1110.

  In step S3003, image control unit 1226 reads data for main display 150 to display a menu stored in flash memory 144. When the data is read, image control unit 1226 outputs the read data to main display 150. The main display 150 displays a menu based on the data read and output by the image control unit 1226.

In step S3004, main operation unit 120 or sub operation unit 130 accepts a user operation. This operation represents one of the operations included in the menu displayed on the main display 150. When the operation is accepted, operation unit 1100 searches a database representing the relationship between the operation represented by the menu and the signal output from main operation unit 120 or sub operation unit 130. As a result, the operation unit 1100 identifies an operation corresponding to the signal output by the main operation unit 120 or the sub operation unit 130 in response to a user operation. When the action is specified, the action unit 1100 determines whether or not the specified action corresponds to the action represented by the operation history queue. If it is determined that the operation specified by operation unit 1100 corresponds to the operation represented by the operation history queue (YES in step S3004), the process proceeds to step S3005. If not (NO in step S3004), the process ends.

  In step S3005, the RAM 146 stores data necessary for operation based on the operation history queue in a predetermined area under the control of the operation unit 1100.

  An operation of mobile phone 100 based on the structure and flowchart as described above will be described.

[When sending a photo after taking a photo]
FIG. 9 is a flowchart showing an example of use of mobile phone 100 according to the present embodiment. In the case of the example illustrated in FIG. 9, in step S <b> 4001, the user takes a photograph using the main camera 140 of the mobile phone 100. In step S <b> 4002, the user creates a mail addressed to “Mr. A” using the mobile phone 100. In step S 4003, the user transmits a mail addressed to “Mr. A” using mobile phone 100. The operation when the process shown in FIG. 9 is performed will be described below.

  In this case, the user first selects “capture with camera” from the menu displayed on the main display 150 in order to capture a photograph. Application software for controlling the main camera 140 and the sub camera 142 is activated (step S2000). The application software for controlling the main camera 140 and the sub camera 142 is a program for controlling the main camera 140 and the sub camera 142 based on an instruction from the user and taking a photograph.

  When the application software is activated, the first generation unit 1102 determines whether or not the application is activated in conjunction with the termination of the other application software (step S2002). If this is not the case (NO in step S2002), first generation unit 1102 stores a predetermined value at a predetermined address in RAM 146 (step S2006). When the predetermined value is stored, operation unit 1100 operates in accordance with the contents of the application software (step S2008). In this case, if no data is stored in the “predetermined area” of the RAM 146 in step S3005, the operation unit 1100 must prompt the user to input data necessary for imaging. However, in this case, since there is no data that needs to be input, if the process for prompting the input is not performed, the first generation unit 1102 uses the application software for the application to generate data for use by the user. Is determined (step S2010). In this case, since the application software for controlling the main camera 140 and the sub camera 142 is such a program (YES in step and S2010), the operation unit 1100 stores data representing the captured still image in the RAM 146. The data is output to the main display 150 (S2012).

  When the data is output, the user confirms the captured image. Thereafter, in response to a user operation on the main operation unit 120 or the sub operation unit 130, the composition unit 1224 searches for an operation history queue that can be displayed as a menu on the main display 150 or the sub display 160 (step S3000). When the operation history queue is searched, the composition unit 1224 determines whether there is an operation history queue that can be presented to the user as a part of the menu (step S3001). In this case, if there is no such operation history queue (NO in step S3001), main display 150 displays a menu based on the data read and output by image control unit 1226 (step S3003).

The user selects “send by e-mail” from among the operations displayed on the menu. When an operation is selected, the operation unit 1100 determines whether or not the selected operation is an operation represented by an operation history queue (step S3004). In this case, since the operation represented by the operation history queue should not be displayed on the menu (NO in step S3004), operation unit 1100 terminates the application software that controls main camera 140 and sub camera 142. Still image data used by application software for sending mail is already stored in the RAM 146. Thereafter, the operation unit 1100 activates application software that transmits mail (step S2016).

  When the application software for transmitting mail is activated, the second generation unit 1104 adds data representing an object as a code to the data representing the operation history queue being generated (step S2018).

  When data representing an object as a code is added, the first generation unit 1102 determines whether or not the application is activated in conjunction with the termination of other application software (step S2002). In this case, since the application software that transmits the mail is started in conjunction with the termination of the application software that controls the main camera 140 and the sub camera 142 (YES in step S2002), the second generation unit 1104 is generating Data representing the operation history queue is read (step S2004).

  When the data is read, operation unit 1100 operates according to the contents of the application software that transmits the mail (step S2008). In this case, the operation unit 1100 executes a process for prompting input of a destination mail address as part of the process based on the application software.

  Thereafter, the first generation unit 1102 determines whether the application is application software for generating data to be used by the user (step S2010). In this case, since the application is not application software for generating data to be used by the user (NO in step S2010), operation unit 1100 transmits an email (step S2020). In this case, it is assumed that a mail with still image data is transmitted to Mr. A.

  After the mail is transmitted, the second generation unit 1104 adds data representing the target person with code and data representing the action with code to the data representing the operation history queue being generated. As a result, data representing the subject “Mr. A” and data representing the operation “send mail” are added to the data representing the operation history queue being generated. The second generation unit 1104 adds the completed data to the queue database stored in the flash memory 144 (step S2022).

  Thereafter, it is assumed that application software for controlling the main camera 140 and the sub camera 142 is activated again (step S2000). In this case, through the processing from step S2002 to step S2012, the composition unit 1224 searches for an operation history queue that can be displayed as a menu on the main display 150 or the sub display 160 (step S3000). When the operation history queue is searched, the composition unit 1224 determines whether there is an operation history queue that can be presented to the user as a part of the menu (step S3001). In this case, there is an operation history queue including data representing the subject “Mr. A”, data representing the address where the object “still image data” is stored, and data representing the operation “send mail”. (YES in step S3001), main display 150 displays a menu including an operation of “sending still image data to Mr. A by mail” based on the data created and output by combining unit 1224 (step S3001). S3002).

  It is assumed that the user selects the operation “send still image data to Mr. A by mail” among the operations displayed on the menu. When an operation is selected, the operation unit 1100 determines whether or not the selected operation is an operation represented by an operation history queue (step S3004). In this case, since the selected operation is an operation represented by the operation history queue (YES in step S3004), RAM 146 stores data necessary for the operation based on the operation history queue under the control of operation unit 1100. Store in the area (step S3005). When the data is stored, the operation unit 1100 ends the application software that controls the main camera 140 and the sub camera 142. Still image data used by application software for sending mail is already stored in the RAM 146. Thereafter, the operation unit 1100 activates application software that transmits mail (step S2016).

Thereafter, the user adds the body of the mail using application software and sends the completed mail. The data indicating the address and the still image data are automatically attached.

  Since the body of an email is often different each time it is sent, the control unit 110 prompts the user to input text to be sent as email.

[When downloading downloaded photos by email using the operation history queue for sending photos by email]
Once the processing from step S2000 to step S2022 is performed, the queue database includes data representing an operation history queue for mailing still image data to Mr. A.

  Thereafter, it is assumed that application software for downloading still image data is activated via the communication unit 102 (step S2000). Further, after the processing from step S2002 to step S2010, the operation unit 1100 transmits the mail, and then ends the execution of the application software that transmits the mail (step S2020). As a result, the mail with the downloaded still image data is transmitted to Mr. A.

  As described above, the cellular phone according to the present embodiment can simplify the processing over a plurality of application software. In a conventional mobile phone or other information processing apparatus, using data generated by application software that has already been completed requires many complicated and complicated operations. The mobile phone according to the present embodiment uses the operation history queue to easily identify which data should be used among the data generated in the application software that has been terminated. Thereby, the complicated operation using a plurality of application software can be simplified. As a result, it is possible to provide an information processing apparatus, an information processing method, an information processing program, and a computer-readable recording medium on which the information processing program is recorded, which can simplify complicated operations using a plurality of applications.

  Note that the word database does not have to be included in the first modification of the present embodiment. When the word database is not included, the mobile phone 100 configures an operation history queue using information representing a mail address instead of code data.

  In the second modification example of the present embodiment, the queue database and the word database may be stored in one storage device for reference by a plurality of devices. In this case, each of the plurality of devices can access a queue database and a word database in order to process information. If necessary, these devices can process information cooperatively based on a set of queue and word databases. Examples of such multiple devices include a system in which a user owns an IC card and is authenticated by the card to access a network computer system, or schedule management in a LAN (Local Area Network) in a company There are systems and systems that provide high-level services such as department stores. In the case of these devices, the IC card can store an operation history queue and a word database. Thus, a device that communicates with the IC card can provide a high-level service to the IC card user. For example, in the case where the plurality of devices described above is a system that is authenticated by an IC card and accesses a network computer system, the user of the IC card can perform one operation included in the menu regardless of which terminal the user accesses. You can withdraw a certain amount of cash simply by selecting.

  In the third modification of the present embodiment, the first generation unit 1102 backs up the operation history queue being generated before storing a predetermined value at a predetermined address in the RAM 146 in step S2006. Processing may be performed. In this case, the data representing the operation history queue being generated is stored again at the original address when the requirement is satisfied. Thus, another operation history queue may be created during creation of the operation history queue, and then the original operation history queue may be generated.

Further, in the fourth modification example of the present embodiment, at the time of the processing in step S3002, the combining unit 1224 follows the procedure from the first procedure to the fourth procedure in the main display 1.
Data representing a menu to be displayed on 50 may be created. In the first procedure, the synthesis unit 1224 reads data for the main display 150 to display a menu, which is stored in the flash memory 144. In the second procedure, combining unit 1224 reads data representing the operation history queue stored in RAM 146 in step S3000. In the third procedure, the synthesis unit 1224 includes data corresponding to the data representing the operation history queue stored in the RAM 146 in step S3000 among the “data representing the character string to be displayed instead of the code” included in the word database. Read out. In the fourth procedure, the synthesizer 1224 adds the data read in the third procedure to “data for displaying the menu”. The data to be added is added according to the natural language of the character string included in the word database, that is, in accordance with the Japanese word order in the present embodiment. As a result, if the data to be added is “send mail”, “Mr. A”, and “still image data”, the order of these data is “Mr. A”, “still image data”, “email transmission”. Or “send image data”, “Mr. A”, and “send mail”. This process is made possible by predetermining the word order in association with the displayed word. In addition, in the addition of data, data representing a predetermined particle may be added. For example, if the order of data is “still image data”, “Mr. A”, and “send mail”, data representing “” is added between “still image data” and “Mr. A”. , Data representing “ni” is added between “Mr. A” and “send mail”.

<Second Embodiment>
Hereinafter, an information communication device according to the second embodiment of the present invention will be described.

  The hardware configuration of the information communication device according to the present embodiment is the same as that of the first embodiment described above. The function about them is the same. Therefore, detailed description thereof will not be repeated here.

  Referring to FIG. 10, the program executed on the mobile phone executes the following control with respect to generation of data for displaying the menu.

  In step S5000, combining unit 1224 refers to the data stored in RAM 146 to determine whether there is an operation history queue specified by the search in step S3000. Thereby, the composition unit 1224 determines whether or not there is an operation history queue that can be presented to the user as a part of the menu. If there is an operation history queue specified by the search in step S3000, that is, if there is an operation history queue that can be presented to the user as part of the menu (YES in step S5000), the process proceeds to step S5001. If not (NO in step S5000), the process proceeds to step S3003.

  In step S5001, the first generation unit 1102 counts the number of locations in the data stored in the RAM 146 in step S2012 that include a character string representing the file name. The first generation unit 1102 determines whether or not a character string representing the name of the file is included by comparing data in a database (not shown) representing the extension with data stored in the RAM 146. When the number of locations including the character string representing the file name is counted, the first generation unit 1102 determines whether there is only one type of data stored in the RAM 146 in step S2012 based on the number. Judging. If it is determined in step S2012 that there is only one type of data stored in RAM 146 (YES in step S5001), the process proceeds to step S3002. If not (NO in step S5001), the process proceeds to step S5002.

In step S5002, the first generation unit 1102 adds a new operation history queue to the queue database. The operation history queue to be added is an operation history queue having the same contents as the operation history queue added last. When a new operation history queue is added, the first generation unit 1102 adds new data to the word database. Newly added data is data representing an object by a code. This data represents the address of the file stored last in the data stored in the RAM 146 in step S2012, that is, the file having the largest address value. In this data, data representing a character string to be displayed instead of a code on the main display 150 or the sub display 160 represents a character string “two or more types of data”. When new data is added to the word database, the first generation unit 1102 updates the content of the operation history queue added last among the data included in the queue database. Of the operation history queue added at the end, the data to be updated is data representing an object with a code. The first generation unit 1102 replaces this data with the data last added to the word database. When the data is replaced, composition unit 1224 creates data representing a menu to be displayed on main display 150. The procedure for creating the data is the same as that in step S3002. When the data representing the menu is created, first generation unit 1102 updates the database representing the relationship between the operation represented by the menu and the signal output from main operation unit 120 or sub operation unit 130. This database is stored in the flash memory 144. The procedure for creating this database is the same as that in step S3002. When data representing the menu is created and the database representing the relationship between the operation and the signal is updated, image control unit 1226 outputs the data representing the menu to main display 150. The main display 150 displays a menu based on the data created and output by the display control unit 1110.

  An operation of mobile phone 100 based on the structure and flowchart as described above will be described. In this case, an example will be described in which a user takes a plurality of photographs together and sends them to Mr. A by e-mail.

  In this case, the user causes the control unit 110 to execute application software for controlling the camera, takes a plurality of photos, and causes the control unit 110 to execute application software for creating an email, thereby sending the photo to Mr. A. (From step S2000 to step S2012). In this series of processing, if there is an operation history queue for taking a picture and sending it to Mr. A, when the main display 150 displays the menu, the menu displays “Send still image data to Mr. A”. In addition to the above operation, an operation of “send two or more types of data to Mr. A” is displayed (step S5002). When the latter operation is selected, a plurality of photographs can be immediately transmitted to Mr. A through the processing from step S3004.

  As described above, in the mobile phone according to the present embodiment, when a plurality of data is stored, the user can select whether to transmit one type of data or all of the data. And processing across a plurality of application software can be simplified. As a result, an information processing apparatus, an information processing method, an information processing program, and a computer-readable recording medium on which the information processing program is recorded can simplify a complicated operation of handling a plurality of types of data using a plurality of applications Can be provided.

  In the above description, it is assumed that the message “Send a photo to Mr. A by email” is displayed in Japanese. However, for example, in a language that has different grammar between the singular and plural, such as English. When displaying, an operation may be displayed on the menu in consideration of these. In other words, when there is a single photo, “Send” “a photograph” “to Mr. A” “by mail” is displayed, but when multiple photos are selected, a menu is created using the operation history queue. In this case, “Send”, “photographs”, “to Mr. A”, and “by mail” can be displayed. In this case, the object “photographs” has a ending that matches the natural language, that is, English grammar.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is the 1st figure showing the structure of the mobile telephone which concerns on the 1st Embodiment of this invention. It is a 2nd figure showing the structure of the mobile telephone based on the 1st Embodiment of this invention. It is the 3rd figure showing the structure of the mobile telephone based on the 1st Embodiment of this invention. It is a functional block diagram of the control part which concerns on the 1st Embodiment of this invention. It is a figure showing the structure of the queue database which concerns on the 1st Embodiment of this invention. It is a figure showing the structure of the word database which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the procedure of control of the production | generation process of the operation history queue which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the procedure of control of the production | generation process of the data for displaying the menu which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the example of the use procedure of the mobile telephone which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the procedure of control of the production | generation process of the data for displaying the menu which concerns on the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Mobile phone, 102 Communication part, 104 1st housing | casing, 106 2nd housing | casing, 108 Antenna, 110 Control part, 120 Main operation part, 121 Up / down movement key, 122
Cursor move key, 124 12 keys, 130 sub operation unit, 140 main camera, 142 sub camera, 144 flash memory, 146 RAM, 148 data ROM, 150 main display, 160 sub display, 170 audio signal processing unit, 172 microphone, 174 speaker, 176 LED, 178 data communication IF, 180 memory card drive unit, 182 memory card, 184 vibrator, 1100 operation unit, 1102 first generation unit, 1104 second generation unit, 1106 third generation unit, 1108 storage control unit 1110 Display control unit, 1120 1st control unit, 1122 2nd control unit, 1224 composition unit, 1226 image control unit, 1302, 1306 cursor movement key, 1304 determination key.

Claims (9)

First application software that is application software for generating data, second application software that is application software for processing the data, at least one of the first application software and the second application software Storage means for storing a plurality of types of information each representing an operation realized by one side;
Display means for displaying an image;
An input means for inputting information corresponding to any of the plurality of types of information;
Control means for operating according to the first application software and the second application software, and for controlling the display means,
The control means includes
Operating means for operating according to at least one of the first application software and the second application software so as to correspond to the information input to the input means;
First generating means for generating information representing the data generated by the operating means when the operating means operates in accordance with the first application software;
Second generating means for generating information representing the content of processing on the data when the operating means processes the data according to the second application software;
A third generation unit for generating information representing an operation performed by the operation unit based on the information generated by the first generation unit and the information generated by the second generation unit;
Storage control means for controlling the storage means so as to store the information generated by the third generation means as a kind of the plurality of types of information;
An information processing apparatus comprising: display control means for controlling the display means so as to display the images corresponding to the plurality of types of information.   The information processing apparatus according to claim 1, wherein the third generation unit includes a combination unit for combining the information generated by the first generation unit and the information generated by the second generation unit. The combination means includes
First control means for controlling the storage means to store information generated by the first generation means;
And a second control unit for controlling the storage unit so as to store the information generated by the second generation unit in association with the information generated by the first generation unit. The information processing apparatus described. The storage means includes, in addition to the first application software, the second application software, and the plurality of types of information, character string information corresponding to the data, and character string information representing the contents of the processing And remember
The display control unit is configured to display the image corresponding to the character string information corresponding to the data, the character string information representing the contents of the processing, and the plurality of types of information. The information processing apparatus according to claim 1, further comprising means for controlling The display control means includes
Synthesizing means for synthesizing character string information having a character string corresponding to the data and a character string representing the content of the processing based on a combination of information representing the data and information representing the content of the processing When,
The information processing apparatus according to claim 4, further comprising: means for controlling the display means so as to display the image representing the information synthesized by the synthesis means.   The information processing apparatus according to claim 5, wherein the character string corresponding to the data includes a character string representing a word corresponding to the data. A first generation step of generating information representing the data when operating according to first application software which is application software for generating data;
A second generation step of generating information representing the content of the processing for the data when operating according to second application software which is application software for processing the data;
A third generation step for generating information representing an operation based on the information generated in the first generation step and the information generated in the second generation step;
A storage step of storing the information generated in the third generation step as one of a plurality of types of information each representing the operation;
A display step for displaying images corresponding to the plurality of types of information;
An input step of inputting information corresponding to any of the plurality of types of information;
An information processing method comprising: an operation step that operates according to at least one of the first application software and the second application software so as to correspond to the information input in the input step. A first generation step of generating information representing the data when operating according to first application software which is application software for generating data;
A second generation step of generating information representing the content of the processing for the data when operating according to second application software which is application software for processing the data;
A third generation step for generating information representing an operation based on the information generated in the first generation step and the information generated in the second generation step;
A storage step of storing the information generated in the third generation step as one of a plurality of types of information each representing the operation;
A display step for displaying images corresponding to the plurality of types of information;
An input step of inputting information corresponding to any of the plurality of types of information;
An information processing program that causes a computer to execute each step including an operation step that operates according to at least one of the first application software and the second application software so as to correspond to the information input in the input step. A computer-readable recording medium recording an information processing program,
A first generation step of generating information representing the data when operating according to first application software which is application software for generating data;
A second generation step of generating information representing the content of the processing for the data when operating according to second application software which is application software for processing the data;
A third generation step for generating information representing an operation based on the information generated in the first generation step and the information generated in the second generation step;
A storage step of storing the information generated in the third generation step as one of a plurality of types of information each representing the operation;
A display step for displaying images corresponding to the plurality of types of information;
An input step of inputting information corresponding to any of the plurality of types of information;
An information processing program for causing a computer to execute each step including an operation step that operates according to at least one of the first application software and the second application software so as to correspond to the information input in the input step Computer-readable recording medium.

Images